FORECASTING AND MODEL SELECTION

Transcription

1 FORECASTING AND MODEL SELECTION Anurag Prasad Department of Mathematics and Statistics Indian Institute of Technology Kanpur, India REACH Symposium, March 15-18, Forecasting and Model Selection

2 Outline Modeling and Forecasting 1 Modeling and Forecasting Forecasting and Model Selection

3 Assumptions of Forecasting 1 Element of Uncertainty 2 Blind Spots 3 Change in Forecast Accuracy 3 Forecasting and Model Selection

4 Framework of a Forecast System Model Building Phase Forecasting Phase Theory and/or Historical Data Model Specification Model Estimation No Diagnostic Checking Yes Forecast Generation No Stability Checking Yes Forecast Updation Data New Observations 4 Forecasting and Model Selection

5 Choice of a Particular Forecast Model 1 Degree of Accuracy Required 2 Cost of Producing Forecasts 3 Forecast Horizon 4 Degree of Complexity Required 5 Available Data 5 Forecasting and Model Selection

6 Classification of Estimation Methods 1 Time Series Methods 2 Causal Methods 3 Judgemental Methods 6 Forecasting and Model Selection

7 Time Series Methods Use historical data as a basis Underlying patterns are fairly stable 1 Autoregressive Moving Average (ARMA) 2 Exponential Smoothing 3 Extrapolation 4 Linear Prediction 5 Trend Estimation 6 Growth Curve 7 Box-Jenkins Approach 7 Forecasting and Model Selection

8 Causal Methods Belief that some other time series can be useful Assumption that it is possible to identify the underlying factors 1 Regression Analysis * Linear Regression * Non-Linear Regression 2 Econometrics 8 Forecasting and Model Selection

9 Judgemental Methods Incorporate intuitive judgements, opinions and probability estimates 1 Composite Forecasts 2 Surveys 3 Delphi Method 4 Scenario Building 5 Technology Forecasting 6 Forecast by Analogy 9 Forecasting and Model Selection

10 Forecast Error For t = 1,..., N, y(t) : Actual value at period t, ŷ(t) : Forecast value at period t; e(t) : Forecast error at period t; e(t) = y(t) ŷ(t) y(t) y(t ^ ) i y(t ) i t t i 10 Forecasting and Model Selection

11 Graphical Measures of Forecast Accuracy Plot of y(t) versus ŷ(t) Keep the same scale for both the axes. Departure of points from the 45 0 line through origin indicates imperfect forecasts. 11 Forecasting and Model Selection

12 y(t) y(t) ^y(t) Correct Model Form ^y(t) Incorrect Model Form 12 Forecasting and Model Selection

13 Plot of e(t) versus t Reveals patterns of variability which the model has failed to explain. For a good model, the forecast errors should vary in a horizontal band around zero. 13 Forecasting and Model Selection

14 e(t) 0 e(t) 0 t Correct Model Form t Incorrect Model Form e(t) 0 e(t) 0 t Incorrect Model Form t Incorrect Model Form 14 Forecasting and Model Selection

15 Descriptive Measures of Forecast Accuracy Descriptive Measures of Forecast Accuracy are used to... 1 Provide a single, easily interpreted measure of model s reliability 2 Compare the accuracy of two different models 3 Search for an optimal model 4 Monitor a model s performance 15 Forecasting and Model Selection

16 Descriptive Measures of Forecast Accuracy Mean Absolute Deviation (MAD) MAD = forecast error number of forecasts = N t=1 e(t) N Mean Square Error (MSE) MSE = (forecast error)2 number of forecasts = N t=1 e(t)2 N Root Mean Square Error (RMSE) RMSE = MSE 16 Forecasting and Model Selection

17 Descriptive Measures of Forecast Accuracy Mean Absolute Percent Error (MAPE) MAPE = = N t=1 e(t)/y(t) N.100% forecast error/actual value number of forecasts.100% Pearson s Correlation Coefficient (r) between y(t) and ŷ(t) r = N (y(t) ȳ)( y(t) t=1 y) N t=1 (y(t) ȳ)2 N ( y(t) t=1 y) 2 17 Forecasting and Model Selection

18 Descriptive Measures of Forecast Accuracy "No Change" model is : ŷ(t + 1) = y(t) Theil s Inequality Coefficient (U) U = RMSE("new" model) RMSE("no change" model) U > 1 worse than "no change" model U = 1 as good as "no change" model U < 1 better than "no change" model 18 Forecasting and Model Selection

19 Further Readings Quantitative, N.R. Farnum and L.W. Stanton, 1989, PWS-KENT Publishing Co. Statistical Methods for Forecasting, B. Abraham and J. Ledolter, 1983, John Wiley & Sons Introduction to Time Series and Forecasting, P.J. Brockwell and R.A. Davis, 2002 (Second Edition), Springer-Verlag Time Series Analysis and Forecasting, R. Yaffee, 2000, Academic Press 19 Forecasting and Model Selection

20 Thank You 20 Forecasting and Model Selection