10. Time series regression and forecasting

Transcription

1 10. Time series regression and forecasting Key feature of this section: Analysis of data on a single entity observed at multiple points in time (time series data) Typical research questions: What is the causal effect on a variable of interest, Y, of a change in another variable, X, over time? What is the best forecast of the value of an economic variable (e.g. the stock price) at some future date? 282

2 Remarks: The analysis of time series data requires knowledge on several specific concepts Some of these key concepts (to be explained) are forecasting estimation of dynamic causal effects stationarity (non-stationarity) Aim of this section: Description of these basic concepts (For detailed information refer to the many special lectures on this topic) 283

3 10.1. Time series data and serial correlation Point of departure: We consider an economic / financial variable over time (for example, the inflation rate or the unemployment rate) We denote the observation on that variable at date t by Y t We denote the number of all observations by T The period of time between observations (i.e. between observation t and t + 1) is some unit of time (a day, week, month, quarter,...) In a first step, we always plot the observations on the variable over time 284

4 Inflation and unemployment in the US, (quarterly data) 285

5 Definition 10.1: (Lags, first differences) We consider a time series variable Y t observed through time (t = 1,..., T ). We use the following notation: The first lag of a time series Y t is Y t 1. The jth lag is Y t j. The first difference of a series, Y t, is its change between periods t 1 and t: Y t = Y t Y t 1. The first difference of the logarithm of Y t is ln(y t ) = ln(y t ) ln(y t 1 ). 286

6 Remarks: The percentage change of a time series Y t between periods t 1 and t is approximately 100 ln(y t ) This approximation is most accurate when the percentage change is small (see Slide 92) 287

7 Autocorrelation: An important issue in the analysis of time series data is whether and how a variable Y t is related to its own past values As a measure of this relation, we use the covariance and the correlation of Y t with its own past values Autocovariance and autocorrelation (serial correlation) Definition 10.2: (Autocovariance, autocorrelation) The jth autocovariance (autocorrelation) of a series Y t is the covariance (correlation coefficient) between Y t and its jth lag: jth autocovariance γ j = Cov(Y t, Y t j ), jth autocorrelation ρ j = Cov(Y t, Y t j ) Var(Yt ) Var(Y t j ). 288

8 Remarks: The formulas in Definition 10.2 represent the probabilistic autocovariances and autocorrelations when we think of the series Y t as a sequence of random variables (jth population autocovariances and autocorrelations) implicitly assume that the population autocovariances γ j and autocorrelations ρ j remain constant over time, for example in the case of j = 1 γ 1 = Cov(Y 2, Y 1 ) = Cov(Y 3, Y 2 ) =... ρ 1 = Corr(Y 2, Y 1 ) = Corr(Y 3, Y 2 ) =... Assumption of stationarity (to be discussed later) 289

9 Question: How can we estimate the theoretical population autocovariances γ j and autocorrelations ρ j on the basis of the time series observations Y 1, Y 2,..., Y T? Definition 10.3: (Estimating autocovariances, autocorrelations) The conventional estimators of the jth autocovariance γ j and the jth autocorrelation ρ j based on the observations Y 1, Y 2,..., Y T are defined as follows: ˆγ j = 1 T T t=j+1 ( Yt Ȳ ) ( Y t j Ȳ ), (10.1) ˆρ j = ˆγ j ˆγ 0, (10.2) where Ȳ denotes the sampling mean of the Y t. 290

10 Remarks: The estimators (10.1) and (10.2) are consistent Both estimators implicitly assume stationarity Example: Sample autocorrelations of the U.S. CPI inflation rate and its changes up to lag 10 (see next slides) We denote the U.S. CPI inflation rate in EViews by INF t INF t itself exhibits a strongly positive autocorrelation Its first difference, INF t, exhibits a strongly negative autocorrelation 291

11 U.S. CPI inflation rate and its changes U.S. CPI inflation rate First differences in the U.S. CPI inflation rate

12 Sample autocorrelations of the U.S. CPI inflation rate and its changes Variable: INF Date: 25/06/12 Time: 11:21 Sample: 1957Q1 2005Q1 Included observations: 192 Autocorrelation Partial Correlation AC PAC Q-Stat Prob. ******. ****** *****. * ******. ** ***** *.. ****... ****... ****... *** * ***. * ***. * Variable: D(INF) Date: 25/06/12 Time: 11:22 Sample: 1957Q1 2005Q1 Included observations: 191 Autocorrelation Partial Correlation AC PAC Q-Stat Prob **. ** **. *** **. * * * * *. * **. * * * *

13 Four economic time series 294

14 10.2. Autoregressions Aim of this section: Forecasts made using a regression model that relates a time series variable to its own past values Definition 10.4: (First order autoregressive model) We consider an economic time series variable Y t and define the first order autoregressive population model (the AR(1) model) for Y t as where u t is an error term. Y t = β 0 + β 1 Y t 1 + u t, (10.3) 295

15 Example: Fit of an AR(1) model to the change in the U.S. CPI inflation rate EViews output: Fit of an AR(1) model to INF t Dependent Variable: D(INF) Method: Least Squares Date: 28/06/12 Time: 10:29 Sample (adjusted): 1957Q4 2004Q4 Included observations: 189 after adjustments Convergence achieved after 3 iterations Variable Coefficient Std. Error t-statistic Prob. C AR(1) R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood F-statistic Prob(F-statistic) tatis Inverted AR Roots Mean dependent var S.D. dependent var Akaike info criterion Schwarz criterion Hannan-Quinn criter Durbin-Watson stat

16 Forecasting: Consider the AR(1) model in Eq. (10.3) We estimate the unknown parameters β 0 and β 1 by OLS using the data Y 1,..., Y T We obtain the OLS estimates ˆβ 0 and ˆβ 1 We aim at forecasting the future value Y T +1 based on the observed value Y T and the estimated coefficients ˆβ 0 and ˆβ 1 We denote this forecast by Ŷ T +1 T and it is given by Ŷ T +1 T = ˆβ 0 + ˆβ 1 Y T (10.4) The forecast error is the difference between the value of Y T +1 that actually occurs and its forecasted value based on Y T : Forecast error = Y T +1 Ŷ T +1 T (10.5) 297

17 Forecasting: [continued] Forecasts versus predicted values Forecasts pertain to out-of-sample observations Predicted values pertain to in-sample observations The root mean squared forecast error (RMSFE) is a measure of the magnitude of a typical mistake using a forecasting model and is defined by RMSFE = The RMSFE has two sources: The unknown future values of u t The errors in the estimates ˆβ 0 and ˆβ 1 E [ Y T +1 Ŷ T +1 T ] 2 (10.6) 298

18 Forecasting: [continued] The RMSFE can be estimated by the standard error of the regression (to be discussed in Section 10.3.) Example: We consider the estimation output on Slide 296 The estimated AR(1) model for the changes in INF t is INF t = INF t 1 (10.7) The estimation period is 1957:Q1 2004:Q4 We aim at forecasting the inflation rate for 2005:Q1 299

19 Example: [continued] In general, we have ÎNF T +1 T = INF T + ÎNF T +1 T INF T = INF T + INF T +1 T (10.8) Setting T = 2004:Q4, we find from the data set From Eq. (10.7), we have INF 2004:Q4 = % (10.9) INF 2004:Q4 = % (10.10) INF 2005:Q1 = INF 2004:Q4 = % (10.11) 300

20 Example: [continued] It follows from the Eqs. (10.8) (10.11) that ÎNF 2005:Q1 2004:Q4 = = % Accuracy of the forecast: From the data set we find that INF 2005:Q1 = % The forecast error is INF 2005:Q1 ÎNF 2005:Q1 2004:Q4 = = % 301

21 Next: Extension of the AR(1) model by including potentially useful information in more distant past values of the time series Definition 10.5: (pth-order autoregressive model) The pth-order autoregressive model (the AR(p) model) represents Y t as a linear function of p of its lagged values: Y t = β 0 + β 1 Y t 1 + β 2 Y t β p Y t p + u t, (10.12) where E(u t Y t 1, Y t 2,...) = 0. The number of lags p is called the order, or the lag length, of the autoregression. 302

22 Implications of the assumption E(u t Y t 1, Y t 2,...) = 0: 1. The best forecast of Y T +1 based on its entire history depends only on the most recent p past values It can be shown that if Y t follows an AR(p) model, then the best forecast (in the sense of having smallest RMSFE) of Y T +1 based on Y T, Y T 1,... is Y T +1 T = β 0 +β 1 Y T +β 2 Y T β p Y T p+1 (10.13) Since the coefficients β 0,..., β p are unknown, we use the forecast from Eq. (10.13) with estimated coefficients 2. The errors u t are serially uncorrelated 303

23 Fit of an AR(4) model to INF t Dependent Variable: D(INF) Method: Least Squares Date: 28/06/12 Time: 10:30 Sample (adjusted): 1958Q3 2004Q4 Included observations: 186 after adjustments Convergence achieved after 3 iterations Variable Coefficient Std. Error t-statistic Prob. C AR(1) AR(2) AR(3) AR(4) R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood F-statistic Prob(F-statistic) tatis Mean dependent var S.D. dependent var Akaike info criterion Schwarz criterion Hannan-Quinn criter Durbin-Watson stat Inverted AR Roots i i i i 304

24 Estimation results: Estimated equation: INF t = INF t INF t 2 (0.0758) (0.0741) (0.0762) INF t INF t 4 (10.14) (0.0764) (0.0744) The coefficients on INF t 2, INF t 3, INF t 4 are jointly statistically different from zero (F -statistic = , p-value < ) R 2 improves from for the AR(1) model on Slide 296 to SER improves from for the AR(1) model on Slide 296 to

25 Inflation forecast for 2005:Q1: Recall Eq. (10.8) on Slide 300 with T = 2004:Q4 ÎNF T +1 T = INF T + From the data set we have INF T +1 T INF 2004:Q4 = INF 2004:Q4 = , INF 2004:Q3 = INF 2004:Q2 = , INF 2004:Q1 = Using the estimates in Eq. (10.14) on Slide 305, we obtain INF 2005:Q1 2004:Q4 = ( ) =

26 Inflation forecast for 2005:Q1: [continued] From Eq. (10.8) we thus have ÎNF 2005:Q1 2004:Q4 = = % Accuracy of the forecast: From the data set we find that INF 2005:Q1 = % The forecast error is INF 2005:Q1 ÎNF 2005:Q1 2004:Q4 = =

27 Obviously: Surprisingly, the AR(4) forecast error (-1.441) is larger in absolute value than the AR(1) forecast error ( ) (to be explained in Section 10.3.) 308

28 10.3. Time series regression with additional predictors and the autoregressive distributed lag model Next: Other variables than past values of the Y -variable may help to forecast the variable of interest These variables (called predictors) should be included on the right-hand side of the autoregression Eq. (10.12) on Slide 302 Autoregressive distributed lag models Example: Forecasting changes in the inflation rate using past unemployment rates (short-run Phillips curve) We denote the unemployment rate in EViews by UNEMP 309

29 Change in the U.S. CPI inflation rate between year t and year t + 1 versus the unemployment rate in year t 310

30 Fit of an AR(4) model plus UNEMP t 1 to INF t Dependent Variable: D(INF) Method: Least Squares Date: 28/06/12 Time: 10:31 Sample (adjusted): 1958Q3 2004Q4 Included observations: 186 after adjustments Convergence achieved after 9 iterations Variable Coefficient Std. Error t-statistic Prob. C UNEMP(-1) AR(1) AR(2) AR(3) AR(4) R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood F-statistic Prob(F-statistic) tatis Mean dependent var S.D. dependent var Akaike info criterion Schwarz criterion Hannan-Quinn criter Durbin-Watson stat Inverted AR Roots i i i i 311

31 Estimation results: Lagged predictor UNEMP t 1 is significant at the 5% level Improvement of the R 2 from for the pure AR(4) model to Using the estimates from Slide 311 and the data set including the observations for UNEMP, we compute the inflation forecast for 2005:Q1 as The forecast error is ÎNF 2005:Q1 2004:Q4 = % INF 2005:Q1 ÎNF 2005:Q1 2004:Q4 = =

32 Fit of an AR(4) model plus (UNEMP t 1,..., UNEMP t 4 ) to INF t Dependent Variable: D(INF) Method: Least Squares Date: 28/06/12 Time: 10:39 Sample (adjusted): 1959Q1 2004Q4 Included observations: 184 after adjustments Convergence achieved after 11 iterations Variable Coefficient Std. Error t-statistic Prob. C UNEMP(-1) UNEMP(-2) UNEMP(-3) UNEMP(-4) AR(1) AR(2) AR(3) AR(4) R-squared Adjusted R-squared S.E. of regression Sum squared resid Log likelihood F-statistic Prob(F-statistic) tatis Mean dependent var S.D. dependent var Akaike info criterion Schwarz criterion Hannan-Quinn criter Durbin-Watson stat Inverted AR Roots i i i i 313

33 Estimation results: Predictor lags UNEMP t 1, UNEMP t 2, UNEMP t 3 are individually significant at the 1% level, UNEMP t 4 at the 10% level Substantial improvement of the R 2 from to Using the estimates from Slide 313 and the data set including the observations for UNEMP, we compute the inflation forecast for 2005:Q1 as The forecast error is ÎNF 2005:Q1 2004:Q4 = % INF 2005:Q1 ÎNF 2005:Q1 2004:Q4 = =

34 Now: Formal definition of this autoregressive model including one additional predictor Definition 10.6: (Autoregressive distributed lag model) The autoregressive distributed lag model with p lags of Y t and q lags of the predictor X t, denoted by ADL(p, q), is Y t = β 0 + β 1 Y t 1 + β 2 Y t β p Y t p δ 1 X t 1 + δ 2 X t δ q X t q + u t, (10.15) where β 0, β 1,..., β p, δ 1,..., δ q are unknown coefficients and u t is the error term with E(u t Y t 1, Y t 2,..., X t 1, X t 2,...) =

35 Remarks: The assumption E(u t Y t 1, Y t 2,..., X t 1, X t 2,...) = 0 implies that no additional lags of either Y or X belong in the ADL model (the lag lengths p and q are the true lag lengths) The ADL model contains lags of the dependent variable (autoregressive component) a distributed lag of a single additional predictor X In general, forecasts can be improved by using multiple predictors 316

36 Stationarity: Forecasting future values of a time series Y t based on past relationships implicitly require that the relationships remain stable over time Concept of stationarity Definition 10.7: (Stationarity) A time series Y t is stationary if its probability distribution does not change over time, that is, if the joint distribution (Y s+1, Y s+2,..., Y s+t ) does not depend on s regardless of the value of T ; otherwise, Y t is said to be nonstationary. A pair of time series, X t and Y t, are said to be jointly stationary, if the joint distribution (X s+1, Y s+1, X s+2, Y s+2,..., X s+t, Y s+t ) does not depend on s regardless of the value of T. Stationarity requires the future to be like the past, at least in a probabilistic sense. 317

37 Definition 10.8: (Time series regression with multiple predictors) The general times series regression model allows for k additonal predictors X 1,... X k with q 1 included lags of X 1, q 2 included lags for X 2, and so forth: where Y t = β 0 + β 1 Y t 1 + β 2 Y t β p Y t p + δ 11 X 1t 1 + δ 12 X 1t δ 1q1 X 1t q1 (10.16) δ k1 X kt 1 + δ k2 X kt δ kqk X kt qk + u t, 1. E(u t Y t 1, Y t 2,..., X 1t 1, X 1t 2,..., X kt 1, X kt 2,...) = The random variables (Y t, X 1t,..., X kt ) have a stationary distribution. (Y t, X 1t,..., X kt ) and (Y t j, X 1t j,..., X kt j ) become independent as j gets large. 318

38 Definition 10.8: [continued] 3. Large outliers are unlikely: X 1t,..., X kt and Y t have nonzero, finite fourth moments. 4. There is no perfect multicollinearity. Remarks: The first part of Assumption #2 requires that the distribution of the data today is the same as its distribution in the past The second part of Assumption #2 requires that the random variables become independently distributed when the amount of time separating them becomes large Both parts replace the cross-sectional OLS Assumption #2 on Slide

39 Statistical inference: Given the assumptions in Definition 10.8, we can apply OLS in the ususal way to make inference on the regression coefficients We can use the F -statistic to test whether the lags of one of the included regressors have useful predictive content Granger causality tests Definition 10.9: (Granger causality test) The Granger causality statistic is the F -statistic testing the hypothesis that the coefficients on all the values of one of the variables in Eq. (10.16) are simultaneously equal to zero (for example, the coefficients on X 1t 1, X 1t 2,... X 1t q1 ). This null hypothesis implies that these regressors have no predictive content for Y t beyond that contained in the other regressors. 320

40 Remarks: Granger causality means that if X Granger-causes Y, then X is a useful predictor of Y, given the other variables in the regression A more accurate phrasing than Granger causality would be Granger predictability Example: Consider the relationship between INF t and its past values and past values of UNEMP on Slide 313 Test the null hypothesis on the UNEMP coefficients H 0 : δ 11 = 0, δ 12 = 0, δ 13 = 0, δ 14 = 0 321

41 Example: [continued] F -statistic: , p-value < UNEMP appears to contain information that is useful for forecasting the change in the inflation rate Forecast uncertainty: We consider the RMSFE defined in Eq. (10.6) on Slide 298 as a measure of uncertainty of a forecast In general, the RMSFE consists of two components: uncertainty arising from the estimation of the regression coefficients uncertainty about the future unknown value of u t 322

42 Example: Consider an ADL(1, 1) model with a single predictor: Y t = β 0 + β 1 Y t 1 + δ 1 X t 1 + u t Assume further that u t is homoskedastic The forecast of Y T +1 is Ŷ T +1 T = ˆβ 0 + ˆβ 1 Y T + ˆδ 1 X T The forecast error is Y T +1 Ŷ T +1 T = u T +1 [ ] (ˆβ 0 β 0 ) + (ˆβ 1 β 1 )Y T + (ˆδ 1 δ 1 )X T (10.17) 323

43 Example: [continued] Since u t is homoskedastic, u T +1 has variance σu 2 and it can be shown that [ (YT ) ] 2 MSFE = E +1 Ŷ T +1 T = σ 2 u + Var [ (ˆβ 0 β 0 ) + (ˆβ 1 β 1 )Y T + (ˆδ 1 δ 1 )X T ], (10.18) so that RMSFE = MSFE Forecast uncertainty: [continued] The term σu 2 appearing in Eq. (10.18) can be estimated by the standard error of the regression defined on Slide 21: ˆσ 2 u = SER 2 = 1 T 3 T ût 2 t=1 324

44 Forecast uncertainty: [continued] The second term appearing on the right-hand side of Eq. (10.18) can be estimated by specific statistical techniques (not to be discussed here) Adding the two latter estimates yield the estimate RMSFE is (approxi- In practice, a 95% forecast interval of Y T +1 mately) given by Ŷ T +1 T ± 1.96 RMSFE Eq. (10.18) illuminates that highly parameterized models may produce larger forecast errors than parsimoneous models (see our previous examples) 325

45 10.4. Lag length selection using information criteria Important practical issue: How many lags should be included in a time series regression? Aim of this section: Presentation of statistical methods for choosing the number of lags in an autoregression a time series regression with multiple predictors 326

46 Determining the order of an autoregression Potential trade-off: If the order p of an estimated autoregression is too low, we omit potentially valuable information contained in the more distant lagged values too high, we estimate more coefficients than necessary thus introducing additional estimation error into our forecasts Two approaches: t-statistic approach Use of information criteria 327

47 t-statistic approach: Consider a model with many lags (that is with a high value of p) and perform hypothesis tests on the final lag Example: Start by estimating an AR(6) model and test whether the coefficient on the sixth lag is significant at the 5% level if not, drop it and estimate an AR(5) model, test the significance of the fifth lag, and so forth Remarks: The t-statistic approach has the tendency to produce too large a model 328

48 Remarks: [continued] Reasoning: Even if the true AR order is 5 (that is β 6 = 0), a t-test at the 5% level will incorrectly reject the null hypothesis 5% of the time by chance H 0 : β 6 = 0 When the true value of p is five, this approach will estimate p to be six 5% of the time Some textbooks suggest the t-statistic approach starting with the order p = 0 and then successively including AR terms whenever the t-statistic indicates significance at the 5% level (modeling from small to large) This is not a recommended procedure because of potential omitted-variable bias 329

49 Information criteria: Information criteria are measures reflecting the trade-off described on Slide 327 in the selection of the order p in an autoregression We can estimate the order p of an autoregression by minimizing such an information criterion The most popular criteria are the Schwarz (SIC) and the Akaike (AIC) information criteria Both, the SIC and the AIC, are based on the sum of squared residuals (SSR) of the AR(p) model estimated by OLS: SSR(p) = (see Slide 22) T t=1 û 2 t = T t=1 ( Yt ˆβ 0 ˆβ 1 Y t 1... ˆβ p Y t p ) 2 330

50 Definition 10.10: (Schwarz, Akaike information criteria) The Schwarz (SIC) and the Akaike (AIC) information criteria of an AR(p) model estimated by OLS are respectively defined as [ ] SSR(p) SIC(p) = ln + (p + 1) ln(t ) T T, (10.19) [ ] SSR(p) AIC(p) = ln + (p + 1) 2 T T. (10.20) The SIC and the AIC estimators of p are the values that respectively minimize SIC(p) and AIC(p) among the possible choices p = 0, 1,..., p max, where p max is the largest value of p considered and p = 0 corresponds to the model that contains only an intercept. 331

51 Information criteria: [continued] The first term on the right-hand side of the Eqs. (10.19) and (10.20) necessarily decrease when adding a lag to the autoregression The respective second terms, (p+1) ln(t )/T and (p+1) 2/T, necessarily increase when adding a lag to the autoregression (terms punishing the inclusion of further lags) The two terms in the Eqs. (10.19) and (10.20) reflect the trade-off Both information criteria are routinely computed by EViews (see the outputs on Slides 296, 304) 332

52 Information criteria: [continued] It can be proved that the SIC estimator of p is consistent the AIC estimator of p is not consistent (AIC overestimates p with nonzero probability) The Schwarz (SIC) and Akaike (AIC) information criteria of distinct AR(p) models for INF t, 1958:Q4 2005:Q1 (T = 186) AR order p SIC(p) AIC(p)

53 Remark: The SIC and AIC estimates of p should be determined by running all autoregressions involved over the same sampling period (thus, using the same number of observations) Example: Our U.S. inflation dataset originally covers the sampling period 1957:Q1 2005:Q1 (T = 193) Since we estimate, inter alia, an AR(5) model involving the highest lag INF t 5 the feasible sampling period adjusts to 1958:Q4 2005:Q1 (T = 186) It is this period, 1958:Q4 2005:Q1 with T = 186, that should also be used to compute the SIC and AIC values in all other autoregressions with p 4 334

54 Lag length selection in time series regression with multiple predictors Potential trade-off here: The choice of the number of predictors plus the corresponding lag numbers must balance the benefit of using additional information the cost of estimating additional coefficients Two approaches: F -statistic approach Information criteria 335

55 F -statistic approach: Use the F -statistic to test joint hypotheses that sets of coefficients are simultaneously equal to zero Example: Consider Eq. (10.16) on Slide 318 Use the F -statistic to test the null hypothesis H 0 : δ 11 = 0, δ 12 = 0,..., δ 1q1 = 0 (the predictor X 1 has no predictive content) Similar to the t-statistic approach, the F -statistic approach has the tendency to produce too large models 336

56 Information criteria: Consider the general time series regression model with multiple predictors defined in Eq. (10.16) on Slide 318 Denote the number of all coefficients (including the intercept) by K The SIC and AIC information criteria are then modified as [ ] SSR(K) SIC(K) = ln + K ln(t ) T T [ ] SSR(K) AIC(K) = ln + K 2 T T Evaluate the SIC (or AIC) for each candidate model The model with the minimal value of SIC (or AIC) is the preferred model 337

57 Information criteria: [continued] Two important practical considerations: Again, all candidate models must be estimated over the same sampling period (see Slide 334) When there are multiple predictors, the approach gets computationally demanding since it requires computing many different models (many combinations of the lag parameters) Convenient shortcut: Require all the regressors to have the same number of lags, that is, require that p = q 1 =... = q k, so that only p max + 1 models need to be compared (corresponding to p = 0, 1,..., p max ) 338

58 Thank you for your attention! 339