Forecasting and Estimation

Transcription

1 February 3, 2009

2 Forecasting I Very frequently the goal of estimating time series is to provide forecasts of future values. This typically means you treat the data di erently than if you were simply tting a model. A crucial step in choosing your forecast is to de ne a loss function. Examples are: Minimizing the absolute di erence between the forecast and the eventual value. Minimizing the square di erence. Asymmetric loss functions which penalize too high forecasts more than too low (GDP, state budgets). This is a potential topic for end-of-semester presentation. Many other options for loss functions.

3 Forecasting II It is typically not the case that the model that provides the best t is the best model for forecasting. Example: Consider the following ARMA(2,1) model: y t = µ + φ 1 y t 1 + φ 2 y t 2 + ε t + θε t 1 If we knew the true model, we would forecast y t+1 at time t as y t+1jt = µ + φ 1 y t + φ 2 y t 1 + θε t We d make the mistake of: y t+1 y t+1jt = µ + φ 1 y t + φ 2 y t 1 + ε t+1 + θε t (µ + φ 1 y t + φ 2 y t 1 + θε t ) = ε t+1

4 Forecasting III Only the inherently unpredictable white noise error. In reality, however, we have to use y t+1jt = ˆµ + ˆφ 1 y t + ˆφ 2 y t 1 + ˆθˆε t The mistake we make is: y t+1 y t+1jt = µ + φ 1 y t + φ 2 y t 1 + ε t+1 + θε t ˆµ + ˆφ 1 y t + ˆφ 2 y t 1 + ˆθˆε t = ε t+1 + (µ ˆµ) + φ 1 ˆφ 1 yt + φ 2 ˆφ 2 yt 1 + θ ˆθ ε t + (ε t ˆε t ) ˆθ

5 Forecasting IV Clearly the estimation of the parameters and the residuals add to the error we make. We cannot do much about the residuals. It is possible that approximating the model by an AR(1) or and MA(1) may lead to more accurate forecast. For goodness of t, this does not happen. There the "true" model always gives us the best result.

6 Forecasts with MSE loss I More generally we might have more regressors than just lagged values of y. Let X t be the regressors, possibly containing lagged values of y. The most commonly used loss function would be the quadratic loss function or the Mean Squared Error associated with the forecast: MSE y t+1jt = E y t+1 y t+1jt Once a loss function is determined, the forecast is chosen to minimize this function. Result The forecast which minimizes the MSE loss function is y t+1jt = E (y t+1jx t ) 2

7 Forecasts with MSE loss II In general this is as much as we can say. We can make more progress if we restrict out attention to linear forecasts. That is: y t+1jt = α0 X t Result The linear forecast which minimizes the MSE loss among all possible linear forecasts is the projection of y t+1 onto X t : α 0 = E [y t+1 X t ] E X 0 1 t X t, α = E X 0 t X t 1 E X 0 t y t+1

8 Forecasts with MSE loss III and y t+1jt = α0 X t = E [y t+1 X t ] E X 0 t X t 1 X t Note that this expression is closely connected to the OLS estimator from the regression: where y t+1 = βx t + ε t ˆβ = 1 T! 1 T 1 Xt 0 X t t=1 T T Xt 0 y t+1 t=1

9 Forecasts with MSE loss IV If the process fy t+1, X t g is covariance stationary and ergodic for second moments, then as T! and 1 T 1 T T Xt 0 X t P! E X 0 t X t t=1 T X 0 P t y t+1! E X 0 t y t+1 t=1

10 Forecasts with MSE loss V If this holds, we have ˆβ P! α and OLS can be used to nd the optimal linear forcasts. Pay attention to violations of stationarity! Note that we need less assumptions than standard OLS. We are simply detecting patterns.

11 Forecasts Based on an In nite Number of Known Errors I For now assume that we have an in nite number of observations. First consider a process with an MA ( ) representation such that: where y t = µ + ψ (L) ε t ψ (L) = ψ j L j, ψ 0 = 1, j=0 fε t g is white noise and j=0 ψ j <

12 Forecasts Based on an In nite Number of Known Errors II Suppose that we know the process up to and including date t and want to forecast y t+s. Note that y t+s = µ + ε t+s + ψ 1 ε t+s ψ s 1 ε t+1 + ψ s ε t +... The optimal linear forecast is the projection of y t+s on all known ε t and a constant. Hamilton denotes this: y t+sjt = Ê [y t+s jε t, ε t 1, ε t 2,...] = µ + ψ s ε t + ψ s+1 ε t The forecast error is: y t+s y t+sjt = ε t+s + ψ 1 ε t+s ψ s 1 ε t+1

13 Forecasts Based on an In nite Number of Known Errors III And the mean squared error is: 2 E y t+s y t+sjt = σ ψ ψ2 s 1 If we instead of a MA ( ) process had a MA (q) process, the same principle can be applied. We get that, if q s y t+s = µ + ε t+s + ψ 1 ε t+s ψ s 1 ε t+1 + ψ s ε t ψ q ε t+s q and if q < s where y t+s = µ + ε t+s + ψ 1 ε t+s ψ q ε t+s q t + s q > t

14 Forecasts Based on an In nite Number of Known Errors IV The optimal linear forcast is: For q s For q < s The MSE is: For q s y t+sjt = Ê [y t+s jε t, ε t 1, ε t 2,...] = µ + ψ s ε t ψ q ε t q y t+sjt = Ê [y t+s jε t, ε t 1, ε t 2,...] = µ y t+s y t+sjt = ε t+s + ψ 1 ε t+s ψ s 1 ε t+1 and E y t+s 2 yt+sjt = σ ψ ψ2 s 1

15 Forecasts Based on an In nite Number of Known Errors V For q < s y t+s y t+sjt = y t+s µ = ε t+s + ψ 1 ε t+s ψ q ε t+s q and E y t+s 2 yt+sjt = σ ψ ψ2 q A complicated but useful way of writing this. Consider the polynomial ψ (L) = q ψ j L j, ψ 0 = 1 j=0 q can be nite or in nite.

16 Forecasts Based on an In nite Number of Known Errors VI Now divide by L s : q ψ (L) L s = ψ j L j s, ψ 0 = 1 j=0 The Annihilation Operator replaces all L j with j < 0 with 0. The notation is: ψ (L) q L s = ψ j L j s + j=s This one makes nding the forecast easier. Recall that the forecast projects onto the errors starting s periods earlier.

17 Forecasts Based on an In nite Number of Known Errors VII For the MA ( ) process we had: y t+sjt = µ + ψ s ε t + ψ s+1 ε t This can easily be written as ψ (L) y t+sjt = µ + = µ + j=s L s + ε t = µ + ψ j ε t+s j = µ + j=s i=0 ψ j L j ψ s+i ε t So, this prediction formula is based on an MA (q) representation (with q ) and an in nite number of errors. s ε t i

18 Forecasts Based on an In nite Number of Observations I While we can calculate all these theoretically, we wish to forecast using the data, that is past values of y, not the errors. Suppose we have a well-behaved process. Basically: AR process is stable and stationary MA process is invertible (details in next problemset) AR coe cients are absolutely summable. If these are true, we can use the observations fy i g t i= errors fε i g t i=. to nd the

19 Forecasts Based on an In nite Number of Observations II Examples: Suppose we have an AR (1) process: y t = c + φy t 1 + ε t then we can nd ε i = (1 φl) y i c or ε i = (1 φl) (yi µ)

20 Forecasts Based on an In nite Number of Observations III Suppose we have an MA (1) process: y t = µ + (1 + θl) ε t Now, if jθj < 1 so (1 + θl) 1 (y t µ) = ε t ε t = θ i L i (y t µ) i =0

21 Forecasts Based on an In nite Number of Observations IV The optimal linear forcast is: For s = 1 For 1 < s y t+sjt = Ê [y t+s jε t, ε t 1, ε t 2,...] = µ + θ y t+sjt = Ê [y t+s jε t, ε t 1, ε t 2,...] = µ Using the annihilation operator, we would have: ψ (L) y t+sjt = µ + L s ε t + i =0 θ i L i (y t µ)

22 Forecasts Based on an In nite Number of Observations V For s = 1 yt+1jt ψ (L) = µ + ε t = µ + θε t L + For 1 < s = µ + θ θ i L i (y t µ) i =0 y t+sjt = µ θl L s ε t = µ +

23 Forecasts Based on an In nite Number of Observations VI Under the assumptions we made earlier any standard processes can be written as AR ( ) processes. We can the nd the ε t as η (L) (y t µ) = ε t where η (L) = η j L j j=0

24 Forecasts Based on an In nite Number of Observations VII Then we can write y t+sjt = µ + ψ (L) L s where ψ (L) = η (L) 1. ε t + To get this as a function of the data, as opposed to the errors, we then write: ψ (L) y t+sjt = µ + L s η (L) (y t µ) + ψ (L) = µ + ψ (L) 1 (y t µ) L s This is called the Wiener-Kolmogorov prediction formula. +