Lecture 7: Exponential Smoothing Methods Please read Chapter 4 and Chapter 2 of MWH Book 1
Big Picture 1. In lecture 6, smoothing (averaging) method is used to estimate the trend-cycle (decomposition) 2. Now, modified smoothing method is used to forecast future values. That means, in general, the averaging is one-sided, as opposed to two-sided 3. Another difference is, we focus on out-of-sample forecasting errors, other than the in-sample fitting errors (residuals) 4. Again, which method works best depends on whether trend or seasonality is present 2
Forecasting Scenario 1. The present is the t-th period 2. Past (in-sample) observations Y t,y t 1,...,Y 1 are used to estimate the model 3. Then forecasts F t+1,f t+2,... are computed for the future (out-of-sample) values Y t+1,y t+2,... 4. Do not confuse the out-of-sample forecasting errors Y t+1 F t+1,y t+2 F t+2,... with the in-sample fitting errors (residuals) Y t F t,...,y 1 F 1 3
Averaging Method Mean This method uses all past data available F t+1 = 1 t F t+2 = t Y i (1) i=1 t+1 1 t + 1 i=1 Y i (2) The mean method works well only when there is no trend or seasonality or change in the mean value. In other words, it assumes the underlying process has a constant mean (stationarity). 4
Averaging Method Moving Average Moving average, to some degree, can allow for time-varying mean value. It does so by including only the most recent k observations. More explicitly, the MA(k) forecast (not smoother) is F t+1 = 1 k F t+2 = 1 k t Y i (3) i=t k+1 t+1 Y i (4) i=t k+2 It follows that F t+2 = F t+1 + 1 k (Y t+1 Y t k+1 ) (5) So each new forecast F t+2 is an adjustment of the immediately preceding forecast F t+1. The adjustment term is 1 k (Y t+1 Y t k+1 ). Bigger k leads to smaller adjustment (or smoother forecasts). Figure 4-5 in the book clearly shows that the mean method cannot capture the change in the mean value, but the moving average can, though with some lags. 5
Exponential Smoothing Method Single Exponential Smoothing (SES) I We hope the forecast can catch up with the change in mean value more quickly. One way is to assign a bigger weight to the latest observation. Consider F t+1 = F t + α(y t F t ) (6) F t+2 = F t+1 + α(y t+1 F t+1 ) (7) F 1 = Y 1 (initialization) (8) where 0 < α < 1. There is substantial adjustment when α is close to one. In fact, the single exponential smoothing utilizes the idea of error correction we revise up the forecast, i.e., F t+1 > F t if F t underestimates the true value Y t F t > 0. The error correcting process is also called negative feedback. 6
Single Exponential Smoothing II Equation (6) can be rewritten as F t+1 = (1 α)f t + αy t (9) By using the lag operator, we can show that the single exponential smoothing is equivalent to MA( ) process: F t+1 = α[(1 (1 α)l] 1 Y t = αy t + α(1 α)y t 1 + α(1 α) 2 Y t 2 +... (10) Moreover, notice that the weights α,α(1 α),α(1 α) 2,... decay exponentially toward zero. 7
Mean Square Error (MSE) Different smoothing parameter α yields different forecast. We can evaluate or rank various forecasts using mean square error (MSE), which is the average of squared error (in-sample or out-of-sample) 1 MSE = total count (Y i F i ) 2 i We square the error in order to avoid cancellation. The forecast with the smallest MSE is the best one. This fact suggests that we can obtain the optimal smoothing parameter α by minimizing MSE. 8
Theil s U-Statistic An alternative way to rank forecasts is considering the Theil s U-statistic ( ) n 1 Yt+1 F 2 t+1 t=1 U = Y t where Y t+1 F t+1 Y t n 1 t=1 ( Yt+1 Y t ) 2 (11) Y t is the forecast relative change based on a particular method, while Y t+1 Y t Y t is the forecast relative change based on a naive method F t+1 = Y t. A U-statistic less than one indicates a method better than the naive method. The method with smallest U-statistic is the best one. 9
Autocorrelation of Forecasting Error The forecasting error should be serially uncorrelated if no pattern is left unexploited. So a method that produces a white noise forecasting error is better than the one with serially correlated forecasting error. 10
Single Exponential Smoothing III The equation (6) for SES is recursive. This suggests that we may use loop. The R code below, for example, computes the SES forecast, and in the end, compute the MSE for α = 0.5. alpha = 0.5 f = rep(0, length(y)) f[1] = y[1] tse = 0 for (j in 2:length(y)) { f[j] = f[(j-1)] + alpha*(y[j-1]-f[(j-1)]) tse = tse + } (y[j] - f[j])ˆ2 mse =tse/(length(y)-1) 11
Adaptive-Response-Rate SES (ARRSES) The single SES has three limitations: (i) it is arbitrary regarding which α to use; (2) it employs a unique α to compute all forecasts. By contrast, ARRSES allows the value of α to be determined in a data-driven fashion. That is, α is adjusted automatically if there is change in the pattern of data, which can be signalized by big forecasting error. Intuitively, we prefer bigger α when bigger forecasting error arises: F t+1 = (1 α t )F t + α t Y t (12) α t+1 = A t (13) M t A t = βe t + (1 β)a t 1 (14) M t = β E t + (1 β)m t 1 (15) E t = Y t F t (16) So E t is the forecasting error; M t is a smoothed estimate of absolute forecasting error, and A t is a smoothed estimate of forecasting error. Notice that it is α t+1, not α t, in (13). 12
Holt s Linear Method (HLM) The third limitation of SES is that it cannot catch up with trend quickly enough, while Holt s Linear Method (HLM) is able to account for trend. L t = αy t + (1 α)(l t 1 + b t 1 ) (17) b t = β(l t L t 1 ) + (1 β)b t 1 (18) F t+m = L t + b t m (19) L 1 = Y 1 (20) b 1 = Y 2 Y 1 (21) where 0 < α < 1,0 < β < 1, and m is the number of periods ahead to be forecasted. Here L t denotes the level of the series, and b t the slope of trend. HLM is also called double exponential smoothing. The HLM has the drawback of failing to account for seasonal effect. 13
Holt-Winter s Method (HWM) We use HWM to account for both trend and seasonality. HWM involves three equations: one for level, one for trend; and one for seasonality. L t = α Y t + (1 α)(l t 1 + b t 1 ) S t s (22) b t = β(l t L t 1 ) + (1 β)b t 1 (23) S t = γ Y t + (1 γ)s t s L t (24) F t+m = (L t + b t m)s t s+m (25) where s is the length of seasonality (e.g., the number of months in a year), and the seasonality has a multiplicative form. 14
Prediction Intervals We can use prediction intervals (PI) to illustrate the inherent uncertainty in the point forecast: 95% PI = F t+1 ± 1.96 MSE (26) where F t+1 is the point forecast, and MSE is the mean square error for the method that produces F t+1 15