Forecasting with ARIMA models This version: 14 January 2018

Transcription

1 Forecasting with ARIMA models This version: 14 January 2018 Notes for Intermediate Econometrics / Time Series Analysis and Forecasting Anthony Tay Elsewhere we showed that the optimal forecast for a mean square forecast error minimizer is the conditional mean of the target variable, conditioned on the information set being used to generate the forecast. We explore in this note forecasts based on the past history of the series itself, generated from ARIMA and trend-stationary models, assuming mean square forecast error loss functions. We assume initially that we know the DGP, and the parameters of the model. This is an unrealistic assumption, of course, but simplifies arguments so that we can focus on the properties of forecasts and forecast errors in the best case scenario. AR(1) Suppose Y t = β 0 + β 1 Y t 1 + ɛ t, ɛ t iid (0, σ 2 ), β 1 < 1, Suppose at time T, we wish to make a forecast of the realization of Y T +h. We refer to h as the forecast horizon. For h = 1, the appropriate forecast rule is obvious: Y T +1 T = E[Y T +1 Y T,...] = β 0 + β 1 Y T with forecast error ˆɛ T +1 T = ɛ T +1. For h = 2, we have Y T +2 = β 0 + β 1 Y T +1 + ɛ T +2 = β 0 (1 + β 1 ) + β1y 2 T + ɛ T +2 + β 1 ɛ T +1 Therefore Y T +2 T = E[Y T +2 Y T,...] = β 0 (1 + β 1 ) + β1y 2 T with forecast error ˆɛ T +2 T = ɛ T +2 + β 1 ɛ T +1. Notice that you will have obtained the same result by replacing Y T +1 with Y T +1 T : Y T +2 T = E[Y T +2 Y T,...] = β 0 + β 1 E[Y T +1 Y T,...] + E[ɛ T +2 Y T,...] = β 0 + β 1 Y T +1 T = β 0 (1 + β 1 ) + β1y 2 T We can use replace future unknowns by their forecasts. 1

2 Intermediate Econometrics / Time Series Analysis and Forecasting 2 We can generalize these arguments to h-step ahead forecasts. It should be clear that Y T +h T = β 0 (1 + β 1 + β β h 1 1 ) + β h 1 Y T with forecast error ˆɛ T +h T = ɛ T +h + β 1 ɛ T +h β h 1 ɛ T. Notice that as h, we have Y T +h T β 0 i=0 β i 1 = β 0 1 β 1 as h which is the conditional mean of the AR(1) process. The intuition is that information available at period T becomes less and less informative. In the limit, the best forecast we can make is one with no information from predictors, i.e., the unconditional mean. It is also straightforward to show that if β 1 is positive (we continue to assume it is less than one) then Y T +h T converges monotonically to the unconditional mean as h : if Y T > β 0 /(1 β 1 ), then Y T +h T will be decreasing in h, converging toward β 0 /(1 β 1 ). If Y T < β 0 /(1 β 1 ), then Y T +h T will be rising in h, converging toward β 0 /(1 β 1 ). As the forecast converges to the unconditional mean, it should not be surprising that the the forecast error variance converges to the unconditional variance var[ˆɛ T +h T ] σ 2 ( β1 2i ) = σ2 1 β1 2 i=0 as h. The series Y in the file timeseries_quarterly.csv is quarterly frequency from 1980Q1 to 2011Q4. library(tidyverse) library(xts) library(forecast) library(ggfortify) library(gridextra) library(grid) # Theme settings for figures: ts_thm <- theme(text = element_text(size=14), axis.title = element_text(size=11), axis.line = element_line(linetype = 'solid'), panel.background = element_blank()) # We use data from "timeseries_quarter.csv" and timeseries_monthly.csv" df_qtr <- read_csv("timeseries_quarterly.csv") glimpse(df_qtr)

3 Intermediate Econometrics / Time Series Analysis and Forecasting 3 Observations: 128 Variables: 3 $ Period <chr> "1980Q1", "1980Q2", "1980Q3", "1980Q4", "1981Q1", " $ Y <dbl> , , , , , $ Y1 <dbl> , , , , , Y.xts <- xts(df_qtr$y, order.by = as.yearqtr(df_qtr$period)) Y.ts <- as.ts(y.xts, start=start(y.xts)) autoplot(y.ts) + ts_thm + theme(aspect.ratio = 1/2) We will suppose we are are the end of 2004, and generate a long-range forecast of up to 2011Q4, without information about the realizations from 2005 onwards. fit1 <- Arima(window(Y.ts, start=c(1980,1), end=c(2004,4)), order=c(1,0,0)) summary(fit1) Series: window(y.ts, start = c(1980, 1), end = c(2004, 4)) ARIMA(1,0,0) with non-zero mean Coefficients: ar1 mean s.e sigma^2 estimated as : log likelihood= AIC= AICc= BIC= Training set error measures: ME RMSE MAE MPE MAPE MASE Training set ACF1 Training set

4 Intermediate Econometrics / Time Series Analysis and Forecasting 4 sse <- sum(fit1$residuals^2) sst <- sum((fit1$x - mean(fit1$x))^2) Rsqr <- 1 - sse/sst print(paste0("r-sqr is ", as.character(round(rsqr,2)))) [1] "R-sqr is 0.81" fcst <- forecast(fit1, h=7*4) # forecast 1 to 28 steps (7 years x 4 quarters) ahead autoplot(ts.union(y.ts, fcst$mean, fcst$lower, fcst$upper)) + scale_color_manual(values=rep("black", 6)) + ylab("") + xlab("") + aes(linetype=series) + scale_linetype_manual(values=c("solid", "dashed", rep("dotted",4))) + ts_thm + theme(legend.position="none") We do not show or analyze the fit of the model, though we present the model summary, and the (in-sample) R 2 which is The forecasts (dashed line) are shown against the actual realizations (solid) and the 80% intervals (inner dotted) and 95% intervals (outer dotted). The monotonic convergence of the forecast toward the unconditional mean is clear, as is the convergence of the forecast variance toward the unconditional variance, as indicated by the prediction intervals. The following code produces a series of one-step ahead forecasts # 1980Q1 to 2004Q4 is observation 1:100 # 2005Q1 to 2011Q4 is observation 101 to 128 (28 observations) fcst1step<-ts(matrix(rep(na,28*3),ncol=3), start=c(2005,1), frequency=4) # to store forecasts colnames(fcst1step) <- c("mean", "lower", "upper") for (i in 1:28){ fit1 <- Arima(Y.ts[1:(100+i-1)], order=c(1,0,0))

5 Intermediate Econometrics / Time Series Analysis and Forecasting 5 temp <- forecast(fit1, h=1) fcst1step[i,]<-cbind(temp$mean, temp$lower[,"95%"], temp$upper[,"95%"]) } ts.plot <- ts.union(actual=window(y.ts,start=c(2002,1)),fcst1step) autoplot(ts.plot) + scale_color_manual(values=rep("black", 4)) + ylab("") + xlab("") + aes(linetype=series) + scale_linetype_manual(values=c("solid", "dashed", rep("dotted",2))) + ts_thm + theme(legend.position="none") f1err <- ts.union(actual=window(y.ts,start=c(2005,1)),fcst=fcst1step[,1]) sse <- sum((f1err[,"actual"]-f1err[,"fcst"])^2) sst <- sum((f1err[,"actual"]-mean(f1err[,"actual"]))^2) OOSR2 <- 1-sse/sst print(paste0("out-of-sample RMSE is ",as.character(round(sqrt(sse/28),2)))) [1] "Out-of-sample RMSE is 0.54" print(paste0("out-of-sample R-sqr is ",as.character(round(oosr2,2)))) [1] "Out-of-sample R-sqr is 0.65" It is not surprising that the forecast performance looks a lot better than in the previous case, as we are only forecasting 1-step ahead, updating our information set each period (we also update the parameter estimates). As a quick evaluation of the 1-step-ahead forecasts, we can compute the

6 Intermediate Econometrics / Time Series Analysis and Forecasting 6 out-of-sample Root Mean Square Forecast Error (RMSE) by RMSE = 1 N N ˆɛ 2 T +i T +i 1 i=1 which turns out to be 0.54, highly than the in-sample RMSE of For stationary time series, it is informative to report the out-of-sample Forecast R 2 defined by Forecast-R 2 = 1 Ni=1 ˆɛ 2 T +i T +i 1 Ni (Y T +i Y ) 2 which you can compare with the in-sample R 2. Note that despite knowing the true model, and updating the fit each period, the out-of-sample Forecast R 2 is 0.65, which is lower than the in-sample R 2 of The model explains 81% of the variation in Y, but is able out-of-sample to forecast only 65% of the total variation in Y. Stochastic Trend vs Deterministic Trend For the stochastic trend model, the equations are Y T +1 = β 0 + Y }{{ T + ɛ } T +1 }{{} Y T +1 T ˆɛ T +1 T Y T +2 = β 0 + Y T +1 + ɛ T +2 = 2β 0 + Y }{{ T + ɛ } T +1 + ɛ T +2 }{{} Y T +2 T ˆɛ T +2 T Y T +3 = β 0 + Y T +2 + ɛ T +3 = 3β 0 + Y }{{ T + ɛ } T +1 + ɛ T +2 + ɛ T +3 }{{} Y T +3 T ˆɛ T +3 T The characteristics of stochastic trend forecasts are easily surmised from these equations: from Y T, the random walk with drift will forecast a linear trend (with drift term β 0 as the slope), starting from Y T, and with forecast errors that have variances increasing without bound. The forecasts from a deterministic linear trend are different from those of the stochastic trend in a nunber of ways: Y T +1 = β 0 + β 1 (T + 1) + ɛ T +1 }{{}}{{} Y T +1 T ˆɛ T +1 T Y T +2 = β 0 + β 1 (T + 2) + ɛ T +2 = β 0 + β 1 (T + 2) + ɛ T +2 }{{}}{{} Y T +2 T ˆɛ T +2 T

7 Intermediate Econometrics / Time Series Analysis and Forecasting 7 Y T +3 = β 0 + β 1 (T + 3) + ɛ T +3 = β 0 + β 1 (T + 3) + ɛ T +3 }{{}}{{} Y T +3 T ˆɛ T +3 T The forecast is a continuation of the estimated trend line, and the forecast standard errors are constant (no matter how far into the future you project). This seems unrealistic of long-term projections of real economic time series. We use industrial production as an example. We fit deterministic and stochastic trend models to the series from Jan 1983 to Dec 2007, and project over Jan 2008 to Dec One would ordinarily not (shouldn t) make such a long term projection, but we do so to illustrate some differences between projecting with the two kinds of trend. df_mth <- read_csv("timeseries_monthly.csv") glimpse(df_mth) Observations: 420 Variables: 6 $ DATE <chr> "Jan-83", "Feb-83", "Mar-83", "Apr-83", "May-83",... $ ELEC_GEN_SG <dbl> 667.3, 586.9, 727.4, 719.0, 727.1, 728.4, 741.1, 7... $ TOUR_SG <int> , , , , , , $ IP_SG <dbl> 14.34, 11.37, 14.50, 12.86, 13.01, 12.62, 13.56, 1... $ CPI_US <dbl> 97.8, 97.9, 97.9, 98.6, 99.2, 99.5, 99.9, 100.2, 1... $ DOMEX5_SG <dbl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA... IP_SG.xts <- xts(df_mth$ip_sg, order.by = as.yearmon(df_mth$date, "%B-%y")) IP_SG.ts <- as.ts(ip_sg.xts, start=start(ip_sg.xts)) autoplot(log(ip_sg.ts)) + ts_thm + theme(aspect.ratio = 1/2) In the following code, the raw series is fed into the Arima function, where it is put through a

8 Intermediate Econometrics / Time Series Analysis and Forecasting 8 box-cox transformation Y λ t 1 λ An application of L hopital s rule shows that this transformation approaches the natural log transformation when λ approaches zero. Setting λ = 0 in the Arima function gives the natural log transformation. When the forecast is made, the data is transformed back into the original scale. We begin with a pure deterministic trend plus seasonal dummies model. trendvar <- seq_along(ip_sg.ts) seasonalvars <- seasonaldummy(ip_sg.ts) Xreg = cbind(trendvar, seasonalvars) fit2 <- Arima(window(IP_SG.ts,start=c(1983,1), end=c(2008,12)), summary(fit2) order=c(0,0,0), xreg=xreg[1:312,], # Jan 1983 to Dec 2007 lambda=0, # box-cox transformation, lambda=0 is log-transformation biasadj = T) Series: window(ip_sg.ts, start = c(1983, 1), end = c(2008, 12)) Regression with ARIMA(0,0,0) errors Box Cox transformation: lambda= 0 Coefficients: intercept trendvar Jan Feb Mar Apr May s.e Jun Jul Aug Sep Oct Nov s.e sigma^2 estimated as : log likelihood= AIC= AICc= BIC= Training set error measures: ME RMSE MAE MPE MAPE MASE Training set ACF1 Training set fcst <- forecast(fit2, h=108, xreg=xreg[313:420,], biasadj=t) # forecast 1 to 108 steps (9 years x 12 quarters) ahead

9 Intermediate Econometrics / Time Series Analysis and Forecasting 9 fcst.plot <- ts.union(ip_sg.ts, fitted(fit2), fcst$mean, fcst$lower[,1], fcst$upper[,1]) # 80% autoplot(window(fcst.plot, start=c(2001,1))) + scale_color_manual(values=rep("black", 5)) + ylab("") + xlab("") + aes(linetype=series, size=series) + scale_size_manual(values=c(0.5,1,0.5,0.5,0.5)) + scale_linetype_manual(values=c("solid", "solid", "dashed", rep("dotted",4))) + ts_thm + theme(legend.position="none") The forecast plot is of the back-transformed series, which is why the projection is exponential rather than linear despite having fit a linear trend. We plot the actual (both in-sample and out-of-sample), the fitted (bold), the forecast (dashed), and 95% bounds (dotted). You can see that the projection is a continuation of the fitted line, and the forecast interval quickly becomes constant width, even though you are projecting many years into the future from a fixed time-point. The forecasts are obviously bad, since the series deviated from the fitted trend significantly over the forecast sample. Xreg = cbind(seasonalvars) fit3 <- Arima(window(IP_SG.ts,start=c(1983,1), end=c(2008,12)), order=c(1,1,0), xreg=xreg[1:312,], include.drift = T, lambda=0, # box-cox transformation, lambda=0 is log-transformation biasadj = T)

10 Intermediate Econometrics / Time Series Analysis and Forecasting 10 summary(fit3) Series: window(ip_sg.ts, start = c(1983, 1), end = c(2008, 12)) Regression with ARIMA(1,1,0) errors Box Cox transformation: lambda= 0 Coefficients: ar1 drift Jan Feb Mar Apr May Jun s.e Jul Aug Sep Oct Nov s.e sigma^2 estimated as : log likelihood= AIC= AICc= BIC= Training set error measures: ME RMSE MAE MPE MAPE MASE Training set ACF1 Training set fcst <- forecast(fit3, h=108, xreg=xreg[313:420,], biasadj=t) # forecast 1 to 120 steps (10 years x 12 quarters) ahead fcst.plot <- ts.union(ip_sg.ts, fitted(fit3), fcst$mean, fcst$lower[,1], fcst$upper[,1]) # 80% autoplot(window(fcst.plot, start=c(2001,1))) + scale_color_manual(values=rep("black", 5)) + ylab("") + xlab("") + aes(linetype=series, size=series) + scale_size_manual(values=c(0.5,1,0.5,0.5,0.5)) + scale_linetype_manual(values=c("solid", "solid", "dashed", rep("dotted",4))) + ts_thm + theme(legend.position="none")

11 Intermediate Econometrics / Time Series Analysis and Forecasting There are two main differences between the stochastic trend forecasts and the deterministic trend forecasts. The stochastic trend projection is ultimate merely a projection of the fitted trend (drift) but it starts from the last realization. The forecast interval increases with horizon, but this time without bound, which is much more realistic. In both cases there are periods where the long-run predictions appear to do very well. These episodes are quite coincidental. The differences between linear deterministic trend and stochastic trend forecasts are much more stark when looking at the sequence of one-step ahead forecasts, i.e., you update your sample every period, and forecast only one-step ahead each time. #Xreg = cbind(trendvar, seasonalvars) Xreg = seasonalvars fcst1step<-ts(matrix(rep(na,108*3),ncol=3), start=c(2009,1), frequency=12) # to store forecasts colnames(fcst1step) <- c("mean", "lower", "upper") for (i in 1:108){ fit1 <- Arima(IP_SG.ts[1:(312+i-1)], order=c(0,1,0), xreg=xreg[1:(312+i-1),], include.constant = T, biasadj = T, lambda=0) temp <- forecast(fit1, h=1, xreg=matrix(xreg[(312+i):(312+i),], nrow=1))

12 Intermediate Econometrics / Time Series Analysis and Forecasting 12 fcst1step[i,]<-cbind(temp$mean, temp$lower[,"80%"], temp$upper[,"80%"]) } ts.plot <- ts.union(actual=window(ip_sg.ts,start=c(2004,1)),fcst1step) autoplot(ts.plot) + scale_color_manual(values=rep("black", 4)) + ylab("") + xlab("") + aes(linetype=series) + scale_linetype_manual(values=c("solid", "dashed", rep("dotted",2))) + ts_thm + theme(legend.position="none") Xreg = cbind(trendvar, seasonalvars) fcst1step<-ts(matrix(rep(na,108*3),ncol=3), start=c(2009,1), frequency=12) # to store forecasts colnames(fcst1step) <- c("mean", "lower", "upper") for (i in 1:108){ fit1 <- Arima(IP_SG.ts[1:(312+i-1)], order=c(0,0,0), xreg=xreg[1:(312+i-1),], include.constant = T, biasadj = T, lambda=0) temp <- forecast(fit1, h=1, xreg=matrix(xreg[(312+i):(312+i),], nrow=1)) fcst1step[i,]<-cbind(temp$mean, temp$lower[,"80%"], temp$upper[,"80%"])

13 Intermediate Econometrics / Time Series Analysis and Forecasting 13 } ts.plot <- ts.union(actual=window(ip_sg.ts,start=c(2004,1)),fcst1step) autoplot(ts.plot) + scale_color_manual(values=rep("black", 4)) + ylab("") + xlab("") + aes(linetype=series) + scale_linetype_manual(values=c("solid", "dashed", rep("dotted",2))) + ts_thm + theme(legend.position="none") The one-step ahead forecasts from a deterministic time trend model is essentially the same as the long-range forecast, since it merely follows the fitted trend line. Finally, we end off with forecasts with models selected from the forecast package auto.arima. In our implementation, we allow the algorithm to select a new model each period. The algorithm decides (via KPSS) whether or not to difference the series (both first and seasonal differencing), and then to choose the order (both non-seasonal and seasonal ARMA orders). In the code below, there is a line that you can un-comment (print(fit1$arma)) to ask the algorithm to display the model choice (ar, ma seasonal ar, seasonal ma, frequency, no of differencing, no of seasonal differencing). There is some variation at first, but it eventually settles down to ARMA(0, 1, 1)(0, 0, 2) 4. fcst1step<-ts(matrix(rep(na,108*3),ncol=3), start=c(2009,1), frequency=12) # to store forecasts colnames(fcst1step) <- c("mean", "lower", "upper") for (i in 1:108){

14 Intermediate Econometrics / Time Series Analysis and Forecasting 14 fit1 <- auto.arima(ts(ip_sg.ts[1:(312+i-1)], start=c(1983,1), frequency=12), lambda=0, biasadj = TRUE, seasonal = T) # print(fit1$arma) temp <- forecast(fit1, h=1) fcst1step[i,]<-cbind(temp$mean, temp$lower[,"80%"], temp$upper[,"80%"]) } ts.plot <- ts.union(actual=window(ip_sg.ts,start=c(2004,1)),fcst1step) autoplot(ts.plot) + scale_color_manual(values=rep("black", 4)) + ylab("") + xlab("") + aes(linetype=series) + scale_linetype_manual(values=c("solid", "dashed", rep("dotted",2))) + ts_thm + theme(legend.position="none") Exercises 1. Show that the forecasts Y T +h T from a stationary AR(1) with β 1 (0, 1) decreases monotonically towards the unconditional mean of the AR(1) if Y T is above it, and increases monotonically towards the unconditional mean of the AR(1) if Y T is below it. (Hint: we have already shown convergence to the mean; check the direction of convergence in each case by deriving an appropriate expression for Y T +h+1 T Y T +h T ).