Forecasting using R. Rob J Hyndman. 3.2 Dynamic regression. Forecasting using R 1

Transcription

1 Forecasting using R Rob J Hyndman 3.2 Dynamic regression Forecasting using R 1

2 Outline 1 Regression with ARIMA errors 2 Stochastic and deterministic trends 3 Periodic seasonality 4 Lab session 14 5 Dynamic regression models Forecasting using R Regression with ARIMA errors 2

3 Regression with ARIMA errors Regression models y t = β 0 + β 1 x 1,t + + β k x k,t + e t, y t modeled as function of k explanatory variables x 1,t,..., x k,t. In regression, we assume that e t was WN. Now we want to allow e t to be autocorrelated. Example: ARIMA(1,1,1) errors y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, where e t is white noise. Forecasting using R Regression with ARIMA errors 3

4 Regression with ARIMA errors Regression models y t = β 0 + β 1 x 1,t + + β k x k,t + e t, y t modeled as function of k explanatory variables x 1,t,..., x k,t. In regression, we assume that e t was WN. Now we want to allow e t to be autocorrelated. Example: ARIMA(1,1,1) errors y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, where e t is white noise. Forecasting using R Regression with ARIMA errors 3

5 Residuals and errors Example: N t = ARIMA(1,1,1) y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, Be careful in distinguishing n t from e t. Only the errors n t are assumed to be white noise. In ordinary regression, n t is assumed to be white noise and so n t = e t. Forecasting using R Regression with ARIMA errors 4

6 Residuals and errors Example: N t = ARIMA(1,1,1) y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, Be careful in distinguishing n t from e t. Only the errors n t are assumed to be white noise. In ordinary regression, n t is assumed to be white noise and so n t = e t. Forecasting using R Regression with ARIMA errors 4

7 Estimation If we minimize n 2 t (by using ordinary regression): 1 Estimated coefficients ˆβ 0,..., ˆβ k are no longer optimal as some information ignored; 2 Statistical tests associated with the model (e.g., t-tests on the coefficients) are incorrect. 3 p-values for coefficients usually too small ( spurious regression ). 4 AIC of fitted models misleading. Minimizing e 2 t avoids these problems. Maximizing likelihood is similar to minimizing e 2 t. Forecasting using R Regression with ARIMA errors 5

8 Estimation If we minimize n 2 t (by using ordinary regression): 1 Estimated coefficients ˆβ 0,..., ˆβ k are no longer optimal as some information ignored; 2 Statistical tests associated with the model (e.g., t-tests on the coefficients) are incorrect. 3 p-values for coefficients usually too small ( spurious regression ). 4 AIC of fitted models misleading. Minimizing e 2 t avoids these problems. Maximizing likelihood is similar to minimizing e 2 t. Forecasting using R Regression with ARIMA errors 5

9 Stationarity Regression with ARMA errors y t = β 0 + β 1 x 1,t + + β k x k,t + n t, where n t is an ARMA process. All variables in the model must be stationary. If we estimate the model while any of these are non-stationary, the estimated coefficients can be incorrect. Difference variables until all stationary. If necessary, apply same differencing to all variables. Forecasting using R Regression with ARIMA errors 6

10 Stationarity Model with ARIMA(1,1,1) errors y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, Equivalent to model with ARIMA(1,0,1) errors y t = β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)n t = (1 + θ 1 B)e t, where y t = y t y t 1, x t,i = x t,i x t 1,i and n t = n t n t 1. Forecasting using R Regression with ARIMA errors 7

11 Stationarity Model with ARIMA(1,1,1) errors y t = β 0 + β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)(1 B)n t = (1 + θ 1 B)e t, Equivalent to model with ARIMA(1,0,1) errors y t = β 1 x 1,t + + β k x k,t + n t, (1 φ 1 B)n t = (1 + θ 1 B)e t, where y t = y t y t 1, x t,i = x t,i x t 1,i and n t = n t n t 1. Forecasting using R Regression with ARIMA errors 7

12 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = β 0 + β 1 x 1,t + + β k x k,t + n t where φ(b)(1 B) d N t = θ(b)e t After differencing all variables y t = β 1 x 1,t + + β k x k,t + n t. where and φ(b)n t = θ(b)e t y t = (1 B) d y t Forecasting using R Regression with ARIMA errors 8

13 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = β 0 + β 1 x 1,t + + β k x k,t + n t where φ(b)(1 B) d N t = θ(b)e t After differencing all variables y t = β 1 x 1,t + + β k x k,t + n t. where and φ(b)n t = θ(b)e t y t = (1 B) d y t Forecasting using R Regression with ARIMA errors 8

14 Regression with ARIMA errors Any regression with an ARIMA error can be rewritten as a regression with an ARMA error by differencing all variables with the same differencing operator as in the ARIMA model. Original data y t = β 0 + β 1 x 1,t + + β k x k,t + n t where φ(b)(1 B) d N t = θ(b)e t After differencing all variables y t = β 1 x 1,t + + β k x k,t + n t. where and φ(b)n t = θ(b)e t y t = (1 B) d y t Forecasting using R Regression with ARIMA errors 8

15 Model selection Check that all variables are stationary. If not, apply differencing. Where appropriate, use the same differencing for all variables to preserve interpretability. Fit regression model with automatically selected ARIMA errors. Check that e t series looks like white noise. Selecting predictors AICc can be calculated for final model. Repeat procedure for all subsets of predictors to be considered, and select model with lowest AIC value. Forecasting using R Regression with ARIMA errors 9

16 US personal consumption & income 2 Quarterly changes in US consumption and personal income consumption income Year Forecasting using R Regression with ARIMA errors 10

17 US personal consumption & income Quarterly changes in US consumption and personal income 2 1 consumption income Forecasting using R Regression with ARIMA errors 11

18 US Personal Consumption and income No need for transformations or further differencing. Increase in income does not necessarily translate into instant increase in consumption (e.g., after the loss of a job, it may take a few months for expenses to be reduced to allow for the new circumstances). We will ignore this for now. Forecasting using R Regression with ARIMA errors 12

19 US personal consumption & income (fit <- auto.arima(usconsumption[,1], xreg=usconsumption[,2])) ## Series: usconsumption[, 1] ## ARIMA(1,0,2) with non-zero mean ## ## Coefficients: ## ar1 ma1 ma2 intercept usconsumption[, 2] ## ## s.e ## ## sigma^2 estimated as : log likelihood= ## AIC= AICc= BIC= Forecasting using R Regression with ARIMA errors 13

20 US personal consumption & income ggtsdisplay(arima.errors(fit), main="arima errors") 1 0 ARIMA errors x Time ACF PACF Lag Lag Forecasting using R Regression with ARIMA errors 14

21 US personal consumption & income ggtsdisplay(residuals(fit), main="arima residuals") x ARIMA residuals Time ACF 0.0 PACF Lag Lag Forecasting using R Regression with ARIMA errors 15

22 US Personal Consumption and Income A Ljung-Box test shows the residuals are uncorrelated. Box.test(residuals(fit), fitdf=5, lag=10, type="ljung") ## ## Box-Ljung test ## ## data: residuals(fit) ## X-squared = , df = 5, p-value = Forecasting using R Regression with ARIMA errors 16

23 US Personal Consumption and Income fcast <- forecast(fit, xreg=rep(mean(usconsumption[,2]),8), h=8) autoplot(fcast) + xlab("year") + ylab("percentage change") + ggtitle("forecasts from regression with ARIMA(1,0,2) errors" Forecasts from regression with ARIMA(1,0,2) errors 2 Percentage change level Year Forecasting using R Regression with ARIMA errors 17

24 Forecasting To forecast a regression model with ARIMA errors, we need to forecast the regression part of the model and the ARIMA part of the model and combine the results. Some explanatory variable are known into the future (e.g., time, dummies). Separate forecasting models may be needed for other explanatory variables. Forecasting using R Regression with ARIMA errors 18

25 Outline 1 Regression with ARIMA errors 2 Stochastic and deterministic trends 3 Periodic seasonality 4 Lab session 14 5 Dynamic regression models Forecasting using R Stochastic and deterministic trends 19

26 Stochastic & deterministic trends Deterministic trend y t = β 0 + β 1 t + n t where n t is ARMA process. Stochastic trend y t = β 0 + β 1 t + n t where n t is ARIMA process with d 1. Difference both sides until n t is stationary: y t = β 1 + n t where n t is ARMA process. Forecasting using R Stochastic and deterministic trends 20

27 Stochastic & deterministic trends Deterministic trend y t = β 0 + β 1 t + n t where n t is ARMA process. Stochastic trend y t = β 0 + β 1 t + n t where n t is ARIMA process with d 1. Difference both sides until n t is stationary: y t = β 1 + n t where n t is ARMA process. Forecasting using R Stochastic and deterministic trends 20

28 Stochastic & deterministic trends Deterministic trend y t = β 0 + β 1 t + n t where n t is ARMA process. Stochastic trend y t = β 0 + β 1 t + n t where n t is ARIMA process with d 1. Difference both sides until n t is stationary: y t = β 1 + n t where n t is ARMA process. Forecasting using R Stochastic and deterministic trends 20

29 International visitors Total annual international visitors to Australia 5 millions of people Year Forecasting using R Stochastic and deterministic trends 21

30 International visitors Deterministic trend (fit1 <- auto.arima(austa, d=0, xreg=1:length(austa))) ## Series: austa ## ARIMA(2,0,0) with non-zero mean ## ## Coefficients: ## ar1 ar2 intercept 1:length(austa) ## ## s.e ## ## sigma^2 estimated as : log likelihood=12.7 ## AIC=-15.4 AICc=-13 BIC=-8.23 y t = t + n t n t = n t n t 2 + e t e t NID(0, ). Forecasting using R Stochastic and deterministic trends 22

31 International visitors Deterministic trend (fit1 <- auto.arima(austa, d=0, xreg=1:length(austa))) ## Series: austa ## ARIMA(2,0,0) with non-zero mean ## ## Coefficients: ## ar1 ar2 intercept 1:length(austa) ## ## s.e ## ## sigma^2 estimated as : log likelihood=12.7 ## AIC=-15.4 AICc=-13 BIC=-8.23 y t = t + n t n t = n t n t 2 + e t e t NID(0, ). Forecasting using R Stochastic and deterministic trends 22

32 Forecasting using R y t y t 1 = e t y t = y t + n t n t = n t 1 + e t e NID(0, ). Stochastic and deterministic trends 23 International visitors Stochastic trend (fit2 <- auto.arima(austa,d=1)) ## Series: austa ## ARIMA(0,1,0) with drift ## ## Coefficients: ## drift ## ## s.e ## ## sigma^2 estimated as : log likelihood=9.38 ## AIC= AICc= BIC=-11.96

33 Forecasting using R y t y t 1 = e t y t = y t + n t n t = n t 1 + e t e NID(0, ). Stochastic and deterministic trends 23 International visitors Stochastic trend (fit2 <- auto.arima(austa,d=1)) ## Series: austa ## ARIMA(0,1,0) with drift ## ## Coefficients: ## drift ## ## s.e ## ## sigma^2 estimated as : log likelihood=9.38 ## AIC= AICc= BIC=-11.96

34 International visitors 8 Forecasts from linear trend with AR(2) error level Year Forecasts from ARIMA(0,1,0) with drift level Year Forecasting using R Stochastic and deterministic trends 24

35 Forecasting with trend Point forecasts are almost identical, but prediction intervals differ. Stochastic trends have much wider prediction intervals because the errors are non-stationary. Be careful of forecasting with deterministic trends too far ahead. Forecasting using R Stochastic and deterministic trends 25

36 Outline 1 Regression with ARIMA errors 2 Stochastic and deterministic trends 3 Periodic seasonality 4 Lab session 14 5 Dynamic regression models Forecasting using R Periodic seasonality 26

37 Fourier terms for seasonality Periodic seasonality can be handled using pairs of Fourier terms: ( 2πkt s k (t) = sin m ) ( 2πkt c k (t) = cos m y t = K [α k s k (t) + β k c k (t)] + n t k=1 n t is non-seasonal ARIMA process. Every periodic function can be approximated by sums of sin and cos terms for large enough K. Choose K by minimizing AICc. ) Forecasting using R Periodic seasonality 27

38 US Accidental Deaths fit <- auto.arima(usaccdeaths, xreg=fourier(usaccdeaths, 5), seasonal=false) fc <- forecast(fit, xreg=fourier(usaccdeaths, 5, 24)) Forecasting using R Periodic seasonality 28

39 US Accidental Deaths autoplot(fc) Forecasts from ARIMA(0,1,1) y level Forecasting using R Time Periodic seasonality 29

40 Outline 1 Regression with ARIMA errors 2 Stochastic and deterministic trends 3 Periodic seasonality 4 Lab session 14 5 Dynamic regression models Forecasting using R Lab session 14 30

41 Lab Session 14 Forecasting using R Lab session 14 31

42 Outline 1 Regression with ARIMA errors 2 Stochastic and deterministic trends 3 Periodic seasonality 4 Lab session 14 5 Dynamic regression models Forecasting using R Dynamic regression models 32

43 Dynamic regression models Sometimes a change in x t does not affect y t instantaneously y t = sales, x t = advertising. y t = stream flow, x t = rainfall. y t = size of herd, x t = breeding stock. These are dynamic systems with input (x t ) and output (y t ). x t is often a leading indicator. There can be multiple predictors. Forecasting using R Dynamic regression models 33

44 Dynamic regression models Sometimes a change in x t does not affect y t instantaneously y t = sales, x t = advertising. y t = stream flow, x t = rainfall. y t = size of herd, x t = breeding stock. These are dynamic systems with input (x t ) and output (y t ). x t is often a leading indicator. There can be multiple predictors. Forecasting using R Dynamic regression models 33

45 Dynamic regression models Sometimes a change in x t does not affect y t instantaneously y t = sales, x t = advertising. y t = stream flow, x t = rainfall. y t = size of herd, x t = breeding stock. These are dynamic systems with input (x t ) and output (y t ). x t is often a leading indicator. There can be multiple predictors. Forecasting using R Dynamic regression models 33

46 Lagged explanatory variables The model include present and past values of predictor: x t, x t 1, x t 2,.... y t = a + ν 0 x t + ν 1 x t ν k x t k + n t where n t is an ARIMA process. Rewrite model as y t = a + (ν 0 + ν 1 B + ν 2 B ν k B k )x t + n t = a + ν(b)x t + n t. ν(b) is called a transfer function since it describes how change in x t is transferred to y t. x can influence y, but y is not allowed to influence x. Forecasting using R Dynamic regression models 34

47 Lagged explanatory variables The model include present and past values of predictor: x t, x t 1, x t 2,.... y t = a + ν 0 x t + ν 1 x t ν k x t k + n t where n t is an ARIMA process. Rewrite model as y t = a + (ν 0 + ν 1 B + ν 2 B ν k B k )x t + n t = a + ν(b)x t + n t. ν(b) is called a transfer function since it describes how change in x t is transferred to y t. x can influence y, but y is not allowed to influence x. Forecasting using R Dynamic regression models 34

48 Lagged explanatory variables The model include present and past values of predictor: x t, x t 1, x t 2,.... y t = a + ν 0 x t + ν 1 x t ν k x t k + n t where n t is an ARIMA process. Rewrite model as y t = a + (ν 0 + ν 1 B + ν 2 B ν k B k )x t + n t = a + ν(b)x t + n t. ν(b) is called a transfer function since it describes how change in x t is transferred to y t. x can influence y, but y is not allowed to influence x. Forecasting using R Dynamic regression models 34

49 Example: Insurance quotes and TV adverts Insurance advertising and quotations Quotes TV.advert Year Forecasting using R Dynamic regression models 35

50 Example: Insurance quotes and TV adverts Advert <- cbind(insurance[,2], c(na,insurance[1:39,2])) colnames(advert) <- paste("adlag",0:1,sep="") (fit <- auto.arima(insurance[,1], xreg=advert, d=0)) ## Series: insurance[, 1] ## ARIMA(3,0,0) with non-zero mean ## ## Coefficients: ## ar1 ar2 ar3 intercept AdLag0 AdLag1 ## ## s.e ## ## sigma^2 estimated as : log likelihood= ## AIC=61.78 AICc=65.28 BIC=73.6 y t = x t x t 1 + n t n t = 1.41n t 1 093n t n t 3 Forecasting using R Dynamic regression models 36

51 Example: Insurance quotes and TV adverts Advert <- cbind(insurance[,2], c(na,insurance[1:39,2])) colnames(advert) <- paste("adlag",0:1,sep="") (fit <- auto.arima(insurance[,1], xreg=advert, d=0)) ## Series: insurance[, 1] ## ARIMA(3,0,0) with non-zero mean ## ## Coefficients: ## ar1 ar2 ar3 intercept AdLag0 AdLag1 ## ## s.e ## ## sigma^2 estimated as : log likelihood= ## AIC=61.78 AICc=65.28 BIC=73.6 y t = x t x t 1 + n t n t = 1.41n t 1 093n t n t 3 Forecasting using R Dynamic regression models 36

52 Example: Insurance quotes and TV adverts fc <- forecast(fit, h=20, xreg=cbind(c(advert[40,1],rep(10,19)), rep(10,20))) autoplot(fc) 18 Forecasts from ARIMA(3,0,0) with non zero mean 16 y level Time Forecasting using R Dynamic regression models 37

53 Example: Insurance quotes and TV adverts fc <- forecast(fit, h=20, xreg=cbind(c(advert[40,1],rep(8,19)), rep(8,20))) autoplot(fc) 18 Forecasts from ARIMA(3,0,0) with non zero mean 16 y level Time Forecasting using R Dynamic regression models 38

54 Example: Insurance quotes and TV adverts fc <- forecast(fit, h=20, xreg=cbind(c(advert[40,1],rep(6,19)), rep(6,20))) autoplot(fc) 18 Forecasts from ARIMA(3,0,0) with non zero mean 16 y level Time Forecasting using R Dynamic regression models 39

55 Dynamic regression models y t = a + ν(b)x t + n t where n t is an ARMA process. So φ(b)n t = θ(b)e t or n t = θ(b) φ(b) e t = ψ(b)e t. y t = a + ν(b)x t + ψ(b)e t ARMA models are rational approximations to general transfer functions of e t. We can also replace ν(b) by a rational approximation. There is no R package for forecasting using a general transfer function approach. Forecasting using R Dynamic regression models 40

56 Dynamic regression models y t = a + ν(b)x t + n t where n t is an ARMA process. So φ(b)n t = θ(b)e t or n t = θ(b) φ(b) e t = ψ(b)e t. y t = a + ν(b)x t + ψ(b)e t ARMA models are rational approximations to general transfer functions of e t. We can also replace ν(b) by a rational approximation. There is no R package for forecasting using a general transfer function approach. Forecasting using R Dynamic regression models 40

57 Dynamic regression models y t = a + ν(b)x t + n t where n t is an ARMA process. So φ(b)n t = θ(b)e t or n t = θ(b) φ(b) e t = ψ(b)e t. y t = a + ν(b)x t + ψ(b)e t ARMA models are rational approximations to general transfer functions of e t. We can also replace ν(b) by a rational approximation. There is no R package for forecasting using a general transfer function approach. Forecasting using R Dynamic regression models 40