Least angle regression for time series forecasting with many predictors. Sarah Gelper & Christophe Croux Faculty of Business and Economics K.U.

Transcription

1 Least angle regression for time series forecasting with many predictors Sarah Gelper & Christophe Croux Faculty of Business and Economics K.U.Leuven

2 I ve got all these variables, but I don t know which ones to use. Goals in model selection: accurate predictions interpretable models Review article: Hesterberg, Choi, Meier and Fraley (2008) Statistics Surveys. 2

3 Application: The role of confidence Business and consumer surveys in Europe are carried out in different sectors Business: industry, retail, services and construction Consumers on a monthly basis in every EU country huge data set Which variables should be used for predicting next month s Belgian industrial production growth? 3

4 Outline 1. The LARS algorithm Efron, Hastie, Johnstone, Tibshirani (2004) The Annals of Statistics 2. Generalization for time series: the TS LARS algorithm 3. Simulation results 4. Application: Predicting Belgian industrial production using sentiment surveys 4

5 1. The LARS algorithm Linear forecast model: y = c + β 1 x 1 + β 2 x β p x p + ε. p is large. PROBLEM: Not all predictors are relevant, i.e. many β-coefficients are zero or very small, but we do not know which ones. OLS estimates have high variability when p is large, result: inaccurate forecasts. AIM of the LARS algorithm: Identify the most relevant predictors. Obtain a parsimonious model with good forecast power. 5

6 1. The LARS algorithm Linear forecast model: y = c + β 1 x 1 + β 2 x β p x p + ε. The LARS algorithm: 1. variable ranking 2. variable selection 3. parameter estimation 6

7 1. The LARS algorithm: variable ranking Assumption: all variables are standardized. Without predictors, best fit for y is ŷ = 0, residuals z = y ŷ. Active set A = set with ranked variables = STEP 0: Look for x j with highest cor(y, x j ) x (1) and A = {(1)} STEP 1: Look for γ 1 as small as possible such that cor(z γ 1 x (1), x (1) ) = cor(z γ 1 x (1), x j ) for j A c x (2) and A = {(1), (2)} Update fit ŷ ŷ + γ 1 x (1) and residual z z γ 1 x (1). 7

8 1. The LARS algorithm: variable ranking x2 fit0 fit1 x1 8

9 1. The LARS algorithm: variable ranking From STEP 1: current fit ŷ and active set A = {(1), (2)} STEP 2: Define u 2 the equiangular vector, a linear combination of x (1) and x (2) such that cor(u 2, x (1) ) = cor(u 2, x (2) ) and look for γ 2 as small as possible such that cor(z γ 2 u 2, x (1) ) = cor(z γ 2 u 2, x j ) for j A c x (3) and A = {(1), (2), (3)} Update fit ŷ ŷ + γ 2 u 2 and residual z z γ 2 u 2. 9

10 1. The LARS algorithm: variable ranking x2 x2 u2 fit0 fit1 x1 10

11 1. The LARS algorithm: variable ranking x2 x2 x3 u2 fit0 fit1 x1 fit2 11

12 1. The LARS algorithm Linear forecast model: y = c + β 1 x 1 + β 2 x β p x p + ε. The LARS algorithm: 1. variable ranking 2. variable selection: using information criteria 3. parameter estimation: OLS or shrinkage 12

13 2. Generalization for time series Linear time series forecast model: y t+h = β 0,0 y t β 0,p0 y t p0 + β 1,0 x 1,t β 1,p1 x 1,t p β m,0 x m,t β m,pm x m,t pm + ε t+h. Lagged values of response and predictors are included. Many β-coefficients are zero or very small. TS LARS: Select predictors as blocks of lagged values of the time series, a block is denoted by x j. Use R 2 (y x) = measure of multiple correlation between univariate y and multivariate x. 13

14 2. The TS LARS algorithm: variable ranking Assumption: all time series are standardized. Without predictors, best fit for y is obtained from an AR-model y t+h = β 0,0 y t β 0,p0 y t p0 + ε t+h, denote the fitted values by ŷ and the residuals by z. Active set A = Can the predictors improve the autoregressive fit? 14

15 2. The TS LARS algorithm: variable ranking STEP 0: Look for x j with highest R 2 (z x j ) x (1) and A = {(1)} Define x (1) = fitted values of regression z x (1). STEP 1: Look for γ 1 as small as possible such that R 2 (z γ 1 x (1) x (1) ) = R 2 (z γ 1 x (1) x j ) for j A c x (2) and A = {(1), (2)} Update the fit ŷ ŷ + γ 1 x (1) and the residuals z z γ 1 x (1). Define x (2) = fitted values of regression z x (2). 15

16 2. The TS LARS algorithm: variable ranking From STEP 1: current fit ŷ and active set A = {(1), (2)} STEP 2: Define u 2 a linear combination of x (1) and x (2) such that cor(u 2, x (1) ) = cor(u 2, x (2) ) and look for γ 2 as small as possible such that R 2 (z γ 2 u 2 x (1) ) = R 2 (z γ 2 u 2 x j ) for j A c x (3) and A = {(1), (2), (3)} Update fit ŷ ŷ + γ 2 u 2 and residuals z z γ 2 u 2. Define x (3) = fitted values of regression z x (3). 16

17 2. The TS LARS algorithm Linear time series forecast model: y t+h = β 0,0 y t β 0,p0 y t p0 + β 1,0 x 1,t β 1,p1 x 1,t p β m,0 x m,t β m,pm x m,t pm + ε t+h. The TS LARS algorithm: 1. variable ranking 2. variable & lag-length selection: use BIC! taking the dynamics into account 17

18 2. The TS LARS algorithm Implementation in R available Function TS.LARS(y,x,h) Input y: response x: matrix of predictors h: forecast horizon Output ordered.row: ordered row of predictors k.opt: number of selected predictors p.opt: selected lag length 18

19 3. Simulation Results Simulation setting: y t+1 = β 0,0 y t + β 0,1 y t j=1 β j,0 x j,t + 20 j=1 β j,1 x j,t 1 + ε t+1, 5 relevant predictors 15 redundant predictors lag length = 2 ε t+1 is i.i.d. N(0, 2) length of the time series is simulation runs 19

20 3. Simulation Results Variable ranking performance: Number of ranked relevant predictors TS LARS LARS TS FS Number of ranked predictors 20

21 3. Simulation Results Forecast performance: MSFE h = i=1 (y i,150+h ŷ i,150+h ) 2, h = 1 h = 2 h = 5 TS LARS LARS (< 0.01) (< 0.01) (0.67) TS FS (< 0.01) (< 0.01) (< 0.01) p-values for comparison to TS LARS in parenthesis. 21

22 4. Predicting Belgian industrial production using sentiment surveys Why use confidence indicators for forecasting economic activity? Many macro-economic models include expectations Farmer (1999), Matsasuka & Sbordone (1995). Data availability, timeliness Today, most recent data release on confidence indicators: March 2008 industrial production: February 2008, estimation Eurostat 22

23 4. Predicting Belgian industrial production using sentiment surveys Data set: y = Belgian industrial production growth 75 predictors: 5 sentiment surveys: Consumers Business: industry, retail, services and construction EU15 countries range = [April 1995, October 2007] 23

24 4. Predicting Belgian industrial production using sentiment surveys Top-5 ranking of predictors for one-month-ahead forecasts: Ranking Predictor 1 Industrial Confidence, France 2 Industrial Confidence, Belgium 3 Consumer Confidence, the Netherlands 4 Retail Confidence, Germany 5 Retail Confidence, France 24

25 4. Predicting Belgian industrial production using sentiment surveys Forecast performance MSFE h : h = 1 h = 2 h = 3 h = 6 h = 12 TS LARS LARS (0.01) (0.09) (0.01) (0.89) (0.75) TS FS (0.02) (0.13) (<0.01) (0.56) (0.75) DFM (<0.01) (0.45) (<0.01) (0.89) (0.25) p-values for comparison to TS LARS in parenthesis (Diebold-Mariano test). 25

26 Conclusion The TS LARS algorithm takes the time series dynamics into account. identifies the most relevant predictors. achieves good forecast performance. is fast. is not greedy. allows to identify both short-term and long-term predictors. 26