Frontiers in Forecasting, Minneapolis February 21-23, Sparse VAR-Models. Christophe Croux. EDHEC Business School (France)

Transcription

1 Frontiers in Forecasting, Minneapolis February 21-23, 2018 Sparse VAR-Models Christophe Croux EDHEC Business School (France) Joint Work with Ines Wilms (Cornell University), Luca Barbaglia (KU leuven), and Sarah Gelper (TU Eindhoven). Christophe Croux Sparse VAR-Models 1 / 32

2 Section 1 VAR Models Christophe Croux Sparse VAR-Models 2 / 32

3 10-dimensional time series Log volatilies ETHA NATG BIOD SOYO SUGA GASO WHEA CRUO CORN Time COFF Time Christophe Croux Sparse VAR-Models 3 / 32

4 Network ETHA NATG BIOD GASO CORN CRUO WHEA SUGA SOYO COFF Christophe Croux Sparse VAR-Models 4 / 32

5 Vector Autoregressive Model (VAR) Two stationary time series y 1,t and y 2,t. VAR(1) in dimension q = 2: { y1,t = Γ 1,11 y 1,t 1 + Γ 1,12 y 2,t 1 + e 1t y 2,t = Γ 1,21 y 1,t 1 + Γ 1,22 y 2,t 1 + e 2t Covariance matrix of (e 1t, e 2t ) is Σ. Vector notation: y t = Γ 1 y t 1 + e t, Christophe Croux Sparse VAR-Models 5 / 32

6 The VAR model Let y t be a q-dimensional stationary time series Vector Autoregressive Model of order p: y t = Γ 1 y t 1 + Γ 2 y t Γ p y t p + e t, Matrices Γ j are autoregressive parameters e t error with covariance matrix Σ = Ω 1. Standard estimation procedure: OLS equation by equation. Christophe Croux Sparse VAR-Models 6 / 32

7 Section 2 Sparse VAR Christophe Croux Sparse VAR-Models 7 / 32

8 Log-Volatilies for commodities (weekly data) VAR model for q = 10 time series (and T = 156 observations) One lag 1 (q q) = 100 regression parameters 45 unique elements in Σ Two lags 2 (q q) = 200 regression parameters 45 unique elements in Σ Explosion of number of parameters Christophe Croux Sparse VAR-Models 8 / 32

9 The VAR model: Overparametrization ML estimators will be Not computable Inaccurate Sparse estimation many estimated parameters equal to zero Suitable if T is small relative to the number of parameters Easier to interpret Automatic variable selection Better estimation and prediction performance Christophe Croux Sparse VAR-Models 9 / 32

10 Sparse Estimation: Lasso In the multiple linear regression model Minimization problem y = β 1 x 1 + β 2 x β k x k + ε ˆβ = argmin (y X β) (y X β) + λ β k β l. l=1 Tibshirani (1996) Christophe Croux Sparse VAR-Models 10 / 32

11 Lasso for the VAR model Multiple equations Partial correlation structure of the error term Glasso of Friedman et al. (2008) Dynamic nature of the model Selecting a time series into one of the equations = selecting the variable and all its lags Group lasso (Yuan and Lin, 2006) Christophe Croux Sparse VAR-Models 11 / 32

12 Penalized ML estimation Rewrite the VAR in matrix notation: Y = Y L Γ + E, where Y = (y p+1,..., y T ) Y L = (X p+1,..., X T ) with X t = (y t 1,..., y t p) Γ = (Γ 1,..., Γ p ) E = (e p+1,..., e T ). Christophe Croux Sparse VAR-Models 12 / 32

13 Penalized ML estimation (cont.) Penalized negative log likelihood: ( Γ, Ω) = argmin Γ,Ω 1 ( ) T tr (Y Y L Γ)Ω(Y Y L Γ) log Ω G +λ 1 γ g 2 + λ 2 g=1 k k Ω kk, with γ g is subvector of Γ G = q 2 total number of groups. Christophe Croux Sparse VAR-Models 13 / 32

14 Algorithm Solving for Γ Ω: Γ Ω = argmin Γ 1 ( ) T tr (Y Y L Γ)Ω(Y Y L Γ) G + λ 1 γ g 2. g=1 groupwise lasso Christophe Croux Sparse VAR-Models 14 / 32

15 Algorithm (cont.) Solving for Ω Γ: Ω Γ = argmin Ω 1 ( ) T tr (Y Y L Γ)Ω(Y Y L Γ) log Ω +λ 2 k k Ω kk. penalized inverse covariance estimation (glasso) Christophe Croux Sparse VAR-Models 15 / 32

16 Selection of tuning parameters In the iteration step Γ Ω, select λ 1 to minimize BIC λ1 = 2 log L λ1 + k λ1 log(t ), L λ1 is the estimated likelihood using λ 1 is the number of non-zero estimated regression coefficients. k λ1 In the iteration step Ω Γ, select λ 2 analogously. Christophe Croux Sparse VAR-Models 16 / 32

17 Commodity prices: log volatilities (weekly data) Log volatilies ETHA NATG BIOD SOYO SUGA GASO WHEA CRUO CORN Time COFF Time Christophe Croux Sparse VAR-Models 17 / 32

18 Networks from the VAR coefficients Γ. Network with q nodes. Each node corresponds with a time series. draw an edge from node i to node j if Additionally (if p = 1) P Γ p,ji 0 p=1 the edge width is the size of the effect the edge color is the sign of the effect (blue if positive, red if negative) Christophe Croux Sparse VAR-Models 18 / 32

19 Network on the AutoRegressive coefficients ETHA NATG BIOD GASO CORN CRUO WHEA SUGA SOYO COFF Diebold and Yilmaz (2015) Christophe Croux Sparse VAR-Models 19 / 32

20 Granger Causality Time series i is Granger Causing time series j Time series i it has incremental predictive power in forecasting series j In the network there is an arrow going from node i to node j Granger Causality test in high dimensions: Wilms, Gelper, Croux, 2016 Christophe Croux Sparse VAR-Models 20 / 32

21 Network on the precision matrix ETHA NATG BIOD GASO CORN CRUO WHEA SUGA SOYO COFF Christophe Croux Sparse VAR-Models 21 / 32

22 References Rothman, Levina, and Zhu (2010), Sparse multivariate regression with covariance estimation, Journal of Computational and Graphical Statistics. Hsu, Hung, and Chang (2008), Subset selection for vector autoregressive processes using lasso, Computational Statistics and Data Analysis. Davis, Zang, and Zheng (2016), Sparse vector autoregressive modeling, Journal of Computational and Graphical Statistics. Basu and Michailidis (2015), Regularized estimation in sparse high-dimensional time series models, Annals of Statistics. Nicholson, Matteson, and Bien, J. (2017), VARX-L: Structured regularization for large vector autoregressions with exogenous variables, International Journal of Forecasting. Wilms, Basu, Bien, and Matteson (2017), bigtime: Sparse Estimation of Large Time Series Models, R package.... Gelper S., Wilms I. and Croux C. (2016), Identifying demand effects in a large network of product categories, Journal of Retailing. Christophe Croux Sparse VAR-Models 22 / 32

23 Section 3 Application: a Market Response Model Christophe Croux Sparse VAR-Models 23 / 32

24 Data Sales, promotion and prices for 17 product categories. T = 77 weekly observations VAR model for q = 3 17 = 51 time series Time Christophe Croux Sparse VAR-Models 24 / 32

25 Christophe Croux Sparse VAR-Models 25 / 32

26 Sales Forecasting One-step ahead, using rolling window. Averaged over the 17 product categories, and the 15 stores. Method Mean Absolute Forecast Error Sparse 736 Bayesian: Minnesota 875 Bayesian: Normal-Inverse Wishart 1078 Least Squares 1298 Restricted LS 784 Sparse method significantly better (P < 0.01, Diebold-Mariano test) Christophe Croux Sparse VAR-Models 26 / 32

27 Simulation Study Bayesian methods Minnesota prior (Koop and Korobilis, 2009) Normal-Inverse Wishart prior (Banbura et al, 2010) Simulation Design: Sparse high-dimensional : q = 10, p = 2, T = 50 Christophe Croux Sparse VAR-Models 27 / 32

28 Method Mean Absolute Estimation Error Sparse Bayesian: Minnesota Bayesian: Normal-Inverse Wishart Least Squares Restricted LS Christophe Croux Sparse VAR-Models 28 / 32

29 Section 4 Extensions Christophe Croux Sparse VAR-Models 29 / 32

30 Multi-Class VAR We have a chain of 17 stores. Estimate a VAR for each store separately. Pool the data for the 17 stores in one single class, and estimate one single VAR (or panel-var.) Multi-class estimation, where the data decide what the classes are. Danaher, Wang, and Witten, D. (2014): joint graphical lasso. Tibshirani, Saunders, Rosset, Zhu, and Knight (2005): Fused Lasso Christophe Croux Sparse VAR-Models 30 / 32

31 Multi-class sparse VAR: K classes Γ (k), k = 1,..., K are minimizing K T (y (k) t k=1 t=1 Γ (k) Y L (k) ) Ω(k) (y (k) t Γ (k) Y L (k) ) TJ log Ω (k) + K J P k=1 i,j=1 p=1 Γ (k) p,ij + K J P k k i,j=1 p=1 Γ (k) p,ij Γ(k ) p,ij +... Wilms, I.; Barbaglia, L.; Croux, C. (2018), JRSSS-C Christophe Croux Sparse VAR-Models 31 / 32

32 Furtermore - t-lasso for estimating VAR - cointegration Christophe Croux Sparse VAR-Models 32 / 32