Variable Selection in Predictive MIDAS Models

Transcription

1 Variable Selection in Predictive MIDAS Models Clément Marsilli November 2013 Abstract In the context of short-term forecasting, it is usually advisable to take into account information about current state of economic activity using most recent indicators which however are sampled at different frequencies. In this respect, the MIDAS model has proved to be a useful tool, especially for predicting economic growth. A remaining issue when using MIDAS concerns the choice of explanatory variables which is the focus of this paper. In this work, we propose solutions based on a combined use of MIDAS jointly with dimension reduction techniques. We especially introduce two new methods, the LASSO augmented MIDAS model and the Bayesian MIDAS model with stochastic search variable selection. A cross-validation procedure integrated with these approaches allows insample selection based on recent forecasting performances and hence provides a direct comparison of these methods. As an empirical test of our theoretical model, we aim at predicting the US economic growth using well-known indicators, either daily or monthly sampled, and we examine the forecasting ability of these models within a recursive window framework over the period Keywords Forecasting, MIDAS, LASSO, Bayesian variable selection. Banque de France, International Macroeconomics Division, and Université de Franche-Comté, Laboratoire de Mathématiques de Besançon. clement.marsilli@banque-france.fr.

2 Introduction Short-term analysis aims at providing forecasting based on the available information set and it usually involves required to use data sampled at different frequencies. The Great Recession endured by the main industrialized countries during the period , in the wake of the American subprimes crisis, has encouraged many forecasters to reconsider their model specifications, especially as regards the interactions between financial and macroeconomic variables. In this respect, Foroni and Marcellino (2013) and Banbura et al. (2012) have recently overviewed the existing mixed-frequency models designed for handling ragged edge data and for nowcasting. In this paper, we focus on the Mixed Data Sampling (MIDAS) method that allows the use of high frequency series to explain low frequency variables. Introduced by Ghysels et al. (2004) and more formally conceptualized in Ghysels et al. (2007) and by Andreou et al. (2010), the MIDAS approach constitutes a parsimonious weighting framework for the handling of distributed lags. This approach allows us to explain a a low frequency variable by using exogenous variables sampled at higher frequencies without resorting to any aggregation procedure. It has proved to be particularly suitable in the macroeconomics forecasting context; good performances have been specially observed at the time of predicting the state of the economy and in capturing early signals of turning points using multifrequency explanatory factors. For example, the MIDAS regression has been used to predict quarterly GDP fluctuations using either monthly real economic data or daily financial series; Andreou et al. (2013) and Ferrara et al. (2014) showed that combining information significantly improves the prediction results. Many works certainly prove that an informed selection of explaining variables, regardless of their sampling frequencies, has a major impact on the performance of this forecasting method. In this context, a natural extension would be to define MIDAS model setting with respect to prediction purposes. In economic forecasting, empirical models are generally based on dimension reduction methods coming from either variable or model selection. The difference between these two close schemes is mainly methodological: variable selection aims at an a priori determination of the relevant predictors, while model selection provides a mechanistic approach to combine models which are typically univariate. In this respect, we propose various techniques using either variable or model selection to tackle the so-called curse of dimensionality within a MIDAS forecasting framework. Principal component and factor models are widely used for economic forecasting, we refer among others to Forni et al. (2000) or Stock and Watson (2002). For example, Banerjee and Marcellino (2006) have studied the leading indicators for US inflation and GDP growth using a factor model 2

3 on a large data set. In the context of mixed-frequency models, Marcellino and Schumacher (2010) have put forward a dynamic factor MIDAS model (FAMIDAS) as a way to tackle the lack of parsimony associated to the profusion of covariates. FAMIDAS is a method to incorporate in a MIDAS framework standard tools of factor analysis that usually produce very good results for short term macroeconomic forecasting (see Giannone et al., 2008 or Barhoumi et al., 2010) and hence represents a competitive benchmark when we compare the performance of different models. Nevertheless, some extension to factor analysis have recently emerges in the literature. Castle et al. (2013) have developed an automatic model selection approach to separately or jointly consider factors and variables in a forecasting context. De Mol et al. (2008) have suggested to use Bayesian regressions or penalized regressions (especially the ridge or the LASSO) models as a dimension reduction technique and therefore as an alternative to principal components analysis. Alternative approaches for using mixed frequency data are the bridge models used, for instance, by Barhoumi et al. (2008) for forecasting purposes. In Bencivelli et al. (2012) bridge based techniques are put together with Bayesian Model Averaging (BMA) to combine predictions coming from various model settings. The literature shows that forecast combinations and, in particular, model averaging like the BMA, seem to yield good forecasting results. Basically, pooling forecasts instead of pooling information. Palm and Zellner (1992) have even shown that "in some instances it is sensible to use a simple average of individual forecasts". Barbieri and Berger (2004) proved that BMA outperforms any others Bayesian selection methods in regard to their predictive performances. The Bayesian model selection strategy is completely different from penalized regression, like the LASSO methodology developed by Tibshirani (1996). Indeed, the LASSO does select explanatory variables while BMA can be viewed as a model competition where each model corresponds to a particular choice of variables. In this direction, Zellner (1971) has proposed to exhibit this competition by comparing the ratios of posterior probabilities associated to every possible combination of explanatory variables (this is usually referred to as Bayes factor analysis). Rodriguez and Puggioni (2010) have also recently adapted Bayesian approaches to estimate MIDAS models for forecast exercises. In their paper, BMA provides a way to set the pattern and the slope of the weights applied on the explanatory variables. Unfortunately, searching the best model using this approach involves maximizing marginal likelihoods and hence requires in general the estimation of all 2 n models which may prove to be numerically expensive. In this context, mixture priors on regressor coefficients with spike and slab components extend the Bayesian selection analysis to stochastic search. We refer to Mitchell and Beauchamp (1988) for Dirac point mass spikes or to George and Mcculloch (1993) for absolutely continuous spikes. This technique have been widely exploited in econometrics, see, for example, Korobilis (2013) 3

4 for a empirical use to predict economic growth or Kaufmann and Schumacher (2012) for finding sparsity on factor models. The recent paper of Scott and Varian (2013) also considered spike and slab regression for variable selection for nowcasting economic time series. This paper introduces and compares various selection methods within the mixed-frequency framework for macroeconomic forecasting. We argue that targeted predictors yield better forecasts. Our approach focuses on four dimension reduction techniques that we combine with the MIDAS regression structure. We especially introduce two new methods: (i) the LASSO augmented MI- DAS model, and (ii) the Bayesian MIDAS model with stochastic search variable selection. Those are compared with (iii) the Factor Augmented MIDAS model, and (iv) the combination forecast technique of univariate MIDAS models. A cross-validation procedure integrated with these four approaches allows in-sample selection based on recent forecasting performances, and provides, as a consequence, a direct comparison of these methods. We provide results on the accuracy of forecasts of the US GDP growth from 2000 to 2013 by comparing point forecasts and their relative errors. Our empirical results can be summarized in a few points. First, there is strong evidence for variable selection in improving forecasting performances (in the "normal" period as well as during the crisis). Second, we observe that the four models were able to identify early warnings of the Great Recession from 3 to 6 months ahead. Third, the set of chosen indicators determined by the automated variable/model selection procedure, predictive cross-validation, reflects the varying nature of current economic outlook. Fourth, according to our metrics based on minimum forecasting errors, it seems really hard to distinguish one method among the four we developed. The paper is structured as follows. We present first the theoretical framework of these modeling (section 1). In the section 2, we introduce the predictive cross-validation and its related recursive window framework, we evaluate how selecting explanatory variables out of a universe of wellknown indicators can really improve short-term forecasts. 4

5 1 Variable selection in MIDAS models 1.1 MIDAS regression framework In this paper, we aim at assessing variable selection techniques in the context of mixed frequency data. We consider the MIDAS regression model as our framework. Let y t be the variable of interest sampled at the lowest frequency and let x t be a vector of n time series whose components x κ i t,i are sampled with periods κ i, that is x κ i t,i is quoted 1/κ i times for each quote of y t. multivariate MIDAS regression model is given by: The y t = β 0 + n i=1 β i m K i (θ i, L) x κ i t,i + ε t, (1) Note that, according to the previous equation (1), each explanatory variable can be sampled at its own frequency. This augmented version of the MIDAS regression has been proposed by Ghysels et al. (2007). Recently, Andreou et al. (2013) and Ferrara et al. (2014) used such specifications to combine daily financial variables with monthly indicators. The temporal aggregation technique relies in this case on the MIDAS regression kernel m K ( ) which smooths out the K past values of the variable x t,i by using a functional polynomial of the form m K (θ, L) := K k=1 f(k, θ) K l=1 f(l, θ)l(k 1)κ, (2) where L (k 1)κ is the lag operator. For example, if we have k = 2, κ = 1/3, then L (k 1)κ x κ t = x t 1/3 ; this configuration corresponds to one lag of a monthly sampled indicator. As we could see in the previous expression, the MIDAS regression kernel is defined as a weighted average, where the weights are given by a function f( ) parametrized by θ. There exists different specifications for this weight function in the econometric literature involving linear or non linear schemes. Ghysels et al. (2007) have presented a large panel of possible forms that the MIDAS function can take. In our study, referring to the forecasting literature, we prefer the exponential Almon Lag function which is usually described by two parameters as follows f(k, θ) = f(k, θ 1, θ 2 ) = exp(θ 1 k + θ 2 k 2 ). This MIDAS kernel allows us to limit the dimensionality of the problem by using a parsimonious way to aggregate high frequency data using only two parameters for each regressor. Among 5

6 others, Clements and Galvão (2009) and Kuzin et al. (2009) have showed that those kind of models can be a useful tool for predicting the quarterly economic growth with monthly indicators. However, when using MIDAS, the literature as a whole usually considers only a few number of indicators because of the non linearity of the weight function that can involve difficulties in the estimation of regression parameters. Therefore, we define two variable selection methods within the mixed frequency framework involved by the MIDAS regression. 1.2 The LASSO augmented MIDAS model The LASSO (Least Absolute Shrinkage and Selection Operator) has been introduced by Tibshirani (1996) as a shrinkage and variable selection method that can be applied for different models. The LASSO belongs to the family of penalized regression model which amounts to performing least squares with some additional constraints on the coefficients. Furthermore, Ng (2012) have shown that LASSO tends to have the lowest risk of misspecification in forecasting models when comparing with usual information criteria. In the econometrics setup Bai and Ng (2008) and Schumacher (2010) have proposed to forecast economic series using a combination of factorial analysis with the LARS approach, which is the model selection algorithm derived from the LASSO (see Efron et al., 2004). More specifically, in the context of multivariate linear regression models, the LASSO takes advantage of the sparsifying properties of the l 1 -norm when solving the penalized optimization problem, b = arg min b ( y t b 0 t i = arg min Y Xb λ lasso b 1, b ) 2 b i x t,i + λ lasso b i (3) where y t is the dependant variable, x t is the vector of covariates, b is the vector containing the regression parameters, and λ lasso is the exogenous parameter which controls the strength of the LASSO sparsifying regularization. The LASSO method does indeed reduce the dimension of the explanatory matrix X by driving non informative β i elements to zero. Increasing λ lasso R + brings gradually elements of the β vector to zero, hence selecting relevant explanatory variables. The choice of the exogenous parameter λ lasso that determines the number of covariates that are eliminated is essential and therefore the key thing of those types of modelling. i 6

7 Instead of using l 1 penalty, the ridge is based on penalized regression with quadratic penalty: l 2 norm. Figure 1 illustrates the underlying principle of both techniques, in the case of a multivariate regression model with two variables: b 1 and b 2. The LASSO is on the left, and the ridge regression on the right. b 2 b 2 ˆbLS ˆbLS ˆbridge ˆblasso b 1 b 1 Figure 1: Least square estimate for l 1 penalty (on the left) and l 2 penalty (on the right) The ellipses around the least square estimator, ˆb LS represent the level sets of the squared error function Y Xb 2 2 and the light colored areas correspond to balls of the l 1 and l 2 norms. In view of the expression (3) the solution of the optimization problem that we are interested in takes place at the points in which both surfaces are tangent. The geometry of the problems makes that in the ˆb l 1 case, the solution is generically located at the vertices of the l 1 -balls and hence the LASSO penalized solutions have entries equal to zero. The ridge based solutions are obviously not located at specific points and are hence not necessarily sparse. We now extend the LASSO model to the MIDAS regression context by proposing the optimization problem: [ ˆβ, ˆθ ] = arg min β, θ ( y t β 0 t ) 2 n β i m K i (θ i ) x κ i t,i + λ β i (4) i=1 = arg min Y X(θ) β λ β 1, β, θ i 7

8 where the matrix X(θ) contains the MIDAS specifications that we previously described in (2), 1 m K 1 (θ 1, L) x κ 1 1,1 m Kn (θ n, L) x κn 1,1 X(θ) = (5) 1 m K 1 (θ 1, L) x κ 1 T,n m Kn (θ n, L) x κn T,n Like in the linear case, the l 1 penalization on the β parameters provides a selection of the most relevant predictors. The number of parameters eliminated can be tuned with respect to the value of the exogenous parameter λ. However, a technical complication in solving (4) via any gradient descent method arises due to the non-smooth nature of the l 1 norm. We overcome this difficulty using the local regularization technique due to Nesterov (2005). Further details are described in the Appendix A. The LASSO technique combined with the MIDAS regression presents the advantage to be a one step procedure. That affects directly calculation time since the selection is involved in the estimation process with respect to λ. Indeed, the underlying assumption is that λ is already determined what is not usually the case. In this paper, our goal is to assess in prediction the selection techniques within a mixed frequency framework. In this respect, the parameter λ will be conditioned to the relative forecasting performances of the subset it selected using that we called a predictive cross-validation method. 1.3 Bayesian variable selection in MIDAS models To define the relevant subset of variables which should be involved in the final model, we also exploit a specific Bayesian variable selection technique. Relying on spike and slab priors, stochastic variable selection is an a alternative to usual Bayesian model selection and therefore avoids comparing all 2 n possible models (see George and Mcculloch (1993)). This approach yields a kind of hierarchy on the covariates with respect to posterior distributions and relative inclusion probabilities. Kaufmann and Schumacher (2012) have recently used this technique to find relevant variables in sparse factor models. According to the Bayes formula, the model selection relies on drawing the posterior ordinate. In this respect, we assume that residuals of the MIDAS regression model, as defined in (1), follow a Gaussian distribution N (0, σ). Thus the conditional likelihood function of the MIDAS model 8

9 under study has the following form: f(y β, θ, σ) = [ ] 1 1 exp (2πσ) T/2 2σ (Y X(θ)β) (Y X(θ)β). (6) Regarding (Y X(θ)β), the matrix form of the MIDAS regression, notations are those used in the LASSO context, see equation (4). Bayesian techniques have been rarely used in the context of MIDAS regression model; the main reference on this topic is given by Rodriguez and Puggioni (2010). In their work, they tackle the non linearity of the weight function (exponential Almon in that case) by using a kind of U-MIDAS model a la Foroni and Marcellino (2012) instead of the regular MIDAS of (1). They focused specially on model selection techniques as a way of choosing the relevant number of lags. In our Bayesian framework, model parameters are derived from the posterior density which is, according to the Bayes formula (7), proportional to the likelihood times the prior: π(β, θ, σ Y ) = f(y β, θ, σ) π(β, θ, σ) f(y ) f(y β, θ, σ) π(β, θ, σ). (8) (7) In our study, we only focus on the equation (8). The calculation of the marginal likelihood, namely f(y ), would be required in the case of model selection, in which maximizing the marginal likelihood determines the best model. In the general regression context, the Bayesian literature suggests to use conjugate priors for β and σ. As though we aim at selecting variables, we exploit priors specifications to determine whether a variable should be included or not. The spike and slab priors technique introduced by Mitchell and Beauchamp (1988) basically constraints regressor coefficients to be zero (coefficient drawn from the "spike" prior) or not (drawn from the flat distribution: the "slab" prior). George and Mcculloch (1993) generalised this method, by putting forward the Stochastic Search Variable Selection (SSVS) in linear regression problems via a mixture of two normal distributions as follows: β i h i h i N (0, ϕ 2 ) + (1 h i )N (0, c ϕ 2 ). (9) With respect to the parameter h i, the distribution of each covariate coefficient switches from a density concentrated around zero to an other with density spread out over a larger interval. When h i = 1, β i has a flat distribution, therefore its variable should be included in the model, 9

10 whereas if h i = c (with c 1), the density coefficient is concentrated very close to zero and the variable should not be involved as regressor). Thus, we define the Bernoulli variable with its respective probabilities as follows: 1 with π(h i = 1) = ω i h i = (10) c with π(h i = c) = 1 ω i We set the variance ϕ 2 large to cover a large set of reasonable values. Thus, as an interpretation of those formula, (9) and (10), ω i can be seen as the prior probability that β i need a nonzero estimate, hence x i should be kept in the selected subset. This particular feature of the SSVS has been often reviewed in the literature and more complex choices of prior can be made as well. While George and Mcculloch (1993) defined an hierarchical prior for the inclusion probability using a Beta distribution, Yuan and Lin (2005) have preferred define a hierarchical Bayes formulation to show that it can be related to the LASSO estimator. In our study, we assume that if 0 < ω i < Ω < 1, the relative predictor x i should not be included in the model. In the context of forecasting, this probability threshold Ω [0, 1] will be determined with respect to the forecasting performance of the selection it involves. Like the LASSO, the stochastic search variable selection involved by the use of spike and slab prior yields both estimation and selection in a one-step procedure. In the second section of the paper, we discuss this approach in a forecasting context by using a predictive cross-validation. Many others strategies have been formulated to estimate the spike and slab variable selection method, see among others Ishwaran and Rao (2005) or Malsiner-Walli and Wagner (2011) for a review. The slab component as it has been specified in (9) allows us to consider β under the standard multivariate normal distribution as prior. The prior for the residual variance σ, we employ the usual inverse gamma distribution. We implement a Gibbs sampler to generate an ergodic Markov chain in which all parameters, i.e. (h, ω, β, θ, σ), are embedded. As regards the particular case of the MIDAS parameter θ, we use a Independence Metropolis Hasting algorithm (imh) within the Gibbs sampler to draw the posterior conditional of θ. The candidate posterior distribution involved by the (imh) is defined by a Normal distribution whose mean and covariance matrix are approximate using the maximum likelihood estimator, ˆθ ML. Details on the algorithm are given in Appendix B. Repeating these steps J times, and following a burn-in sample, the algorithm converges, relatively quickly, to a steady state of the Markov chain. The distribution obtained is the target density, i.e. the posterior distribution, which informs us on the selection that can be made with respect to ω. 10

11 2 Forecasting assessment Beyond selecting variables, our approach consists of choosing most efficient predictors, in a mixed frequency framework. Indeed, the assessment of both variable selection methods we describe, namely the LASSO and the Bayesian shrinkage, will be done according to forecasting performances. Thus, we define the variable ξ i as an dummy indicator that tells whether the i th variable is present or not, in the subset of the model: 1 if x i is selected to be present in the model, ξ i = 0 otherwise. The information of including or excluding one explanatory variable is contained in the vector ξ. Rewriting the MIDAS model within a direct forecasting frame, at the horizon h, with such specifications yields to ŷ t+h t = ˆβ 0 + n i=1 ξ i ˆβi m K i (ˆθ i, L) x κ i t,i, (11) where ( ˆβ 0, ˆβ 1,..., ˆβ n, ˆθ 1,..., ˆθ n ) are parameter estimates (usually by either non-linear least squares or maximum likelihood method). That means that the effective size of the explanatory subset relies on this vector ξ. To determine it and, as a consequence, to shape the optimal reduced form of the regression model, we exploit methods of variable selection adapted to the MIDAS model. To fairly compare the empirical results of both models that we put forward in the first section, namely the LASSO-MIDAS and the Bayesian SSVS MIDAS models, we benchmark them against mixed frequency models that allow dimension reduction or feature selection. In this context, the FAMIDAS (namely Factor Augmented MIDAS) model put forward by Marcellino and Schumacher (2010) is obviously advocated. Models that they have suggested in their paper are based on two techniques successively used: first, pooling information from blocks of covariates that share the same frequency into a certain number of factors, and second, tracking the dependent variable with a MIDAS regression model by incorporating these factors as explanatory variables. We also consider forecast combination as a alternative to usual dimension reduction techniques. Indeed, there is a growing literature that proves that combining forecasts is particularly competitive with specific predictive, we refer specially to the very complete survey of Timmermann (2006). This technique has also been tried in the MIDAS context by Andreou et al. (2013) to 11

12 take advantage of their very rich financial data set. Rodriguez and Puggioni (2010) have preferred using hierarchical priors on the model space in their Bayesian model selection framework. In our paper, we have adopted the strategy that variable selection is only driven by its relative forecasting ability. Therefore, our approach aims at improving forecasting performances using the most relevant predictors: selection among predictors is conditioned with respect to its forecasting gain. In this respect, we exploit the idea of the cross-validation technique coupled with prediction purposes. Basically, we assume that the best forecast for the next quarter relies on the same selection that gave the better track of the GDP in the previous ones. 2.1 Predictive cross-validation The direct multi-step forecasting strategy that we adopt implies parameters estimates β (h) and θ (h) that depend on the horizon of the prediction h which is described at the lowest frequency. We use a recursive window framework over the whole out-of-sample period, that entails a time dependency of the selection made by variable selection methods. Dummy variables ξ depend on λ (h) t for the LASSO model, see (4), and on Ω (h) t for the Bayesian SSVS model, see (9) and (10). We assume that the selection is updated according to its predictive error for the d previous quarters. The value of d would have different meanings, e.g. d = 1 tells that we only base the analysis on the last period whereas d = 20 represents the selection that gave best results over the last 5 years. Instead of the usual MSFE (Mean Squared Forecasting Errors), we prefer focusing on an discounted version of this criterion such as Andreou et al. (2013) used in their paper. Indeed, it promotes recent performances by weighting squared residuals according to their historical records. As regards the pseudo out-of-sample period, we opt for a intermediate parametrization that corresponds to forecasting performances over the last year, i.e. d = 4. More specifically, the cross-validation algorithm yields to model specifications that possess best predictive power. We investigate four families of forecasting models that we specify using the predictive cross-validation: the LASSO augmented MIDAS, the Bayesian-MIDAS Stochastic Search, the FAMIDAS, and the forecast combination of univariate MIDAS regressions. Model settings are described below: (i) In the case of the LASSO, we have the following forecasting equation: ŷ t+h t (λ (h) t ) = ˆβ (h) 0 + n i=1 ξ i (λ (h) (h) t ) ˆβ i m K i (ˆθ (h) i, L) x κ i t,i. (lasso-midas) As opposed to Tibshirani (1996), our goal is not to recover an underlying sparsity in the 12

13 coefficients vector β but to use the penalty to reduce the covariates cardinality. The question that arises in this context is the selection of the optimal strength of the l 1 penalty that ensures a favourable forecasting performance. Our cross-validating procedure follows those prescriptions: for a given value λ > 0, we set its corresponding selection by estimating the equation (4), and we forecast ŷ t t h (λ) such as defined in the lasso-midas equation. Then, repeating that for a range of λ, we determine λ t as the one which minimises forecasting residual at time t: λ (h) t = arg min λ t t=t d δ t t ( y t ŷ t t h (λ (h) t ) ) 2, (12) where δ = 0.8 to be coherent with our wish to involve a decrease on the MSFE weight with respect to the historical performance. That allows to promote recent forecasts accuracy. (ii) The Bayesian Stochastic Search variable selection combined with the MIDAS forecasting model is given by: ŷ t+h t (Ω (h) t ) = ˆβ (h) 0 + n i=1 ξ i (Ω (h) (h) t ) ˆβ i m K i (ˆθ (h) i, L) x κ i t,i. (bayesian-midas) The posterior probability ω i as described in (9) and in (10) specifies the probability that β i has not been draw from the spike prior, namely the probability to include it in the model. The issue that arises here is to choose a threshold Ω [0, 1] below which variables are simply removed. Following exactly the same procedure than in the LASSO case, we forecast ŷ t t h (Ω) according to its relative set of selected variables. Then, we set Ω t as the minimum argument of the square error for the period t: Ω (h) t = arg min Ω t t=t d δ t t ( y t ŷ t t h (Ω (h) t ) ) 2. (iii) The FAMIDAS modelling is basically based on a factor structure assumption for the explanatory variables matrix, That can be described as follows: X τ = ΛF τ + η τ, where τ is given in one of the higher frequencies (daily or monthly in our case). The 13

14 components of the factors vector are denoted as F τ = (f 1,τ,..., f r,τ ). This approach consists of using the standard MIDAS technique described in (1) with the r first estimated principal factors that are employed as explanatory variables. The model is given by ŷ t+h t (r (h) ) = r (h) (h) ˆβ 0 + i=1 ˆβ (h) i m K i (ˆθ (h), L) ˆf κ i i t,i (famidas) Since factors are linearly uncorrelated, the size of the factor vector, r, can be determined with a statistical hypothesis test, like Bai and Ng (2008) have proposed. In our study, we propose to define r depending on the forecasting performances. In that case, the parameter we focus on is the number of factors to include in the final model. r (h) t = arg min r t t=t d ( ) δ t t y t ŷ t t h (r (h) 2 t ) Nevertheless, the r factors only represent a family of variables sampled at the same frequency. Since we mix daily and monthly predictors, the number of factors issue arrises for each frequency, thus we have r = (r D, r M ). (iv) Combining forecasts is often referred as a good alternative to model selection. Formally, we compute n individual forecasts respectively based on the i th variable of the entire set, as follows: ŷ t+h t,i = (h) (h) ˆβ 0 + ˆβ i m K i (ˆθ (h) i, L) x κ i t,i. (13) The combination are then made using a weighted average of the individual forecasts (13), thus it can be written as follows: ŷ t+h t (w (h) t ) = n i=1 w (h) t,i ŷ t+h t,i (combination) The forecast relies on the vector of the time-varying combination weights w (h) i,t which can be estimated using several methods; Stock and Watson (2008) show some of those techniques. In this paper, we determine using an equivalent procedure than others selection methods to fairly compare all models. This model relies on the vector of wi,t that weights the individual 14

15 forecasts, see (13). Those are given as follows: w (h) t,i = µ a t,i n j=1 µ a t,j where µ t,i = t t=t d δ t t ( y t ŷ t t h,i ) 2, and a = 2. Using this cross-validation procedure on previous quarters before the forecast stage t + h within the recursive window framework that we describe above, the selection is updated every period of the out-of-sample. This technique lies in an automated model selection procedure that should lead to both better selection of the leading indicators and greater efficiency in their use. 2.2 Data We assess the performance of the four models we have presented above using a forecasting exercise on US GDP data over the period 2000q1-2012q4 while the full sample covers a longer period going from 1964q3 to 2012q4. We denote by GDP t the quarterly US Gross Domestic Product, and by x a matrix containing 24 covariates monthly or daily sampled. In this forecasting exercise, we focus on a set of variables that includes monthly real indicators and daily financial variables which is described in Table 2. Our list includes a daily spread rate, and three financial times series, for which we consider both daily returns and daily volatility. Our set also incorporates seventeen monthly indicators representing the real US economy and coming from "soft" and "hard" data (production index, housing statistics, unemployment rate, opinion survey, etc.). An entire description of the dataset is available in Appendix C. The severe recession has shed light on the necessary re-assessment of the contribution of financial markets to the economic cycles. There is a huge volume of work in the literature that underlines the leading role of financial variables in the forecasting of macroeconomic fluctuations. Recently, Chauvet et al. (2012) and Ferrara et al. (2014) have even shown that daily volatility of financial time series series have a significant forecasting power in explaining US and UK growth. Using variable selection models within the predictive cross-validation we have put forward, we evaluate whether both returns and volatility of financial time series would be included in the model specifications to forecast US GDP growth. Given that volatility is not directly observable, several methods have been developed in the literature to estimate it. Following Ferrara et al. (2014), we use a GARCH model on whitened and winsorized daily financial series. Let ρ τ denotes the daily returns of a given financial time series (τ corresponds to the daily frequency). The GARCH(1,1) 15

16 specification is given by: ρ τ = v τ η τ, (v τ ) 2 = c + a (v τ 1 ) 2 + b ρ 2 τ 1, (14) where {η τ } WN(0, 1). In order to ensure the existence of a unique stationary solution and the positivity of the volatility, we assume that a > 0, b 0, and a+b < 1. Estimated daily volatilities stemming from equation (14) are considered as explanatory variables of the US macroeconomic fluctuations. 2.3 Forecasting results From 2000 to 2013, the US economy faces different phases of the business cycle. In 2008, in the wake of the financial crisis, the United States entered a severe recession, referred to as the Great Recession. The recovery since 2009 was weak and growth remained uneven. Our approach allows to set the horizon at which leading indicators have early information and send warnings of turning point. In this respect, we assess the four MIDAS-based models by splitting our sample in three parts: early 2000 s (from 2000q1 to 2007q2), the Great Recession (from 2007q3 to 2009q4), and the recovery (from 2010q1 to 2012q4). Results of the forecast comparison exercise for GDP growth are discussed below. Table 1 reports the mean squared forecasting errors in the three periods. Model inclusion for all horizons is exhibited in the Appendix D. 16

17 2000q1-2012q4 2000q1-07q2 2007q3-09q4 2010q1-12q4 Full sample Early 2000 s Great Recession Recovery h = 0 (Nowcasting) lasso-midas 0,34 0,33 0,45 0,21 bayesian-midas 0,32 0,29 0,52 0,21 famidas 0,33 0,28 0,73 0,14 combination 0,38 0,33 0,82 0,12 h = 3 lasso-midas 0,37 0,32 0,79 0,13 bayesian-midas 0,40 0,37 0,79 0,15 famidas 0,40 0,27 1,14 0,12 combination 0,42 0,33 0,94 0,22 h = 6 lasso-midas 0,52 0,34 1,24 0,18 bayesian-midas 0,48 0,37 1,14 0,19 famidas 0,46 0,30 1,39 0,13 combination 0,42 0,30 0,99 0,27 h = 9 lasso-midas 0,55 0,35 1,53 0,23 bayesian-midas 0,47 0,31 1,19 0,21 famidas 0,52 0,33 1,65 0,15 combination 0,42 0,32 1,07 0,16 h = 12 lasso-midas 0,54 0,31 1,61 0,22 bayesian-midas 0,66 0,38 2,10 0,18 famidas 0,52 0,29 1,76 0,15 combination 0,44 0,30 1,12 0,27 Table 1: MSFE (Mean Squared Forecasting Errors) Nowcasting At the short horizons, the indicators chosen by both variable selection techniques, the LASSO as well as the Bayesian SSVS, are primarily related to the real economic activity (production, labor market, housing, consumption). This stylized fact has been observed in empirical papers pointing out the increasing role of hard indicators on macroeconomic forecasts when we are close to the release date. In addition, at this horizon, the financial volatility of the SP500 was among the best predictors (always included in both predictor set). Best performances in nowcasting the Great Recession were given by both variable selection methods. Those indicate that financial instability, especially observed in volatility variables 17

18 and commodity price indices, triggered confusion and fear among consumers and firms. Lower confidence and lower stock price yield to a net decrease in consumption in the current period and hence in GDP growth. These findings are in agreement with the combination model in which IPI and the ISM PMI survey are particularly important during this period. The 3-month to 6-month ahead horizons At the 6-month-ahead horizon, financial variables emerge as the most useful indicators. Spread rate and stock price volatility dominate the top ranks in model inclusion. A key difference between pure nowcasting at 0-month-ahead and 3-month-ahead forecasts is that at the latter horizon, IPI variables are not very prominent. This result is interesting for practitioners in the sense that using industrial production index at this horizon does not appear useful. We also note that the LASSO tends to select variables that are not encompassed by other indicators. In fact, the LASSO would prefer substitution in spite of complementarity, usually involved by the Bayesian shrinkage and the combination. Regarding performances in predicting the Great Recession, these four models have captured early warnings from 6 to 3 months ahead. By the end of 2007, serious short-term risks were looming: uncertainty on financial markets (captured by stocks price volatility), bank loan contraction, rising interest rates. We note that model inclusion of those indicators in both variable selection and an increasing weight in the combination. From early 2010 to the end of 2011, while financial indices remain high, the recovery was slower than expected, it was referred in the literature to as the sluggish recovery. Both FAMIDAS and combination models show this disconnection by either not including anymore the daily factor or by reducing their weights in the regression. Finally, by the end of 2011, in the wake of the sovereign debt crisis, some financial indicators (spread rate, corporate bonds and stocks volatility) were again chosen. The 9-month to 12-month ahead horizons At the 9-month ahead horizon, we note that financial volatility indicators already played an important role in forecasting, especially over the early 2000 s and the recovery, as already noticed for the 6-month-ahead forecasts. In addition at this horizon, we get complementary information from money related variables such currency component of M1, M2 and St Louis monetary base. We also find that inflation rate (CPI) is chosen by both variable selection and highlighted as one the main indicator in the combination model, at both 9-month and 12-month ahead horizons. Findings during the Great Recession period should be interpreted with care since forecasting 18

19 errors were really high. Indeed, from 12 to 9 months ahead, it turns out that the four models provided flat predictions, and hence did not yield informative contents to anticipate the crisis. Summary According to our results of the forecast comparison exercise for GDP growth, four main conclusions can be drawn. First, we note that over the whole sample and in spite of adding autoregressive elements, forecasting errors for all models decrease when the forecasting horizon tends to zero. Second, results significantly differ depending on the period. More specifically, we observe that the four models were not able to anticipate the Great Recession. Early warnings of this complex economic crisis were really identify from 3 to 6 months ahead. In fact, models we have studied in general tend to perform best with short horizons although in some cases the performance extends to three or four quarters. Third, although model specifications are based on a recurrent cross-validation procedure and therefore can evolve over time, results are mostly robust from the variable selection technique. A few stylized facts have been summarized as regards the set of predictors and the forecasting horizon. Indeed, the automated variable/model selection procedure based on predictive cross-validation involve the optimal out-of-sample horizon for each variable. The set of chosen indicators includes reasonable variables from an economic point of view and reflects the varying nature of current economic outlook. Fourth, our forecasting exercise on GDP growth proves that pooling indicators ability provides very reliable models. Observed individually over their respective primary horizons, some of indicators would already give very good results, grouping yields even better performances and minimum forecast errors. Conclusion This paper considers variable selection in short-term macroeconomic forecasting. From the perspective of prediction, we ground our analysis on a mixed frequency framework, the MIDAS, that allows the use of any available data regardless of their sampling frequency. We specially develop in this context, four tools to identify leading indicators of the US GDP growth using an automatic model selection procedure based on recent best performances. More specifically, we introduce a LASSO augmented MIDAS model and a Bayesian MIDAS stochastic search variable selection that we compare with the Factor Augmented MIDAS model, and the combination forecast technique of univariate MIDAS models. Those are combined with a predictive crossvalidation methodology which uses a recursive window and specifies the set of predictors with 19

20 respect to their ability. Dimension reduction methods that we use, goes beyond point forecast and highlights the leading role of some indicators in macroeconomics. We prove that targeted predictors yield better forecasts. Our findings emphasize the role of daily financial information in predicting GDP and show that combining daily and monthly indicators increases the forecasting accuracy. The generic question we addressed focuses on variable selection in predictive mixed frequency models. Forecasting GDP is only one of many examples where our methods can be applied. It would be interesting to extend this approach to the issue of forecasting recessions by binary choice models. References Andreou, E., Ghysels, E., and Kourtellos, A. (2010). Regression models with mixed sampling frequencies. Journal of Econometrics, 158(2): Andreou, E., Ghysels, E., and Kourtellos, A. (2013). Should macroeconomic forecasters use daily financial data and how? Journal of Business and Economic Statistics, 31(2): Bai, J. and Ng, S. (2008). Forecasting economic time series using targeted predictors. Journal of Econometrics, 146(2): Banbura, M., Giannone, D., Modugno, M., and Reichlin, L. (2012). Now-Casting and the Real- Time Data Flow. Banerjee, A. and Marcellino, M. (2006). Are there any reliable leading indicators for US inflation and GDP growth? International Journal of Forecasting, 22(1): Barbieri, M. M. and Berger, J. O. (2004). Optimal predictive model selection. Annals of Statistics, 32(3): Barhoumi, K., Benk, S., Cristadoro, R., Reijer, A. D., Jakaitiene, A., Jelonek, P., Rua, A., Rünstler, G., Ruth, K., Nieuwenhuyze, C. V., and Jakaitiene, A. (2008). Short-term forecasting of GDP using large monthly datasets: a pseudo real-time forecast evaluation exercise. Working paper, European Central Bank. 20

21 Barhoumi, K., Darné, O., and Ferrara, L. (2010). Are disaggregate data useful for factor analysis in forecasting French GDP? Journal of Forecasting, 29(1-2): Bencivelli, L., Marcellino, M., and Moretti, G. (2012). Selecting predictors by using Bayesian model averaging in bridge models. Castle, J. L., Clements, M. P., and Hendry, D. F. (2013). Forecasting by factors, by variables, by both or neither? Journal of Econometrics, null(null). Chauvet, M., Senyuz, Z., and Yoldas, E. (2012). What does financial volatility tell us about macroeconomic fluctuations? Working paper, Federal Reserve Board. Clements, M. P. and Galvão, A. B. (2009). Forecasting US output growth using Leading Indicators: An appraisal using MIDAS models. Journal of Applied Econometrics, 24(7): De Mol, C., Giannone, D., and Reichlin, L. (2008). Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components? Journal of Econometrics, 146(2): Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. Annals of Statistics, 32(2): Ferrara, L., Marsilli, C., and Ortega, J.-P. (2014). Forecasting growth during the Great Recession: is financial volatility the missing ingredient? Economic Modelling, 36: Forni, M., Hallin, M., Lippi, M., and Reichlin, L. (2000). The Generalized Dynamic-Factor Model: Identification And Estimation. The Review of Economics and Statistics, 82(4): Foroni, C. and Marcellino, M. (2012). A comparison of mixed approaches for modelling euro area macroeconomic variables. Foroni, C. and Marcellino, M. (2013). A survey of Econometrics methods for mixed frequency data. George, E. I. and Mcculloch, R. E. (1993). Variable Selection Via Gibbs Sampling. Journal of the American Statistical Association, 88(423): Ghysels, E., Santa-clara, P., and Valkanov, R. (2004). The MIDAS touch : Mixed data sampling regression models. Technical Report 919, mimeo. 21

22 Ghysels, E., Sinko, A., and Valkanov, R. (2007). MIDAS regressions: Further results and new directions. Econometric Reviews, 26(1): Giannone, D., Reichlin, L., and Small, D. (2008). Nowcasting: The real-time informational content of macroeconomic data. Journal of Monetary Economics, 55(4): Ishwaran, H. and Rao, J. S. (2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33(2): Kaufmann, S. and Schumacher, C. (2012). Finding relevant variables in sparse Bayesian factor models: Economic applications and simulation results. Korobilis, D. (2013). Hierarchical shrinkage priors for dynamic regressions with many predictors. International Journal of Forecasting, 29(1): Kuzin, V., Marcellino, M., and Schumacher, C. (2009). MIDAS vs. Mixed-Frequency VAR:Nowcasting GDP in the Euro Area. Discussion paper, CEPR. Malsiner-Walli, G. and Wagner, H. (2011). Comparing Spike and Slab Priors for Bayesian Variable Selection. Austrian Journal of Statistics, 40(4): Marcellino, M. and Schumacher, C. (2010). Factor MIDAS for nowcasting and forecasting with ragged-edge data: A model comparison for German GDP. Oxford Bulletin of Economics and Statistics, 72(4): Mitchell, T. J. and Beauchamp, J. J. (1988). Bayesian Variable Selection in Linear Regression. Journal of the American Statistical Association, 83(404): Nesterov, Y. (2005). Smooth minimization of non-smooth functions. Math. Program., 103(1, Ser. A): Ng, S. (2012). Variable Selection in Predictive Regressions. Palm, F. C. and Zellner, A. (1992). To combine or not to combine? issues of combining forecasts. Journal of Forecasting, 11(8): Rodriguez, A. and Puggioni, G. (2010). Mixed frequency models: Bayesian approaches to estimation and prediction. International Journal of Forecasting, 26(2): Schumacher, C. (2010). Factor forecasting using international targeted predictors: The case of German GDP. Economics Letters, 107(2):

23 Scott, S. L. and Varian, H. R. (2013). Bayesian variable selection for nowcasting economic time series. Stock, J. H. and Watson, M. W. (2002). Forecasting Using Principal Components From a Large Number of Predictors. Journal of the American Statistical Association, 97: Stock, J. H. and Watson, M. W. (2008). Phillips Curve Inflation Forecasts. Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society, 58(1): Timmermann, A. (2006). Forecast combinations. In Elliott, G., Granger, C. W., and Timmermann, A., editors, Handbook of Economic Forecasting, chapter 4. Elsevier edition. Yuan, M. and Lin, Y. (2005). Efficient Empirical Bayes Variable Selection and Estimation in Linear Models. Journal of the American Statistical Association, 100(472): Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. Wiley. A Nesterov regularization technique Let us consider the following regression model: β = arg min β ( y t β 0 t i ) 2 β i x t,i + λ lasso β i (15) i where y t is the dependant variable, x t is the vector of covariates, β is the vector containing the regression parameters, and λ lasso is the exogenous parameter which controls the strength of the LASSO sparsifying regularization. To overcome the estimation of the problem 15, we use the following local regularization technique of Nesterov (2005). We start by noting that the l 1 norm can be expressed using the function g defined as g(β) = β 1 = max γ γ β. 1 23

24 Then, we define the function g µ such that g µ g with respect to µ 0 and µ > 0. We have: g µ (β) := max γ 1 γ β µ 2 γ 2 2, The Nesterov regularization technique consists of replacing the norm g(β) = β 1 by g µ (β) with µ small. The advantage of proceeding in this fashion is that the function g µ is obviously smooth with a gradient g µ (β) whose components are given by sign(β i ) if β i > µ, i g µ (β) = 1 µ β i if β i < µ. B MCMC algorithm To estimate the SSVS-MIDAS model, we implement a Gibbs sampler with respect to feature specifications. The algorithm relies on a few steps which successively sampling h from the spike and slab prior, the hyperparameter ω from a Beta distribution, β and σ from the usual Normal- Inverse Gamma prior, and θ from a candidate generating density using an Independence chain Metropolis-Hastings algorithm. Given initial values for all unknown parameters, the algorithm iteratively updates their values by sampling from their conditional distribution and hence constructing a Markov chain with an invariant distribution. The algorithm is constructed as follows: 1. Sample h i, i = 1,..., n, π(h i β i, ω i ) = (1 ω)π(β i ; 0, cϕ 2 )I {hi =c} + ωπ(β i ; 0, ϕ 2 )I {hi =1}, 2. Sample ω from B(c 0 + n 1, d 0 + n n 1 ), where n 1 = i I {h i =1} 3. Sample β i N (a n, A n ) where A 1 n 4. Sample σ IG(s n, S n ) = 1 σ X(θ) X(θ) + D 1, a n = A n X(θ)Y σ, and D = diag(φ 2 h i ) where s n = s 0 + T 1 2, and S n = 1 2 (Y X(θ)β) (Y X(θ)β) 24

25 5. Sampling θ using an independence chain Metropolis-Hasting algorithm. The acceptance probability α to change to the new value θ new drawn from the candidate density, determines whether the chain moves from areas of low posterior probability to high. It can be written as: [ ] π(θ = θ new y) ι(θ = θ old ) α = min ι(θ = θ new ) π(θ = θ old y), 1. To define the candidate generating density ι, we use an approximation based on the asymptotic normality of the maximum likelihood estimator ˆθ ML, and on its asymptotic variance-covariance matrix var(ˆθ ML ) = I(θ) 1. We compute the Fisher information matrix I(θ) = E 2 ( ) θ θ log f(y β, θ, σ), using numerical differentiation procedures to obtain the approximate variance: var(ˆθ ML ). Thus, we set the candidate generating density as ι(θ) = f T (θ ˆθ ML, var(ˆθ ML )) since we approximate the posterior by a multivariate normal distribution with mean ˆθ ML and covariance matrix var(ˆθ ML ). Draw u U(0, 1). If u < α, retain the new candidate θ new θ = θ old. by setting θ = θ new, otherwise Repeating times these 5 steps yields the chain to converge to a steady state. The posterior distribution will allow us to determine the selection with respect to ω. The MATLAB code will be available soon in my website: 25

26 C Data set Daily series 10y-3m Spread rate: 10y Treasury Rate - 3m Treasury Bill daily CRB CRB Spot index, commodities price index daily log DJ Dow Jones industrial share price index daily log SP500 S&P500 index daily log CRBvolat CRB Spot index, commodities price index daily volatility (see 14) DJvolat Dow Jones industrial share price index daily volatility (see 14) SP500volat S&P500 index daily volatility (see 14) Monthly series AAA Moody Yield on Seasoned Corporate Bonds AAA monthly log AMBSL St Louis Adjusted Monetary Base monthly log BAA Moody Yield on Seasoned Corporate Bonds BAA monthly log BusLoans Commercial and Industrial Loans at Commercial banks monthly log CPI Consumer Price Index for all Urban Consumers: All items monthly log Curr Currency component of M1 monthly log DSPIC Real Disposable Personal Income monthly log Housing New privately owned housing units started monthly log IPI Industrial Production Index monthly log Loans Loans and leases in bank credit, all commercial banks monthly log M2 M2 money stock monthly log Oil Spot oil price: WTI monthly log PCE Personal Consumption Expenditures monthly log PMI ISM manufacturing survey: PMI composite index monthly level PPI Producer Price Index: all commodities monthly log TotalSL Total consumer credit owned and securitized outstanding monthly log Unemploy. Unemployment rate monthly Table 2: US data set from 1964:1 to 2012:4 D Empirical results 26

27 Figure 2: Point forecasts (top) and squared errors (bottom) for h = 0 Figure 3: Point forecasts (top) and squared errors (bottom) for h = 3 27

28 D.1 Selection for h = 0 D.1.1 Bayesian-MIDAS (h = 0) Figure 4: Variable selection from 2000q1 to 2012 q4 with the Bayesian-MIDAS model D.1.2 LASSO-MIDAS (h = 0) Figure 5: Variable selection from 2000q1 to 2012 q4 with the LASSO-MIDAS model 28

29 D.1.3 FAMIDAS (h = 0) Figure 6: Variable selection from 2000q1 to 2012 q4 with the FAMIDAS model D.1.4 Forecast combinations (h = 0) Figure 7: Weights for each variable of the combination from 2000q1 to 2012 q4 29

30 D.2 Results for h = 3 D.2.1 Bayesian-MIDAS (h = 3) Figure 8: Variable selection from 2000q1 to 2012 q4 with the Bayesian-MIDAS model D.2.2 LASSO-MIDAS (h = 3) Figure 9: Variable selection from 2000q1 to 2012 q4 with the LASSO-MIDAS model 30

31 D.2.3 FAMIDAS (h = 3) Figure 10: Variable selection from 2000q1 to 2012 q4 with the FAMIDAS model D.2.4 Forecast Combinations (h = 3) Figure 11: Weights for each variable of the combination from 2000q1 to 2012 q4 31

32 D.3 Results for h = 6 D.3.1 Bayesian-MIDAS (h = 6) Figure 12: Variable selection from 2000q1 to 2012 q4 with the Bayesian-MIDAS model D.3.2 LASSO-MIDAS (h = 6) Figure 13: Variable selection from 2000q1 to 2012 q4 with the LASSO-MIDAS model 32

33 D.3.3 FAMIDAS (h = 6) Figure 14: Variable selection from 2000q1 to 2012 q4 with the FAMIDAS model D.3.4 Forecast combinations (h = 6) Figure 15: Weights for each variable of the combination from 2000q1 to 2012 q4 33

34 D.4 Results for h = 9 D.4.1 Bayesian-MIDAS (h = 9) Figure 16: Variable selection from 2000q1 to 2012 q4 with the Bayesian-MIDAS model D.4.2 LASSO-MIDAS (h = 9) Figure 17: Variable selection from 2000q1 to 2012 q4 with the LASSO-MIDAS model 34

35 D.4.3 FAMIDAS (h = 9) Figure 18: Variable selection from 2000q1 to 2012 q4 with the FAMIDAS model D.4.4 Forecast combinations (h = 9) Figure 19: Weights for each variable of the combination from 2000q1 to 2012 q4 35

36 D.5 Results for h = 12 D.5.1 Bayesian-MIDAS (h = 12) Figure 20: Variable selection from 2000q1 to 2012 q4 with the Bayesian-MIDAS model D.5.2 LASSO-MIDAS (h = 12) Figure 21: Variable selection from 2000q1 to 2012 q4 with the LASSO-MIDAS model 36

37 D.5.3 FAMIDAS (h = 12) Figure 22: Variable selection from 2000q1 to 2012 q4 with the FAMIDAS model D.5.4 Forecast combinations (h = 12) Figure 23: Weights for each variable of the combination from 2000q1 to 2012 q4 37