Methods for Computing Marginal Data Densities from the Gibbs Output

Transcription

1 Methods for Comuting Marginal Data Densities from the Gibbs Outut Cristina Fuentes-Albero Rutgers University Leonardo Melosi London Business School May 2012 Abstract We introduce two estimators for estimating the Marginal Data Density MDD from the Gibbs outut. Our methods are based on exloiting the analytical tractability condition, which requires that some arameter blocks can be analytically integrated out from the conditional osterior densities. This condition is satisfied by several widely used time series models. An emirical alication to six-variate VAR models shows that the bias of a fully comutational estimator is sufficiently large to distort the imlied model rankings. One of the estimators is fast enough to make multile comutations of MDDs in densely arameterized models feasible. Keywords: Marginal likelihood, Gibbs Samler, time series econometrics, Bayesian econometrics, recirocal imortance samling. JEL Classification: C11, C15, C16, C32 Corresondence: Cristina Fuentes-Albero: Deartment of Economics, 75 Hamilton Street, Rutgers University, New Brunswick, NJ 08901: cfuentes@economics.rutgers.edu. Leonardo Melosi: London Business School, Regent s Park, Sussex Place, London NW1 4SA, United Kingdom: lmelosi@london.edu. We thank Frank Schorfheide, Jesús Fernández-Villaverde, Francesco Ravezzolo, Lucrezia Reichlin, Paolo Surico, Herman van Dijk, Daniel Waggonner, the associate editor, and two anonymous referees for very helful comments. We thank seminar articiants at the 4th International Conference on Comutational and Financial Econometrics, the Rimini Bayesian Econometrics Worksho, the 26th Annual Congress of the Euroean Economic Association, the XXXIII SAE-Zaragoza, the 2011 Greater New York Metroolitan Area Econometrics Colloquium, and the 10th Alied Time Series Econometrics Worksho of the St Louis Fed. We also thank Marzie Taheri Sanjani for research assistence.

2 1 Introduction Modern macroeconometric methods are based on densely arameterized models such as vector autoregressive models VAR or dynamic factor models DFM. Densely arameterized models deliver a better in-samle fit. It is well-know, however, that such models can deliver erratic redictions and oor out-of-samle forecasts due to arameter uncertainty. To address this issue, Sims 1980 suggested to use riors to constrain arameter estimates by shrinking them toward a secific oint in the arameter sace. Provided that the direction of shrinkage is chosen accurately, it has been shown that densely arameterized models are extremely successful in forecasting. This exlains the oularity of largely arameterized models in the literature Stock and Watson, 2002, Forni, Hallin, Lii, and Reichlin, 2003, Koo and Porter 2004, Korobilis, forthcoming, Banbura, Giannone, and Reichlin, 2010 and Koo, The direction of shrinkage is often determined by maximizing the marginal likelihood of the data see Carriero, Kaetanios and Marcellino, 2010 and Giannone el al., 2010, also called marginal data density MDD. The marginal data density is defined as the integral of the likelihood function with resect to the rior density of the arameters. In few cases, the MDD has an analytical reresentation. When an analytical solution for this density is not available, we need to rely on comutational methods, such as the Chib s method Chib, 1995, Imortance Samling estimators Hammersley and Handscomb, 1964, Kloek and Van Dijk, 1978, Geweke, 1989, estimators based on the Recirocal Imortance Samling rincile Gelfand and Dey, 1994, imortance samling based on mixture aroximations Frühwirth-Schantter, 1995, the Bridge Samling estimator Meng and Wong, 1996, or the War Bridge Samling estimator Meng and Schilling, Since all these methods rely on comutational methods to integrate the model arameters out of the osterior density, their accuracy deteriorates as the dimensionality of the arameter sace grows large. Hence, there is a tension between the need for using broadly arameterized models for forecasting and the accuracy in estimating the MDD which influences the direction of shrinkage. This aer aims at mitigating this tension by introducing two estimators henceforth, Method 1 and Method 2 that exloit the information about models analytical structure. While Method 1 can be considered as a refinement of the aroach roosed by Chib 1995, Method 2 is based uon the Recirocal Imortance Samling rincile as in Gelfand and Dey Conversely to fully comutational methods, Method 1 and Method 2 rely on 1

3 the analytical integration of some arameter blocks 1. The roosed estimators can be alied to econometric models satisfying two conditions. The first condition henceforth, samling condition requires that the osterior density can be block-artitioned so as to be aroximated via the Gibbs samler. The second condition henceforth, analytical tractability condition states that there exists an integer τ 2 such that the conditional osterior θ 1,..., θ τ θ τ+1,..., θ s, D, Y can be analytically derived, where Y is the samle data, D is a set of unobservable model variables, and s is the total number of arameter blocks θ i, i {1,..., s}. These two conditions are met by a wide range of models, such as Vector AutoRegressive Models VARs, just-identified Structural VAR models SVARs, Reduced Rank Regression Models such as Vector Equilibrium Correction Models VECMs, unrestricted Markov-Switching VAR models MS VARs, Dynamic Factor Models DFMs, Factor Augmented VAR models FAVARs, and Time-Varying Parameter TVP VAR models. By means of a Monte Carlo exeriment, we show that exloiting the analytical tractability condition leads to sizeable gains in accuracy and comutational burden, which quickly grow with the dimensionality of the arameter sace of the model. We consider VAR models, in the form studied by Villani 2009 and Del Negro and Schorfheide 2010 i.e., the socalled mean-adjusted VAR models, from one u to four lags, = 1,..., 4. We fit these four VAR models, under a single-unit-root rior Sims and Zha, 1998, to data sets with increasing number of observable variables. It is comelling to focus on mean-adjusted VAR models because the true conditional redictive density 2 can be analytically derived in closed form. We can comare the erformance of our estimators with their fully comutational counterarts; that is to say the estimator roosed by Chib 1995 and that introduced by Gelfand and Dey Method 1 and Chib s method only differ in the comutation of the 1 Fiorentini, Planas, and Rossi 2011 use Kalman filtering and Gaussian quadrature to integrate scale arameters out of the likelihood function for dynamic mixture models. 2 If one artitions the arameter sace Θ into s vector blocks; that is Θ = {θ 1,..., θ s }, the conditional redictive density Y θ τ+1,..., θ s is defined as Y θ τ+1,..., θ s Y θ 1,..., θ s θ 1,..., θ τ θ τ+1,..., θ s dθ 1...dθ τ where Y θ 1,..., θ s is the likelihood function and θ 1,..., θ τ θ τ+1,..., θ s is the rior for the first τ arameter blocks conditional on the remaining blocks. Note that the conditional redictive density is a comonent of the MDD, Y, that can be exressed as follows: Y = Y θ τ+1,..., θ s θ τ+1,..., θ s dθ τ+1...dθ s where θ τ+1,..., θ s is the rior for the arameter blocks that cannot be analytically integrated out. 2

4 conditional redictive density when alied to mean-adjusted VAR models. While Method 1 evaluates the exact analytical exression for the conditional redictive density, Chib s method aroximates this density comutationally via Monte Carlo integration. Therefore, we can quantify the accuracy gains associated with exloiting the analytical tractability condition by comaring the conditional redictive density estimated by Chib s method with its true value. This assessment would have not been ossible, if we had based our Monte Carlo exeriment on models that require data augmentation to aroximate the osterior, such as DFMs, or on other estimators rather than Chib s method, such as the Bridge Samling estimator. The main findings of the exeriment are: i the fully-comutational estimators that neglect the analytical tractability condition lead to an estimation bias that severely distorts model rankings; ii our two methods deliver very similar results in terms of osterior model rankings, suggesting that their accuracy is of the same order of magnitude in the exeriment; iii exloiting the analytical tractability condition revents our estimators from being affected by the curse of dimensionality i.e., comuting time growing at faster ace as the number of lags or observables in the model increases. Related to this last finding, we argue that Method 2 is suitable for erforming model selection and model averaging across a large number of models, as it is the fastest. The aer is organized as follows. Section 2 introduces the conditions that a model has to satisfy in order to aly our two estimators. In this section, we describe the two methods roosed in this aer for comuting the MDD. Section 3 erforms the Monte Carlo alication. Section 4 concludes. 2 Methods for Comuting the Marginal Data Density The marginal data density MDD, also known as the marginal likelihood of the data, is defined as the integral taken over the likelihood with resect to the rior distribution of the arameters. Let Θ be the arameter set of an econometric model and Y be the samle data. Then, the marginal data density is defined as Y = Y ΘΘdΘ 1 where Y Θ and Θ denote the likelihood and the rior density, resectively. 3

5 In Section 2.1, we describe the two methods roosed in this aer in a canonical situation consisting of four vector blocks. In Section 2.2, we resent the two estimators alied to the general case of s vector blocks. Finally, Section 2.3 deals with the scoe of alication of the roosed estimators. 2.1 Four Vector Blocks Let us consider a model whose set of arameters and latent variables is denoted by Θ D = {D, Θ} where D stands for the latent variables and Θ for the arameters of the model, where Θ = {θ 1, θ 2, θ 3 }. We denote the rior for model s arameters as Θ, which is assumed to have a known analytical reresentation. Furthermore, the likelihood function, Y Θ, is assumed to be known in closed form or easy to evaluate. We focus on models satisfying the following two conditions: i It is ossible to draw from the conditional osterior distributions θ 1 θ 2, θ 3, D, Y, θ 2 θ 1, θ 3, D, Y, θ 3 θ 1, θ 2, D, Y, and from the osterior redictive density, D θ 1, θ 2, θ 3, Y. ii The conditional osterior distribution θ 1, θ 2 θ 3, D, Y is analytically tractable. Condition i imlies that we can aroximate the joint osterior Θ Y and the redictive density D Y through the Gibbs samler. We label this condition as the samling condition. Condition ii is the analytical tractability condition and is most likely to be satisfied through a wise artitioning of the arameter sace and the secification of a conjugate rior. Method 1 is based on interreting the MDD as the normalizing constant of the joint osterior distribution Y = Y Θ Θ θ 1 θ 2, θ 3, Y θ 2 θ 3, Y θ 3 Y 2 where the numerator is the roduct of the likelihood and the rior, with all integrating constants included, and the denominator is the osterior density of Θ. Denote the osterior mode as Θ = [ θ1, θ 2, θ ] 3. Hereafter, let denote a density for which an analytical exression is available and denote a density that needs to be aroximated using comutational 4

6 methods. Method 1 is obtained by factorizing 2 as follows: M1 Y = Y θ 3 θ3 θ 3 Y where θ3 is the rior for the arameter block θ 3 evaluated at the osterior mode, the conditional osterior θ3 Y is aroximated using the Rao-Blackwellization technique roosed by Gelfand, Smith, and Lee 1992, and the conditional redictive density, Y θ 3, is defined as: Y θ 3 = 3 Y Θ θ1, θ 2 θ 3 θ1, θ 4 2 θ 3, Y Note that Y Θ is the likelihood evaluated at the osterior mode and θ1, θ 2 θ 3 is the rior for the blocks θ 1 and θ 2 conditional on θ 3 evaluated at the osterior mode. denominator can be evaluated as follows: θ1, θ 2 θ 3, Y = 1 m m θ1, θ 2 θ 3, D i, Y i=1 where the conditional osterior θ1, θ 2 θ 3, D i, Y can be exactly calculated because of the analytical tractability condition and { D i} m is the outut of a lower dimensional Gibbs i=1 samler usually called reduced Gibbs ste. The { reduced Gibbs } ste delivers draws from the m density D θ 3, Y by iteratively drawing θ i 1, θ i 2, D i from the conditional osterior distributions θ 1 θ 2, θ i=1 3, D, Y and θ 2 θ 1, θ 3, D, Y and from the redictive density D θ 1, θ 2, θ 3, Y. It should be noted that Method 1 is a refinement of the estimator roosed by Chib 1995, whose only difference with Method 1 is the comutation of the conditional osterior distribution θ1, θ 2 θ 3, Y in the denominator of 4. Since Chib s method does not exloit the analytical tractability condition, it estimates this conditional osterior by taking the roduct of θ1 θ 2, θ 3, Y and θ2 θ 3, Y. This imlies that two reduced Gibbs stes need to be erformed to evaluate the denominator of 4: i one to obtain draws from the density D θ 2, θ 3, Y so as to evaluate θ1 θ 2, θ 3, Y and ii another one to obtain draws from the density D, θ 1 θ 3, Y so as to evaluate θ2 θ 3, Y. While Chib s estimator erforms two reduced Gibbs stes, Method 1 only requires one because of the exloiting of the analytical The 5 5

7 tractability condition. Therefore, note that, by construction, Method 1 is more accurate and less comutationally burdensome than Chib s estimator. Method 2 is based on combining the analytical tractability condition with the Recirocal Imortance Samling RIS rincile roosed by Gelfand and Dey The marginal data density is given by Y = [ ] 1 θ 1, θ 2 θ 3, D, Y E D,θ3 Y f θ 3 6 Y θ 1, θ 2, θ 3 θ 1, θ 2 θ 3 θ 3 where E D,θ3 Y denotes the exectations taken with resect to the osterior density D, θ 3 Y and f is a weighting function with the roerty f θ 3 dθ 3 = 1. Therefore, Method estimates the marginal data density as follows: ˆ M2 Y = 1 m m i=1 θ1, θ 2 θ i Y θ 1, θ 2, θ i 3 3, D i, Y θ1, θ 2 θ i 3 θ i 3 f { } m where θ i 3, D i are the draws from the Gibbs samler simulator. The numerator is i=1 the conditional osterior, which is known because of the analytical tractability condition. In the denominator, we have the roduct of the likelihood and the joint rior, with all integrating constants included. Note that the standard RIS estimator roosed by Gelfand and Dey 1994 uses all the osterior draws for θ 1, θ 2, θ 3, D, which makes it a global estimator. By exloiting the analytical tractability condition, Method 2 relies on setting the first two arameter blocks equal to the osterior mode and using only the osterior draws for θ 3 and D. Therefore, Method 2 is a hybrid estimator: local for θ 1, θ 2 and global for θ 3, D. 3 θ i General Case Let us consider an s-block arameter vector, Θ {θ 1,..., θ s }. We assume that the rior distribution, Θ, is known and the likelihood function, Y Θ, is either known in closed 3 If there are no latent data, Method 2 becomes a global estimator since equation 7 becomes M2 y = 1 m m i=1 Y θ i 3 1 θ i 3 f where Y θ i 3 is the conditional redictive density, which is insensitive to evaluation of {θ 1, θ 2 }. θ i 3 1 6

8 form or easy to evaluate. written as: The two necessary conditions to aly our estimators can be i Samling condition: It is ossible to draw from the conditional osterior distributions θ i Θ i, D, Y, where Θ i {θ 1,.., θ i 1, θ i+1,..., θ s }, for any i {1,... s} and from the osterior redictive density, D Θ, Y. ii Analytical tractability condition: The conditional osterior distributions Θ τ Θ >τ, D, Y, where Θ τ {θ 1,..., θ τ } and Θ >τ {θ τ+1,..., θ s }, are analytically tractable, for some τ {2,..., s}. Method 1 is given by where Θ>τ M1 Y = Y Θ >τ Θ>τ Θ >τ Y is the rior for the arameter blocks θ τ+1,.., and θ s and the conditional redictive density is comuted as follows Y Θ >τ = 8 Y Θ Θ τ Θ >τ 9 Θ τ Θ >τ, Y The analytical tractability condition allows us to comute the denominator of 9 as follows Θ τ Θ >τ, Y = 1 m m Θ τ Θ >τ, D i, Y i=1 where { D i} { } m m is the outut of a reduced Gibbs ste that iteratively draws Θ i i=1 τ, Di i=1from the known distributions θ i θ 1,.., θ i 1, θ i+1,..., θ τ, Θ >τ, D, Y, for 1 i τ and the redictive density D Θ τ, Θ >τ, Y. The conditional osterior at the denominator of 8 can be estimated as 4 Θ>τ Y = s τ i=1 θτ+i Θ >τ+i, Y, where the ordinates θτ+i Θ >τ+i, Y, for 1 i < s τ, can be aroximated by running s τ 1 reduced Gibbs stes and the smallest ordinate θs Y can be aroximated via Rao-Blackwellization based on draws from the Gibbs samler. 4 Conventionally Θ >m =. 10 7

9 Method 2 comutes the marginal data density, Y, as follows: ˆ M2 Y = 1 m m i=1 Θ τ Θ i Y Θ τ, Θ i >τ >τ, D i, Y Θ τ Θ i >τ Θ i >τ f Θ i >τ 1 11 { } m where Θ i >τ, D i are the draws from the Gibbs samler simulator. It should be noted i=1 that when τ = s i.e., all the arameter blocks can be integrated out analytically, we have that Θ >τ =, which imlies that Method 1 and Method 2 coincide. 5 To sum u, alying Method 1 requires running s τ reduced Gibbs stes as oosed to the s 1 stes erformed by Chib s method. 6 Thus gains from alying Method 1 relative to Chib s method are exected to become more and more substantial as the number of blocks τ that can be integrated out increases. Nevertheless, Method 1 overlas Chib s method when erforming reduced Gibbs stes for i {τ + 1, s 1}. Note that these simulations are the most comutationally cumbersome among all the reduced Gibbs stes erformed by Chib s method because they are the ones which integrate out the largest number of arameter blocks. When the total number of arameter blocks, s, is much larger than the number of blocks that can be integrated out, τ, Method 1 may still be comutationally cumbersome. In these cases, and when a large number of reeated comutations of MDDs is required e.g., Bayesian averaging over a large number of models, Method 2 rovides the fastest aroach. It is imortant to emhasize that Method 2 only requires erforming the Gibbs samler osterior simulator, no reduced Gibbs ste has to be erformed. 2.3 Scoe of Alication Unlike Chib and Gelfand-Dey estimators, our methods are only alicable when both the samling and the analytical tractability conditions are met. But both conditions can be shown to be satisfied by a large class of time series econometrics models. In articular, we can show that the conditions are met by Vector Autoregressive VAR models, just-identified 5 We thank an anonymous referee to oint this out. 6 Note that when there is no data augmentation, Method 1 requires running one reduced Gibbs ste less, that is, s τ 1. To see why note that the analytical tractability condition imlies that the conditional osterior Θ τ Θ >τ, Y is known when no data augmentation is required. As far as Chib s estimator is concerned, note that the largest ordinate θ 1 Θ >1, Y is usually analytically tractable in many alications e.g., the Monte Carlo exeriment in this aer that do not require data augmentation, imlying that the actual number of reduced Gibbs stes to be erformed is s 2. 8

10 Structural VAR 7, Reduced Rank Regression RRR models, unrestricted Markov-switching VAR models, Dynamic Factor Models DFMs, Factor Augmented VAR models FAVARs, and Time-Varying Parameter TVP VAR model. We exlore in detail the alication to VAR models in the next section. In the aendix, we rovide a guide on how to artition the arameter sace so that the samling and the analytical tractability conditions are satisfied in the remaining models. 3 A Monte Carlo Exeriment In this section, we assess the gains in accuracy and comutational burden of the estimators roosed in the aer by means of a Monte Carlo exeriment. In section 3.1, we describe the modeling framework and the alication of the four estimators used in the exeriment, that is, Chib s estimator, Method 1, Method 2, and Gelfand and Dey s estimator. We discuss the data set and the riors used in the emirical alication in section 3.2. We quantify the gains in accuracy and comutational burden associated with our estimators in sections 3.3 and 3.5, resectively. In section 3.4, we rovide evidence on the ervasive effects that the estimation bias, linked to neglecting the analytical tractability condition, may have on distorting osterior model rankings. 3.1 The Model Following Villani 2009 and Del Negro and Schorfheide 2010, the VAR model in meanadjusted form can be exressed as Y = DΓ + Ỹ 12 Ỹ = XΦ + ε 13 where we denote the samle length as T and [ we define the T n matrix of observables Y = y 1,..., y T, the T l + 1 matrix D = 1 T, 1,..., T,..., 1,..., T l ] with 1 T being a 1 T vector of ones, the l + 1 n matrix Γ = γ 0,..., γ l, the T n matrix of the 7 Let Ω be an orthonormal matrix through which the econometrician secifies the identification restrictions for the VAR. It directly follows that if i the identification scheme does not imose restrictions on the reduced-form arameters and ii the conditional distribution of the matrix Ω does not get udated by the data; then our two estimators are alicable. These conditions are satisfied by recursive VARs and some non-recursive VARs identified with short-run or long-run restrictions. 9

11 de-trended and de-meaned observables Ỹ is defined as Ỹ = ỹ 1,..., ỹ T, the T n matrix X = x 1,..., x T, where we define the n 1 vectors x t = ỹ t 1,..., ỹ t, the n n arameter matrix Φ = [φ 1,..., φ ], and the T n matrix of Gaussian residuals is denoted as ε = ε 1,..., ε T whose covariance matrix is denoted by Σ. We consider thee arameter blocks: the block for the mean and the deterministic trend Γ, the block for the autoregressive arameters Φ of the VAR in deviations, and the arameters of the covariance matrix Σ for the VAR in deviations. The block order is chosen such that θ 1 = Φ, θ 2 = Σ, and θ 3 = Γ. Note that, conditional on the arameter block Γ, the equations can be interreted as a Multivariate Linear Gaussian Regression Model. Therefore, under rior conjugacy, the osterior distribution Φ, Σ Γ, Y is analytically tractable belonging to the Multivariate-Normal-Inverted-Wishart MN IW family. This suffices to guarantee the satisfaction of the analytical tractability condition for τ = 2. Moreover, if the rior for Γ is indeendent and Gaussian, the conditional osterior Γ Φ, Σ, Y can be shown to be also Gaussian see the online aendix. Therefore, the samling condition is satisfied. Since the samling and analytical tractability conditions are satisfied with τ = 2, Method 1 comutes the MDD as follows M1 Y = Y Γ Γ Γ Y where the conditional redictive density, Y Γ, has a closed-form exression. For instance, when the rior for the arameters of the VAR in deviations Φ, Σ Γ is a dummy-observation rior, the conditional redictive density can be shown to be given by Y Γ = π T 0 +T 1 nn 2 X X n 2 S T 0 +T 1 n 2 Γ T0 +T 1 n n π T 0 nn 2 X X n 2 S T 0 n 2 Γ T0 n n where Y and X are matrices that stack dummy observations for the VAR in deviations; Y and X are the data in deviations obtained by de-meaning and de-trending the actual data Y with Γ; T 0 is the number of dummy observations; T 1 is the total number [ of observations ] ; T 1 = T + T 0 ; n is the number of variables; is the number of lags; Y = Y, Ỹ X = [ X, X ] ; Γn is the multivariate gamma function; S = Y X Φ Y X Φ with Φ = X 1 X X Y ; and S = Y X Φ Y X Φ with Φ = X X 1 X Y. Finally, 10

12 the marginalized osterior Γ Y in the denominator of 14 is comuted imlementing a Rao-Blackwell strategy. A naïve alication of Chib s method disregards the fact that the conditional redictive density Y Γ has a known analytically exression and comutes CHIB Y Γ = Y Σ, Φ, Γ Σ, Φ Γ Φ Σ, Γ, Y Σ Γ, Y 16 where Σ Γ, Y is aroximated comutationally using the outut from the reduced Gibbs ste as follows Σ Γ, Y 1 m m Σ Φ i, Γ, Y 17 i=1 Method 2 comutes: [ 1 m f Γ i ] 1 ˆ M2 Y = 18 m Y Γ i Γ i i=1 where the draws Γ i are the draws from the Gibbs samler simulator 8. We analytically evaluate the osterior kernel Y Γ i Γ i and the weighting function f Γ i for each draw of Γ. The alication of Gelfand and Dey s method henceforth, the GD method to the model is straightforward and hence omitted. In the Monte Carlo exercise, we use the weighting function f roosed by Geweke 1999 when imlementing Method 2 and GD method. 9 The degree of freedom of the weighting function is chosen so as to minimize the numerical standard error of the estimator. In what follows, we set Φ, Σ, and Γ to be equal to the osterior mean In order to imlement this aroach, we need the draws {Γ} m i=1 from the marginalized osterior Γ Y. It should be clear that these draws are simly the set of draws {Γ} m i=1 that come from the outut of the Gibbs samler. 9 However, note that while the weighting function for Method 2 is defined over the sace of the vector block Γ, the one for the GD estimator is defined over the entire arameter sace. 10 The results of the exeriment are virtually the same if Φ, Σ, and Γ are set to be equal to the osterior median. 11

13 3.2 Data, Prior Secification, and Number of Simulations We fit four VAR models with different lags to six encomassing data sets. In articular, we fit autoregressive models with lags = 1,..., 4 to data sets containing from one u to six variables, which are in order: Real Gross Domestic Product source: Bureau of Economic Analysis, GDPC96, Imlicit Price Deflator source: Bureau of Economic Analysis, GDPDEF, Personal Consumtion Exenditures source: Bureau of Economic Analysis, PCEC, Fixed Private Investment source: Bureau of Economic Analysis, FPI, Effective Federal Funds Rate source: Board of Governors of the Federal Reserve System, FEDFUNDS, and Average Weekly Hours Duration in the Non-farm Business source: U.S. Deartment of Labor, PRS The encomassing data sets are such that the one-variate models consider the real GDP data, the two-variate models, GDP and the rice deflator, and so on an so forth until the six-variate models, which contain all data series listed above. The quarterly data set ranges from 1954:1 to 2008:4. We elicit the rior density for the arameters of the VAR in deviations, Φ, Σ, by using the single-unit-root rior, suggested by Sims and Zha We follow Del Negro and Schorfheide 2004, Giannone, Lenza, and Primiceri 2010, and Carriero, Kaetanios, and Marcellino 2010 setting the hyerarameters of the rior so as to maximize the conditional redictive density, Y Γ, where Γ stands for the osterior mean. 11 To this end, we erform a stochastic search based on simulated annealing Judd, 1998 with 1,000 stochastic draws. Furthermore, the rior density deends on the first and second moments of some re-samle data. We use the moments of a re-samle ranging from 1947:1 to 1953:4. We run ten chains of m number of draws in the Gibbs samler and in the reduced Gibbs samler, where m = {100, 1, 000, 10, 000, 100, 000}. We also run one chain with one million draws. 3.3 Gains in Accuracy Our estimators rely on the insight that exloiting the analytical tractability condition increases the accuracy of MDD estimators. In this emirical alication, we assess the inaccuracy associated with neglecting the analytical tractability condition. Consider the VAR 11 Very similar results are found using the rocedure roosed by Banbura, Giannone, and Reichlin 2010 that automatically adjusts the rior hyerarameters as the number of observables is increased. For any number of observables from three to six, we set the hyerarameters so that the fit of the VAR4 in deviations in the resamle 1947:1-1953:4 is comarable with that of the trivariate VAR1 estimated with the OLS. The values for the hyerarameters obtained with this rocedure are very similar to those comuted by maximizing the conditional redictive density, Y Γ 12

14 model of the form In this framework, Method 1 differs from Chib s method only on the comutation of the conditional redictive density, Y Γ. While Method 1 exactly calculates the conditional redictive density Y Γ via its analytical exression, Chib s method aroximates it comutationally via equation 16, which requires erforming the reduced Gibbs ste to imlement the integration in 17. Thus, the inaccuracy derived from neglecting the analytical tractability condition can be quantified by the ga between the estimated conditional redictive density using Chib s aroach, CHIB Y Γ, and its true value, Y Γ. Note that, as the number of draws in the reduced Gibbs ste, m, goes to infinity, the size of the ga goes to zero, that is, lim m CHIB Y Γ = Y Γ. In this alication, we assess the convergence of Chib s method to the true conditional redictive density by comuting log CHIB Y Γ log Y Γ 19 We refer to this difference as the estimation bias for the conditional redictive density. We comute the absolute difference in 19 for every chain, VAR model = 1,..., 4, and data set. The uer grah of Figure 1 reorts the across-chain mean of the estimation bias for the conditional redictive density for the 24 models of interest when erforming 1,000,000 draws in both the Gibbs samler and the reduced Gibbs ste. We find worth emhasizing the following two results. First, for a given number of lags, the estimation bias grows at an increasing rate as the number of observable variables increases. Second, for a given number of observables, the estimation bias grows at an increasing rate as the number of lags increases. For examle, the size of the ga for a six-variate VAR4 is about 9 times the size of the bias for the VAR1 model. We document in Table 1 the convergence of the estimation bias as the number of draws in the reduced Gibbs ste increases for six-variate VAR models. We conclude that for a given data set and a given model, the bias is quite stable desite the increase in the number of osterior draws in the reduced Gibbs ste. This suggests that the integration in 17 exhibits a rather slow convergence. The comarison of Method 2 and the GD method is not as straightforward as the one between Method 1 and Chib s estimator. Table 2 reorts the across-chain means and standard deviations of the log MDD for each of the estimators and models for the six-variate data set. The GD method is found to be both biased and quite unstable since the across-chain standard deviations are larger than that for Method 2. In comlex models, such as the VAR4, the standard deviation of the GD method is 17 log-oints when 100,000 osterior draws are used, while that of Method 2 is 0.16 log-oints. The large instability of the GD 13

15 method exlains its limited use in VAR models. 3.4 Model Selection In this section, we analyze the effect of inaccurate estimates when erforming Bayesian model selection. Under a 0-1 loss function, the otimal decision is to select the model with the largest osterior robability see Schorfheide Let us define the model set to be formed by the four VAR models, that is, {V AR, 1 4} 12 in the six-variate data set. We assume that the rior model robabilities, {π,0, 1 4}, are the same across the four candidate models. For every estimator, we ermute MDDs estimated at each chain across the four VAR models which delivers 10, 000 quadrulets of osterior robabilities. The distributions of the osterior robabilities associated with the VAR1 and the VAR2 for Chib s estimator, Method 1, and Method 2 are a mass oint at zero, suggesting that these methods strongly disfavor both the VAR1 and the VAR2. The GD method rarely selects the VAR1 or the VAR2. Therefore, in Figure 2, we only reort the distributions for the 10, 000 osterior robabilities comuted by the four estimators for the VAR3 and VAR4 models. While both Method 1 and Method 2 lead to select the VAR4, the distribution related to Chib s method imlies a median osterior robability of about 20% for the VAR4. Conversely, Chib s method strongly favors the VAR3 model with a median osterior robability of about 80%. These results show that the estimation bias due to a fully comutational aroach may significantly distort model rankings. Finally, the distributions related to the GD method are uniform for both models, which makes it imossible to make inference over models. Two imortant remarks about Figure 2 are in order. First, since Method 1 and Chib s estimator differ only in how they calculate the conditional osterior Σ Γ, Y, the observed bias in model ranking must be due to the inaccuracy associated with the integration 17, which is based on the reduced Gibbs ste. Second, although Method 1 and 2 estimate the MDD through different aroaches, 13 these two methods deliver osterior model rankings that are remarkably similar. Hence, the accuracy of the two methods roosed in the aer is of the same order of magnitude. 12 We have extended the exercise to include VAR5 and VAR6. We have decided to not resent them in the aer because all estimators deliver very small MDDs for these two models. Hence, all the results discussed in this section are unchanged. 13 Recall that Method 1 exloits the fact that the MDD can be exressed as the normalizing constant of the joint osterior density for model arameters. In contrast, Method 2 relies on the rincile of the recirocal imortance samling. 14

16 3.5 Comutation Time Figure 3 shows how the comutation time in seconds associated with each of the estimators under analysis varies as the number of observable variables and the number of lags, increases. Comaring these figures, we conclude that Method 2 and GD method are comutationally more convenient than Method 1 and Chib s method for any model secification and any data set. We observe that for Method 2 i the comuting time is almost invariant to the number of lags included in the model and ii the increases in comuting time due to the inclusion of additional observable variables are quite small. Quite remarkably, estimating the MDD associated with a six-variate VAR4 with the Method 2 and 100,000 osterior draws, 14 takes less that 1/10 seconds. While the comutation burden of the GD method is quite reduced, it increases exonentially with the dimension of the model, that is, it suffers from the curse of dimensionality. In the lower grah of Figure 1, we exlore the difference in comuting time between Chib s method and Method 1. Recall that the these two estimators only differ in how they calculate the conditional osterior Σ Γ, Y. Hence, the figure shows how the comuting time to erform the reduced Gibbs ste changes as the number of lags or observables in the VAR model varies. We conclude that Chib s method suffers of the curse of dimensionality because of the reduced Gibbs ste. These findings suggest that exloiting the analytical tractability condition breaks the curse of dimensionality that characterizes both Chib s estimator and the GD method. 4 Concluding Remarks The aer develos two new estimators for the marginal likelihood of the data. These estimators rely on the fact that in several widely used time series models it is generally ossible to analytically integrate out one or more arameter blocks from the block-conditional osterior densities imlied by the models. An alication based on a standard macroeconomic data set reveals that our estimators translate into significant gains in accuracy and comutational burden when comared to fully-comutational aroaches. We find that the estimation bias associated with fully-comutational estimators may severely distort model rankings. Furthermore, our estimators do not suffer the curse of dimensionality that affects , 000 draws ensure very reliable estimates since the size of the across-chain standard deviation is relatively small. 15

17 the fully-comutational method. In articular, Method 2 is fast enough to be well-suited for alications where the marginal likelihood of VAR models has to be comuted several times e.g., Bayesian selection or average across a large set of models. The aer favors the idea that estimators that are tailored to the secific features of an econometric model are likely to dominate universal estimators, which are alicable to a broader set of models but rely on fully comutational methods. Using estimators that exloit the information about the analytical structure of the model is very rewarding, esecially for densely arameterized models. Furthermore, as we overview in the aendix, estimators that exloit the analytical structure of models to imrove accuracy can be easily obtained for many oular time series models. The assessment of the accuracy gains that can be obtained from alying artly analytical estimators to oular time-series models, such as TVP VAR models, FAVAR models, and DFMs, is an imortant venue for future research. The results of the aer should encourage the develoment of new estimators that exloit the analytical structure of more involved models such as, for instance, restricted MS VAR models e.g., Sims and Zha, 2006 and Sims, Waggoner and Zha, 2008 or over-identified structural VAR models e.g., Waggoner and Zha,

18 References Banbura, M., D. Giannone, and L. Reichlin 2010: Large Bayesian Vector Auto Regressions, Journal of Alied Econometrics, 251, Bernanke, B. S., J. Boivin, and P. Eliasz 2005: Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive FAVAR Aroach, Quarterly Journal of Economics, 1201, Carriero, A., G. Kaetanios, and M. Marcellino 2010: Forecasting Government Bond Yields with Large Bayesian VARs, CEPR Discussion Paer No Carter, C., and R. Kohn 1994: Biometrika, 813, On Gibbs Samling for State Sace Models, Chib, S. 1995: Marginal Likelihood from the Gibbs Outut, Journal of the American Statistical Association, 90432, Del Negro, M., and F. Schorfheide 2004: Priors from General Equilibrium Models for VARS, International Economic Review, 452, : Bayesian Macroeconometrics, in The Handbook of Bayesian Econometrics, ed. by H. K. van Dijk, J. F. Geweke, and G. Koo. Oxford University Press. Fiorentini, G., C. Planas, and A. Rossi 2011: The marginal likelihood of dynamic mixture models: some new results, Mimeo. Forni, M., M. Halllin, M. Lii, and L. Reichlin 2003: Do Financial variables hel forecasting inflation and real activity in the Euro Area?, Journal of Monetary Economics, 50, Frühwirth-Schnatter, S. 1995: Bayesian model discrimination and Bayes factors for linear Gaussian state sace models, Journal of Royal Statistical Society B, 57, Gelfand, A. E., and D. K. Dey 1994: Bayesian Model Choice: Asymtotics and Exact Calculations, Journal of the Royal Statistical Society B, 56, Gelfand, A. E., A. F. M. Smith, and T.-M. Lee 1992: Bayesian Analysis of Constrained Parameter and Truncated Data Problems Using Gibbs Samling, Journal of the American Statistical Association, 87418,

19 Geweke, J. 1989: Bayesian Inference in Econometric Models using Monte Carlo Integration, Econometrics, 57, : Bayesian Reduced Rank Regression in Econometrics, Journal of Econometrics, 751, Geweke, J. F. 1999: Using Simulation Methods for Bayesian Econometric Models: Inference, Develoment and Communication, Econometric Reviews, 18, Giannone, D., M. Lenza, and G. Primiceri 2010: Prior Selection for Vector Autoregressions, mimeo. Hammersley, M. J., and D. C. Handscomb 1964: Monte Carlo Methods. Methuen, London. Judd, K. L. 1998: Numerical Methods in Economics. The MIT Press, Boston. Kloek, T., and H. K. van Dijk 1978: Bayesian Estimates of Equation System Parameters: an Alication of Integration by Monte Carlo, Econometrica, 461, Koo, G. 2011: Forecasting with Medium and Large Bayesian VARs, mimeo University of Strathclyde. Koo, G., and S. Potter 2004: Forecasting in Dynamic Factor Models Using Bayesian Model Averaging, Econometric Journal, 72, Korobilis, D. forthcoming: Forecasting in Vector Autoregressions with many redictors, Advances in Econometrics, Vol 23: Bayesian Macroeconometrics. Meng, X.-L., and S. Shilling 2002: War Bridge Samling, Journal of Comutational and Grahical Statistics, 11, Meng, X.-L., and W. H. Wong 1996: Simulating Ratios of Normalizing Constants Via a Simle Identity: A Theoretical Exloration, Statistica Sinica, 6, Pitt, P. G. M. K., and R. Kohn 2010: Bayesian Inference for Time Series State Sace Models, in The Handbook of Bayesian Econometrics, ed. by H. K. van Dijk, J. F. Geweke, and G. Koo. Oxford University Press. Primiceri, G. 2005: Time Varying Structural Vector Autoregressions and Monetary Policy, Review of Economic Studies, 723,

20 Schorfheide, F. 2000: Loss Function-Based Evaluation of DSGE Models, Journal of Alied Econometrics, 156, Sims, C. A. 1980: Macroeconomics and Reality, Econometrica, 484, Sims, C. A., D. F. Waggoner, and T. Zha 2008: Methods for Inference in Large Multile-Equation Markov-Switching Models, mimeo. Sims, C. A., and T. Zha 1998: Bayesian Methods For Dynamic Multivariate Models, International Economic Review, 394, Sims, C. A., and T. Zha 2006: Were There Regime Switches in US Monetary Policy?, American Economic Review, 961, Stock, J., and M. Watson 2002: Macroeconomic Forecasting Using Diffusion Indexes, Journal of Business and Economic Statistics, 20, Villani, M. 2009: Steady State Priors for Vector Autoregressions, Journal of Alied Econometrics, 244, Waggoner, D. F., and T. Zha 2003: A Gibbs samler for structural vector autoregressions, Journal of Economic Dynamics & Control, 28,

21 A. Figures and Tables Figure 1: Estimation bias Estimation bias Difference in Seconds 5 4 Estimation bias log-oints Number of observables VAR1 VAR2 VAR3 VAR Time Differences Between Chib's Method and M Number of Observables VAR1 VAR2 VAR3 VAR4 20

22 Figure 2: Distribution of Posterior Probabilities for VAR 21

23 Comuting time in seconds Comuting time in seconds Comuting time in seconds Comuting time in seconds Figure 3: Comuting time in seconds 3000 Comuting Time - Chib's Method 100,000 draws in the Gibbs samler and in the reduce-gibbs ste 350 Comuting Time - Method 1 100,000 draws in the Gibbs samler Number of Observables Number of Observables VAR1 VAR2 VAR3 VAR4 VAR1 VAR2 VAR3 VAR Comuting Time - Gelfand and Dey Estimator 100,000 draws in the Gibbs samler Number of Observables Comuting Time - Method 2 100,000 draws in the Gibbs samler Number of Observables VAR1 VAR2 VAR3 VAR4 VAR1 VAR2 VAR3 VAR4 22

24 Table 1: across-chain averages of the estimation bias for the conditional redictive density: six-variate VAR Draws VAR1 VAR2 VAR3 VAR , , , , 000, Notes: Across-chain means of absolute differences. Numerical standard errors in italics. Draws refers to the number of osterior draws and the number of draws in the reduced Gibbs ste. For one million draws, we do not reort numerical standard errors. 23

25 Table 2: Log-Marginal Data Density: Six-variate case Model Draws MDD estimator Chib Method 1 Method 2 GD VAR VAR VAR VAR Notes: Draws refers to the number of osterior draws and the number of draws in the reduced Gibbs ste. Across-chain standard deviations are reorted within brackets. 24

26 B. A Guide to use Method 1 and Method 2 B.1 Reduced Rank Regression Models A reduced rank regression model reads: detailed in Y = XΓ + ZΦ + u t 20 with u t iid N 0, Σ. X is an n k matrix, Γ is L, Z is n, and Φ is k L. The matrix of coefficients, Φ is full-rank, but the matrix Γ, is assumed to have rank q, where q < max {L, }. Let us rearameterize the low-rank matrix as Γ = ΨΩ and assume a normalization scheme restricting Ψ = Ψ. Under an inverted Wishart distribution for Σ and indeendent Gaussian shrinkage riors for each of the elements of Ψ and Ω, Geweke 1996 shows that the conditional redictive densities Φ Σ, Ψ, Ω, Y, Σ Ψ, Ω, Y, Ψ Φ, Σ, Ω, Y, and Ω Φ, Σ, Ψ, Y belong to the MN IW family. Therefore, the samling condition is satisfied. Conditional on Γ, the RRR model in 20 reduces to a multivariate linear Gaussian regression model. Given a MN IW rior on Φ, Σ Γ, we conclude that the osterior Φ, Σ Γ, Y is analytically tractable. Let us artition the arameter sace of the RRR model in 20 as follows θ 1 = Φ, θ 2 = Σ, θ 3 = Ψ, and θ 4 = Ω. Hence, the analytically tractability condition is satisfied for τ = 2. B.2 Unrestricted Markov-Switching MS VARs Let us consider the model y t = x tφ K t + u t 21 where Φ K t = [Φ 1 K t,..., Φ K t, Φ c K t ], y t is a n 1 vector of observable variables, and u t N 0, Σ K t. K t is a discrete M-state Markov rocess with time-invariant transition robabilities π lm = P [K t = l K t 1 = m], l, m {1,..., M}. For simlicity, let us assume that M = 2. Let T be the samle length, K = K 1,..., K T be the history of regimes, [Φ j, Σ j] j {1,2} = {Φ 1, Σ 1, Φ 2, Σ 2}, and π jj j {1,2} = {π 11, π 22 }. Let us artition the arameter sace of the model as follows θ 1 = π jj j {1,2}, θ 2 = Φ 1, θ 3 = Σ 1, θ 4 = Φ 2, θ 5 = Σ 2. Conditional on the history of regimes, K, i the model in 21 reduces to a VAR model with dummy variables that account for known structural breaks and ii the transition robabilities, π jj j {1,2}, are indeendent of the data and of the remaining arameters of the model, [Φ j, Σ j] j {1,2}. As a result, if the rior distributions for Φ l and Σ l, l {1, 2}, are of the MN IW form and π 11 and π 22 are indeendent beta distributions, then the conditional osterior distributions of Φ l, Σ l K, Y, l {1, 2} and 25

27 π ll, l = 1, 2 Y, K, Φ j, Σ j j {1,2} belong to the same family of their corresonding riors. Therefore, the analytical tractability condition is satisfied for τ = As draws from the conditional osterior distribution for the regimes can be obtained using a variant of the Carter and Kohn , we can ensure that the samling condition is also satisfied. B.3 Dynamic Factor Models DFMs DFMs decomose the behavior of n observable variables y i,t, i = 1,... n, into the sum of two unobservable comonents: for any t = 1,..., T, y i,t = a i + λ i f t + ξ i,t, iid ξ i,t N 0, σi 2 22 where a i is a constant; f t is a k 1 vector of factors which are common to all observables, λ i is a 1 k vector of loadings that links the observables to the factors, and ξ i,t is an innovation secific to each observable variable. The factors evolve according to a vector autoregressive rocess: f t = Φ 0,1 f t Φ 0,q f t q + u 0,t, u 0,t iid N 0, Σ 0 23 where u 0,t is a k 1 vector and the matrices Φ 0,j {1,...,} and Σ 0 are k k matrices. The stochastic vector of innovations, u t, has dimension of k 1. Let us define the n 1 vectors y t = y 1,t,..., y n,t, a = a 1,..., a n, λ = λ 1,..., λ k,..., λ n, ξ t = ξ 1,t,..., ξ n,t, and, for any j {1,..., }, the n n diagonal matrix Φ j, whose diagonal elements are φ 1,j,..., φ n,j. It is convenient to recast the DFM in matrix form as follows: Y = XΦ 1 + ε 24 F = F Φ 0 + ε 0 25 where we define the T n matrix Y = y 1,..., y T, the T k + 1 matrix X = [1 T, F ], with 1 T being a 1 T vector of ones and F = f 1,..., f T is a T k matrix of factors, the k + 1 n matrix Φ 1 = [a, λ ] and the T n matrix of residuals is denoted as ε = ξ 1,..., ξ T, where ε N 0, Σ 1. We define the T kq matrix F = f1,,...,, f T with the kq 1 vectors f t = f t 1,..., f t q, the kq k matrix Φ0 = [Φ 0,1,..., Φ 0,q ], and the T k matrix ε 0 = u 0,1,..., u 0,T. 15 These restrictions over riors are only sufficient for satifying the analytical tractability condition. Such condition can be shown to be also satisfied under an imroer flat rior, such as 2 j=1 Φ j, Σ j = 2 j=1 Σ j n+1/2. 16 See Del Negro and Schorfheide 2010 and Pitt and Kohn

28 Let us artition the arameter sace Θ as θ 1 = Φ 1, θ 2 = Σ 1, θ 3 = Φ 0, and θ 4 = Σ 0. The rior for the constant terms and the factor loadings Φ 1 is usually selected to be normal, while the rior for the Σ 1 is chosen to be an Inverted-Wishart. Furthermore, the riors for the arameters of the factor model 25 i.e., Φ 0 and Σ 0 are chosen to belong to the MN IW family. Conditional on the factors, F, the system in 24 boils down to a multivariate linear Gaussian regression model. Hence, it simle to see that the osterior density Φ 1, Σ 1 Φ 0, Σ 0, F, Y = Φ 1, Σ 1 F, Y belongs to the MN IW family. Note that conditional on the factors, F, the likelihood function, Y Θ is not affected by the arameters Φ 0 and Σ 0, that is, Y Φ 0, ε 0, Φ 1, Σ 1, F = Y Φ 1, Σ 1, F. Therefore the osterior densities, Φ 0 Φ 1, Σ 1, F, Y = Φ 0 Σ 0 and Σ 0 Φ 1, Σ 1, F, Y = Σ 0, equal their riors and hence are analytically tractable. Hence it follows that the analytical tractability condition is satisfied for τ = 4. Del Negro and Schorfheide 2010 discuss how the samling condition is also satisfied. B.4 Factor-Augmented Vector Autoregressive Models FAVARs Bernanke, Boivin, and Eliasz 2005 roose a hybrid model between a standard structural VAR model and a DFM model that has been called Factor-Augmented Vector Autoregression henceforth FAVAR. This extension of the DFM aradigm allows for additional observations f y t in the measurement equation 22 such that y i,t = a i + λ y i f y t + λ f i f t c + ξ i,t, t = 1,..., T 26 where λ y i is a 1 m vector and f y t is an m 1 vector, where f c t are the unobserved factors. The joint dynamics of the erfectly observable vector, f y t, and the unobserved factors, f c t, are described by the following state equation [ f c t f y t ] = Φ 0,1 [ f c t 1 f y t 1 ] Φ 0,q [ f c t q f y t q ] + u 0,t, u 0,t iid N 0, Σ 0 27 which is a VARq for ft c, f y t. Comaring equation 27 with equation 23, we have that now the matrices Φ 0,j have dimension k + m k + m. Given equations 26-27, we can recast the FAVAR model in matrix form as we did for DFM models in equations where now the matrix of observables is given by the T n + m matrix X instead of just the T n matrix Y. Therefore, it directly follows that both the samling and analytical tractability conditions are satisfied. B.5 Time-Varying Parameters TVP VAR Models 27