The B.E. Journal of Macroeconomics

Transcription

1 The B.E. Journal of Macroeconomics Topics Volume 8, Issue Article 20 Forecasting with DSGE Models: The Role of Nonlinearities Paul Pichler University of Vienna, paul.pichler@univie.ac.at Recommended Citation Paul Pichler (2008) Forecasting with DSGE Models: The Role of Nonlinearities, The B.E. Journal of Macroeconomics: Vol. 8: Iss. 1 (Topics), Article 20. Available at: Copyright c 2008 The Berkeley Electronic Press. All rights reserved.

2 Forecasting with DSGE Models: The Role of Nonlinearities Paul Pichler Abstract This paper studies whether the out-of-sample forecasting performance of a dynamic stochastic general equilibrium (DSGE) model improves by taking its nonlinear rather than its linear approximation to the data. We address this question within a New Keynesian monetary economy, considering both environments of simulated and real data. Precisely, we estimate our model based on its linear respectively quadratic approximate solution, generate out-of-sample forecasts for three observables (output, inflation, and the nominal interest rate), and compare the quality of forecasts by several statistical measures of accuracy. We find that the value of nonlinearities in terms of predictive power depends crucially on whether the model is well specified. For simulated data, the nonlinear model indeed forecasts noticeably better as compared to its linearized counterpart, whereas for real data, we find no substantial differences in predictive abilities. KEYWORDS: forecasting, DSGE models, nonlinearities, particle filter, Kalman filter I thank Gerhard Sorger, Peter Hackl, Jesus Crespo-Cuaresma, the editor Jesus Fernandez- Villaverde, and two anonymous referees for helpful comments and suggestions. Financial support from the Austrian Science Fund (FWF) under project numbers P17886 and P19686 is gratefully acknowledged.

3 1 Introduction Pichler: Forecasting with DSGE Models Dynamic stochastic general equilibrium (DSGE) models are today among the most prominent tools in quantitative macroeconomics. They are being employed extensively to analyze macroeconomic fluctuations and, recently, to forecast macroeconomic aggregates. Especially in monetary economics, dynamic equilibrium economies featuring monopolistic competition and nominal rigidities have become popular laboratories. Prominent examples include the papers by Smets and Wouters (2003, 2007), Ireland (2004), Christiano et al. (2005), and Adolfson et al. (2007), to name just a few. Dynamic stochastic general equilibrium economies generally lack an analytical solution, such that economists are confined to work with numerical approximations to their theoretical models. Most researchers rely on linearization around a non-stochastic steady state to compute such approximations. This approach is appealing from an econometric perspective, as it allows for the use of Kalman filtering techniques to (i) build the likelihood function implied by the approximate model, and (ii) construct out-of-sample forecasts. Thus, taking linearized models to the data is a relatively straightforward task. However, linearization may be problematic. In environments where nonlinearities are important or the economy travels far away from the steady state, linear approximations are likely to be very inaccurate. Moreover, recent work by Fernandez-Villaverde and Rubio-Ramirez (2005) and Fernandez-Villaverde et al. (2006) points out that estimating DSGE models based on their linearized solution will generally lead to biases in parameter estimates. The fit of a linearized model as measured by the marginal likelihood, therefore, can be substantially worse compared to the nonlinear model. 1 Fernandez-Villaverde and co-authors thus suggest to move to at least second-order approximations when taking DSGE models to the data, even though this approach is computationally more demanding since it necessitates the use of Monte Carlo methods for constructing the likelihood function. The results discussed above raise the question whether linearization has also negative effects on the predictive abilities of a model. The purpose of this paper is to formally address this question. We contrast the out-of-sample forecasting performance of a linearized model with the performance of a secondorder accurate model, whereby we focus on two distinct scenarios. First, using simulated data, we compare predictive abilities in an environment where the theoretical model is de facto correctly specified. Later, using real data, we 1 See Wolman and Couper (2003) and Kim and Kim (2003, 2007) for further potential consequences of linearization in economics. Published by The Berkeley Electronic Press,

4 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 investigate predictive abilities under model misspecification. We find that the value of nonlinearities in terms of forecasting differs crucially between these two scenarios: for simulated data, the nonlinear model indeed forecasts noticeably better as compared to its linearized counterpart, whereas for real data, we find no substantial differences in predictive abilities. The framework we use for our comparison is a relatively small New Keynesian monetary economy featuring monopolistic competition, capital accumulation, and price as well as capital adjustment costs. In this model, technology shocks, preference shocks, and monetary shocks are the driving forces behind macroeconomic fluctuations. We choose a New Keynesian laboratory for mainly two reasons. First, this class of models has recently been explored by many researchers. As all our results are conditional on the particular model being employed, we want to choose a framework that is well-understood and an active area of research. Second, nonlinearities typically do not play an overly important role in New Keynesian environments. 2 Thus, if nonlinearities are important within our framework, they arguably are relevant in many other applications as well. Our paper contributes to a growing literature that emphasizes the role of nonlinearities in dynamic equilibrium economies. Recent papers by Justiniano and Primiceri (2008) and Fernandez-Villaverde and Rubio-Ramirez (2007), for example, investigate DSGE models with stochastic volatility, a feature which induces fundamental nonlinearities in the model. Both contributions conclude that stochastic volatility is crucial for understanding U.S. data, and thus, that linearized models fail to capture important aspects of the U.S. economy. Nonlinearities have recently also been emphasized in the context of the opportunistic approach to disinflation (Orphanides and Wilcox, 2002; Aksoy et al., 2006). Proponents of the opportunistic approach hold that the central bank should not take deliberate anti-inflation actions as long as inflation is within certain bounds, thus calling for nonlinear monetary policy rules. As argued by Van Binsbergen et al. (2008), among others, nonlinearities are furthermore fundamental in models with Epstein-Zin preferences. This class of preferences has become popular over the last years, since it allows to account for asset pricing observations that are hard to address within the standard state-separable utility framework. Finally, nonlinearities have been emphasized recently also in the context of New Keynesian economies. Although there they typically play no fundamental role, papers by An (2005), An and Schorfheide (2007), 2 For example, these models often feature utility functions that are linear-separable, linear Taylor-type monetary policy rules, and shock processes which follow linear laws of motion and display constant volatilities. Consequently, one may argue that many New Keynesian models, like the model we use in our application, are almost linear. 2

5 Pichler: Forecasting with DSGE Models and Amisano and Tristani (2007) have found that exploiting nonlinearities allows for sharper inference compared to the estimation of linearized models, and that nonlinear models can account for richer economic dynamics. The remainder of this paper is organized as follows. Section 2 presents our model economy and characterizes its equilibrium. Section 3 outlines the estimation of structural parameters based on the model s linear respectively quadratic approximation, and discusses the generation of out-of-sample forecasts. Section 4 describes the data we use for evaluating predictive abilities and illustrates the measures of forecast accuracy that are being employed. Sections 5 and 6 contain our results for simulated and real data, respectively. Finally, Section 7 summarizes and concludes. 2 The model This section develops our relatively small New Keynesian dynamic equilibrium economy. We first present the economic environment, and later characterize the model s symmetric competitive equilibrium. 2.1 Environment The economy is populated by a representative household, a representative finished-goods-producing firm, a continuum of intermediate-goods-producing firms indexed by j [0, 1], and a central bank. Time is discrete and goes on forever, i.e. t {0, 1, 2,...} Households The representative household enters period t with M t 1 units of money, B t 1 units of bonds, and k t units of capital. Bonds mature at the beginning of the period, providing B t 1 additional units of money. Furthermore, the household receives a lump-sum transfer from the government, L t. During period t, the household earns factor income from supplying labor h t (j) and capital k t (j) to each intermediate-goods-producing firm j [0, 1], taking nominal factor prices W t and Q t as given. Denoting the total amounts of labor and capital supplied by h t = 1 h 0 t(j)dj and k t = 1 k 0 t(j)dj, nominal factor income is given by W t h t + Q t k t. Finally, the household receives dividend payments from the intermediate-goods-producing firms, D t = 1 D 0 t(j)dj. The household s expenditures are the following. First, the household purchases the final good which is used for both consumption c t and investment Published by The Berkeley Electronic Press,

6 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 x t. In order to transform x t units of the final good into x t units of productive capital, the household pays a transformation cost equal to CAC t = φ k 2 ( ) 2 xt δ k t, k t with δ being the depreciation rate of capital and φ k measuring the size of adjustment costs. The household furthermore purchases new bonds at a nominal price of 1/i t, where i t denotes the gross nominal interest rate between periods t and t + 1. The remainder of funds is carried over into the next period in the form of money, M t. Letting P t denote the price level at time t, the household s budget constraint reads M t 1 + B t 1 + W t h t + Q t k t + D t + L t P t c t + x t + φ k 2 + B t/i t + M t P t. ( ) 2 xt δ k t k t Note finally that, since capital depreciates at the constant rate δ, the capital stock evolves according to k t+1 = (1 δ)k t + x t. The household values consumption, real money holdings, and leisure according to a standard time separable expected utility objective, E 0 t=0 β t u(c t, M t /P t, 1 h t ), with β being the discount factor. function takes the form u(c t, M t c 1 τ t 1, 1 h t ) = a t + χ m log P t 1 τ We assume that the momentary utility ( Mt P t ) + χ h (1 h t ). Utility, therefore, is linear separable in its three arguments, with χ m and χ h being weights associated with utility derived from real money balances and leisure, respectively. The parameter τ measures the inverse of the elasticity of substitution between current and future consumption, and a t is a preference shock that evolves according to log(a t+1 ) = ρ a log(a t ) + ɛ a,t+1, ɛ a,t+1 N(0, σ 2 a). 4

7 Pichler: Forecasting with DSGE Models The representative finished-goods-producing firm The representative finished-goods-producing firm produces y t units of a single output good on a perfectly competitive market. It uses a constant returns-toscale production technology, [ 1 θ/(θ 1) y t = y t (j) dj] (θ 1)/θ. 0 Intermediate goods, y t (j), j [0, 1], serve as the only inputs in the production. The parameter θ measures the constant elasticity of substitution between any two intermediate inputs. The firm s objective is to maximize profits, taking the price of its own output good as well as the prices of all intermediate goods as given. Perfect competition on the final goods market drives the firm s profits in equilibrium to zero, determining the equilibrium price as [ 1 P t = 0 ] 1 P t (j) 1 θ 1 θ dj. The demand for each intermediate good j, in turn, is given by Intermediate-goods-producing firms ( ) θ Pt (j) y t (j) = y t. (1) P t The intermediate-goods-producing firm j hires h t (j) units of labor and k t (j) units of productive capital from the household to produce y t (j) units of the intermediate good j. The production technology is Cobb-Douglas with labor augmenting technological change, i.e., y t (j) = k t (j) α [z t h t (j)] 1 α. (2) The parameter α gives capital s share in output, whereas z t is a technology shock that follows the autoregressive process log(z t+1 ) = (1 ρ z ) log(z) + ρ z log(z t ) + ɛ z,t, ɛ z,t N(0, σ 2 z). Intermediate goods are produced on a monopolistically competitive market. Each firm, therefore, can set its nominal price, P t (j), subject to the requirement that it satisfies demand at the chosen price. When adjusting the nominal Published by The Berkeley Electronic Press,

8 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 price, the firm faces quadratic costs a la Rotemberg (1982), measured in terms of the finished good and given by P AC j t = φ p 2 [ 2 Pt (j)/p t 1 (j) 1] y t P t, π where π denotes the steady state rate of inflation and φ p measures the size of adjustment costs. Finally, in the end of period t, the firm distributes its profit to the representative household as dividend payment D t (j), satisfying D t (j) = P t (j)y t (j) W t h t (j) Q t k t (j) φ [ 2 p Pt (j)/p t 1 (j) 1] y t P t. (3) 2 π The objective of each firm j [0, 1] is to maximize its total market value. Formally, it solves max E 0 h t (j),k t (j),p t (j) t=0 β t λ t D t (j) P t subject to (1), (2), and (3). By λ t we denote the Lagrangian multiplier associated with the household s budget constraint, such that β t λ t gives the household s marginal utility value of one unit of real profits in period t The central bank The monetary authority adjusts the nominal interest rate i t following a linear feedback rule. Specifically, it smoothes the interest rate over time and reacts to deviations of output and inflation from their target values, as formally described by log ( it i ) = ρ i log ( it 1 i ) + ρ y log ( ) yt + ρ π log y ( πt ) + ɛ i,t, (4) π where i, y, and π denote the target (or steady-state) values of the respective variables. The central bank can choose the level of one of these target variables, as well as the parameters ρ i, ρ y and ρ π. In the following we assume that the central bank sets its inflation target, π, and then implements its policy rule by adjusting the nominal money stock so that (4) holds and the money market clears. The term ɛ i,t denotes an i.i.d. normal monetary policy shock with mean zero and standard deviation σ i. 6

9 Pichler: Forecasting with DSGE Models 2.2 The model s symmetric competitive equilibrium We study the model s implications by analyzing its symmetric competitive equilibrium. In this equilibrium, all intermediate firms make identical choices and the market clearing conditions M t = M t 1 + L t and B t = B t 1 = 0 must hold for all t. Letting m t = M t /P t denote real money holdings, w t = W t /P t and q t = Q t /P t real factor prices, and π t = P t /P t 1 inflation, we can characterize the model s symmetric equilibrium by the following system of equations 3 : 0 = χ h a t w t, (5) c τ t 0 = a t 1 a t+1 1 βe c τ t, (6) t i t c τ t+1 π t+1 0 = χ m a ) t (1 1it m c τ t, (7) t 0 = a [ ( )] { [ t xt a t φ c τ k δ βe t (q t δ) t k t φ k 2 c τ t+1 ( ) 2 ( ) ( xt+1 xt+1 xt+1 δ + φ k δ δ + 1 k t+1 k t+1 k t+1 ) ]}, (8) 0 = kt α [z t h t ] 1 α y t, (9) 0 = y t w t h t q t k t φ ( p πt ) 2 2 π 1 yt d t, (10) 0 = αw t h t (1 α)q t k t, (11) 0 = a [ t w t h ( t πt ) ] 1 θ + θ φ c τ p t (1 α)y t π 1 πt π { } a ( t+1 πt+1 ) + βφ p E t c τ t+1 π 1 πt+1 y t+1, (12) π y t ( ) ( ) it 1 yt ( πt ) ( ) it 0 = ρ i log + ρ y log + ρ π log + ɛ i,t log, (13) i y π i 0 = c t + x t + φ ( ) 2 k xt δ k t + φ ( p πt ) 2 2 k t 2 π 1 yt y t, (14) 0 = k t+1 (1 δ)k t x t, (15) 0 = ρ a log(a t ) + ɛ a,t+1 log(a t+1 ), (16) 0 = (1 ρ z ) log(z) + ρ z log(z t ) + ɛ z,t+1 log(z t+1 ). (17) 3 A formal derivation of these conditions from the agents optimization problems is provided in the appendix. Published by The Berkeley Electronic Press,

10 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 Note that expressions (5)-(8) characterize optimal household behavior: (5) equates the marginal rate of substitution between labor and consumption to the real wage, (6) describes the household s indifference between consumption and bond holdings, (7) implicitly defines the household s money demand, and (8) equates the marginal utility cost of one unit of additional investment at time t to the discounted expected marginal utility value of its return in period t + 1. Note furthermore that equation (12), which is derived from the optimal price setting behavior of intermediate firms, represents the model s nonlinear Phillips curve. The equilibrium conditions (5)-(17) can be summarized as where R is a nonlinear operator and E t R(f t+1, f t, s t+1, s t, ε t+1 ; Θ) = 0, s t = (k t, i t 1, a t, z t, ɛ i,t ), f t = (y t, c t, x t, w t, h t, q t, d t, π t, m t ), ε t+1 = (ɛ a,t+1, ɛ z,t+1, ε i,t+1 ), Θ = (β, α, δ, θ, φ k, φ p, τ, χ m, χ h, π, z, ρ a, σ a, ρ z, σ z, ρ i, σ i, ρ y, ρ π ). The vectors s t and f t include the model s state and control variables, respectively, ε t+1 summarizes the exogenous disturbances, and Θ collects the structural parameters. The model s solution, in turn, is given by decision rules Ψ(s t, ε t+1 ) and Φ(s t ) that satisfy E t R(Φ(Ψ(s t, ε t+1 )), Φ(s t ), Ψ(s t, ε t+1 ), s t, ε t+1 ; Θ) = 0 for all s t, ε t+1, and t. Collecting state and decision variables in a vector X t = (s t, f t), this solution can be written compactly as X t+1 = H(X t, ε t+1 ; Θ), where H is a nonlinear function that can easily be constructed from Ψ and Φ, respectively. For the economy under consideration, the functions Φ and Ψ (and therefore H) cannot be computed analytically, but have to be approximated by numerical methods. 3 Estimation and forecasting Having outlined the economic environment, we next describe how the theoretical model can be taken to the data. We first discuss the construction of the model s likelihood function and the estimation of structural parameters via maximum likelihood methods. Later, we illustrate how the estimated model can be used for out-of-sample forecasting observable data series. 8

11 Pichler: Forecasting with DSGE Models 3.1 Constructing the likelihood function Assume that we observe N Y macroeconomic time series of length T, collected in Y T = {Y t } T t=1. Together with the data, our DSGE model forms the following nonlinear state-space system: X t+1 = H(X t, ε t+1 ; Θ), (18) Y t = GX t + ν t. (19) By ν t we denote a vector of normally distributed and uncorrelated measurement errors, i.e., ν t N(0, Σ ν ) with Σ ν being diagonal. Note that we make the assumption that the model s variables are related to the observable data series in a linear way, as represented by the matrix G. 4 From the state-space system (18)-(19) it follows that the likelihood of the data sample Y T conditional on our model H with parameters Ω = {Θ, Σ ν } is given by L(Y T H, Ω) = = T p(y t Y t 1 ; H, Ω) t=1 T t=1 p(y t X t, Y t 1 ; H, Ω)p(X t Y t 1 ; H, Ω)dX t, (20) where p denotes a probability density. Unfortunately, since the model solution H cannot be computed analytically, we cannot evaluate (20) exactly. Filtering techniques, however, allow us to approximate the likelihood based on approximate decision rules Ĥ, i.e., we can compute L(YT Ĥ, Ω). In this paper, we compute the likelihood based on linear and nonlinear approximations to the model s decision rules, denoted by H and H. We derive these approximations using first and second-order perturbation methods as described by Klein (2000) and Schmitt-Grohe and Uribe (2004), respectively. We choose to employ perturbation methods because they are extremely fast, which is an important feature when taking our DSGE model to the data, since likelihood based inference requires to solve the model many hundred times. Based on a first-order approximation of the model, we can construct the likelihood function in a straightforward way. Since H is linear and the innovations ε t+1 are Gaussian, the densities in (20) are normal, such that the Kalman 4 This assumption is unproblematic, though, since it requires only that some of the model s variables, or linear transformations of variables, are observable. It is typically easy to construct the model such that this requirement is met, or to transform the data accordingly. Published by The Berkeley Electronic Press,

12 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 filter can be used to analytically build the likelihood function L(Y T H, Ω). 5 For estimating the model based on its quadratic approximation H, however, the Kalman filter is no longer available. In particular, since H is nonlinear, we cannot evaluate p(x t Y t 1 ; H, Ω) in closed form. To estimate parameters based on the quadratic model, we thus resort to Monte Carlo methods, precisely, the basic particle filter suggested by Fernandez-Villaverde and Rubio- Ramirez (2007). 6 This filter allows us to compute an approximation to the likelihood function as L(Y T H, Ω) = T t=1 1 N N p(y t x i t t 1, Y t 1 ; H, Ω). (21) i=1 In expression (21), { x i t t 1 }N i=1 are draws from each density in the sequence {p(x t Y t 1 ; Ω)} T t=1. Fernandez-Villaverde and Rubio-Ramirez (2007) and An (2005) describe in detail how to efficiently obtain these draws and how to initialize the particle filter. In this paper we closely follow the approach laid out by these two contributions and, therefore, do not present details on particle filtering but refer to the original sources for further information. 3.2 Maximizing the likelihood function Having constructed the likelihood function from either the linearized or the quadratic model, maximum likelihood estimates can be computed numerically as Ω ML = arg max Ω L(Y T H, Ω) and Ω ML = arg max Ω L(Y T H, Ω), respectively. Deriving these estimates, however, is not a trivial exercise, as the following complications arise. First, the likelihood function is almost flat with respect to some parameters; this problem is not specific to our analysis, but is frequently encountered when estimating DSGE models. Second, the likelihood function of our model features many local maxima and minima; again, this is not unusual for likelihood functions of DSGE models, and it obviously makes the maximization task difficult. Finally, when the particle filter is used to construct the likelihood based on the nonlinear solution, the resulting likelihood function is not continuous with respect to the parameter vector Ω; standard 5 See chapter 13 of Hamilton (1994) for a detailed description of the Kalman filter algorithm. 6 We choose the basic particle filter mainly because it allows us to build on earlier work by Fernandez-Villaverde and Rubio-Ramirez (2007) and An and Schorfheide (2007), in particular, to use computer code developed by these authors. In future work we plan to use potentially more efficient filters, such as the conditional particle filter recently proposed by Amisano and Tristani (2007) or the EIS particle filter proposed by De Jong et al. (2007). 10

13 Pichler: Forecasting with DSGE Models gradient-based routines, therefore, will typically fail to maximize the likelihood function. As outlined by Fernandez-Villaverde and Rubio-Ramirez (2007), this problem cannot be avoided when using their particle filter. In light of these difficulties, we proceed as follows. First, as is common in the literature, we calibrate several parameters rather than estimate them via maximum likelihood. In particular, these are the parameters associated with investment, leisure, and real balances, which are hard to pin down without data on the respective variables. To address the latter two problems we employ a simulated annealing approach instead of gradient-based methods for maximizing the likelihood function. As argued by Goffe et al. (1994) and Fernandez-Villaverde and Rubio-Ramirez (2007), this method works well in the presence of local maxima and it allows to deal with the discontinuity of the likelihood function. 3.3 Forecasting Having obtained the maximum likelihood estimates of the model s parameters, we can generate out-of-sample forecasts for the observable data series. In our classical environment, it would be natural to forecast Y T +h using its expectation at time T, i.e., Y T +h T = E T (Y T +h Y T, Ω ML, H) = GE T (X T +h Y T, Ω ML, H) = GX T +h T, where Y T +h T and X T +h T denote forecasts, conditional upon information available at time T, for the vectors Y T +h and X T +h obtained from the model H evaluated at the maximum likelihood parameter estimates Ω ML. Because H (and thus Ω ML ) cannot be computed analytically, however, the exact state predictor X T +h T cannot be obtained analytically either. Resorting again to approximations, we use Ȳ T +h T = GE T (X T +h Y T ; Ω ML, H) = G X T +h T to forecast from the linearized model and Ỹ T +h T = GE T (X T +h Y T ; Ω ML, H) = G X T +h T to forecast from the quadratic model, respectively. Constructing ȲT +h T from the linear model is a straightforward task, since Ȳ T +h T = G X T +h T = G H X T +h 1 T = G H 2 XT +h 2 T =... = G H h XT T. Published by The Berkeley Electronic Press,

14 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 To generate out-of-sample forecasts from the linearized model, we thus only need to obtain an estimate of the state at time T, XT T. This estimate is readily available from the Kalman filter algorithm. Obtaining forecasts from the nonlinear model, however, is not as easy. The complication that arises is that, for h > 1, X T +h is a nonlinear function of X T and all shocks {ε T +1,..., ε T +h }. The state predictor X T +h T, therefore, is not only a function of X T T, but also of higher moments of the shock processes. In particular, this implies that forecasts cannot be computed recursively as for the linear model, since X T +h T = E T (X T +h Y T ; Ω ML, H) E T ( H(X T +h 1 T, ε T +h ; Θ ML ) Y T ; Ω ML, H) for h > 1 due to the nonlinearity of the function H. This difficulty can be addressed, however, in a straightforward way by using Monte Carlo or numerical integration methods to compute E T (X T +h Y T ; Ω ML, H). As discussed in Judd (1998), numerical integration methods are often more accurate and computationally less demanding than Monte Carlo methods, and hence we choose to follow this route to compute E T (X T +h Y T ; Ω ML, H). 7 Finally, as a benchmark, we also construct forecasts from a random walk (RW) model and reduced form VAR(1) and VAR(2) models. Since these models are linear, forecasts for h = 1, 2,... can be generated again recursively; precisely: Y RW T +h T = Y T, Y V AR(1) T +h T Y V AR(2) T +h T = AY V AR(1) T +h 1 T, = B 1 Y V AR(2) T +h 1 T + B2 Y V AR(2) T +h 2 T, where the matrices A, B 1, and B 2 contain the OLS estimates of the respective VAR coefficients. Note that Y V AR(1) T T = Y V AR(2) T T = Y T and Y V AR(2) T 1 T = Y T 1, respectively. 4 Data and forecast accuracy measures In the following, we briefly discuss the data that are being used for our evaluation of forecast accuracy, and we outline the statistical measures that are employed to assess the predictive abilities of our models. 7 Precisely, we use Judd s (1998) formula (7.5.9) for expectations over nonlinear functions of normal variables. To check for robustness, we have also computed the expectations using Monte Carlo methods and came to virtually identical results. 12

15 Pichler: Forecasting with DSGE Models 4.1 Data We use both simulated and real data to evaluate predictive abilities. For the first scenario, we generate several simulated data sets from our DSGE model being solved with a third-order accurate projection method. For the second scenario, we use macroeconomic data for the United States. 8 Our U.S. data set includes quarterly time series on output, inflation, and nominal interest rates. The series are taken from the FRED database maintained by the Federal Reserve Bank of St. Louis. 9 Output corresponds to quarterly gross domestic product (GDP), whereby we remove a linear trend from the (logged) GDP series. Inflation is calculated as the relative change in the GDP deflator (GDPDEF), and nominal interest rates are measured by 3-Month Treasury Bill rates (TB3MS). We select the time period 1979Q3-2007Q2, i.e., we exclude the pre Volcker-Greenspan era. This gives us a sample of 112 observations on each series. To construct artificial data we proceed as follows. First, we assign values to the model s parameters. We set β = and π = to match steady state inflation and interest rates to their averages in U.S. data. Furthermore we choose α = 0.36 and δ = 0.025, which are standard values in the RBC literature. The adjustment cost parameters are chosen to closely match the estimates reported in Ireland (2001), φ p = 80 and φ k = 10. The steady state technology level is given by z = 6900, such that the model s steady state output level corresponds roughly to the average output in U.S. data. The remaining parameters governing the law of motion for z t are chosen in line with the existing RBC literature, i.e., ρ z = 0.95 and σ z = The parameters governing the law of motion for a t are again close to estimates reported by Ireland (2001), ρ a = 0.92 and σ a = The monetary policy parameters are selected such that they resemble values typically associated with a Taylor rule, ρ i = 0.7, ρ y = 0.5, and ρ π = 1.5. The standard deviation of the monetary policy shock is chosen as σ i = 0.002, which is somewhat lower than the estimates in Ireland (2001). With a higher σ i, however, our model would sometimes violate the zero bound on nominal interest rates during the simulations. The preference parameter is set at τ = 2. Finally, following An (2005), the parameter χ h is selected such that the household spends 30% of its time working in the model s steady state, and χ m is chosen such that the steady state ratio between real balances 10 and quarterly output is equal to 2.4, 8 The data, together with our MATLAB codes, are available upon request. 9 The FRED database is available at 10 Real balances are measured by dividing the M2 money stock by the GDP implicit price deflator. Published by The Berkeley Electronic Press,

16 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 which is roughly its average in the U.S. data. Having asserted these values, we solve the model numerically using a third-order Galerkin projection method. Since our model features five state variables, we do not follow the traditional projection approach pioneered by Judd (1992), as we find it too slow in our application. Instead, we use the methods outlined in Pichler (2008), which allow us to obtain a third-order accurate model solution within less than a minute. With the solution at hand, we simulate the model and construct 50 data sets, each of which consists of 88 observations on output, inflation, and nominal interest rates. The first 80 observations will later be used for estimating model parameters, whereas the last 8 observations will be used to compute forecast accuracy measures Measures of forecast accuracy We evaluate forecasts using univariate and multivariate accuracy measures. To make forecast errors between different series comparable, we use scaleindependent measures as suggested by Hyndman and Koehler (2006). These authors propose to scale the forecast error based on the in-sample mean absolute one-step-ahead error of a random walk model. Using e i (h) to denote the vector of h-step ahead forecast errors associated with variable i {1, 2,..., N Y }, the scaled error is therefore given by q i (h) = 1 T 1 e i (h) T j=2 Yi j Yi j 1. As univariate measure of forecast accuracy, Hyndman and Koehler (2006) suggest to use the Mean Absolute Scaled Error, MASE i (h) = mean( q i (h) ). This measure, they argue, is easily interpreted and not very sensitive to outliers. To guarantee that our results are not overly dependent on the accuracy 11 Unfortunately, we cannot use 50 truly independent artificial data sets as the particle filter estimation is very time consuming such that we cannot repeat it 50 times. We thus use the following shortcut: we generate five realizations of the first 80 observations (estimation sample), for each of which we construct 10 different realizations of the subsequent 8 periods (forecasting sample). In this way, we construct 50 data sets with only 5 different estimation samples. Such an approach is obviously suboptimal, however, we do not find a better way given our endowment with computing capacities. Note that even with the current specification, the evaluation exercise runs approximately three days on our standard desktop computer. 14

17 Pichler: Forecasting with DSGE Models measure being employed, we furthermore consider the Root Mean Squared Scaled Error, RMSSE i (h) = mean(q i (h) 2 ). As both measures we use are scale-independent, multivariate versions can be constructed easily as the averages of the univariate measures. In addition to visually inspecting accuracy measures, we run statistical tests to check whether the forecasting performances are significantly different between the considered models. First, for every series and forecast horizon, we conduct Diebold-Mariano tests based on the absolute scaled error measures. 12 This requires from us to construct, for each data series and forecast horizon, a vector of forecast loss differentials between any two models j and l, d i j,l (h), given by d i j,l(h) = q i j(h) q j l (h). In the above expression, qs(h) i denotes the vector of h-step ahead forecast errors produced by model s {j, l} for variable i. Then, as described in Diebold and Mariano (1995), we test whether the loss differentials have a non-zero mean, in which case we would conclude that one method performs significantly better as compared to the other. As suggested by Hyndman and Koehler (2006), we finally analyze, for each data series, whether the quality of forecasts averaged across forecast horizons is different between any two models. To this end, we first summarize the vectors (h), h = 1, 2,..., 8, in a single vector of loss differentials, d i j,l d i j,l = d i j,l (1) d i j,l (2). d i j,l (8) We then employ a Wilcoxon Signed Rank test to check whether mean loss differentials are significantly different from zero.. 5 Results for simulated data This section presents the findings of our forecast evaluation exercise based on simulated data. Considering an artificial environment is interesting, we 12 We have conducted these tests also using the RMSSE measure and came to virtually identical results. Out of space considerations, we only provide the results based on the MASE measures. Published by The Berkeley Electronic Press,

18 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 believe, because it allows us to investigate the value of nonlinearities in terms of forecasting when the theoretical model is almost correctly specified. 13 We first contrast the parameter estimates obtained from both the linearized and the quadratic DSGE model, and later turn to the comparison of forecasting performances. 5.1 Parameter estimates The first step in generating forecasts from the DSGE model is to estimate the structural parameters Θ together with the measurement error variances Σ ν. This is done by using maximum likelihood methods as outlined in Section 3. However, for reasons that were also discussed in this section, we calibrate some parameters prior to estimation. As in Ireland (2001), we find it difficult to estimate α, δ, and φ k without data on the capital stock or investment. We thus set these parameters to their true values of α = 0.36, δ = 0.025, and φ k = 10. For similar reasons, we calibrate the mark-up parameter to θ = 6. Furthermore, we choose χ h such that the household spends 30% of its time working in the steady state, and χ m to match the steady state ratio between real balances and quarterly output. Finally, as in An (2005), the measurement error variances are calibrated rather than estimated. We set these variances equal to 10 % of the variance of the respective data series. The remaining 13 parameters are estimated via maximum likelihood, either using the linear model together with the Kalman filter, or by using the quadratic model together with the particle filter. In the latter case, we use particles for estimation. With a lower number of particles, we find it very hard to maximize the likelihood functions due to the problem of discontinuity. 14 Table 1 summarizes the average parameter estimates obtained from the linear and nonlinear model, respectively, together with the true values used for the simulations. 15 These numbers confirm the results of previous papers 13 Note that the data generating process (DGP) is the third-order accurate model, and hence, the theoretical model is not exactly equal to the DGP. 14 The discontinuity of the likelihood function implied by the nonlinear model arises due to the re-sampling step in the particle filter, as outlined by Fernandez-Villaverde and Rubio- Ramirez (2007). With a higher number of particles, each single particle contributes less to the likelihood, and hence the likelihood estimate is less sensitive to re-sampling. Therefore, as the number of particles grows, the problem of discontinuity becomes less severe. However, due to the computational burden we cannot increase the number of particles until the problem vanishes altogether. 15 Note that we do not provide standard errors. This is because the precision of estimates is not at the core of the present paper, and the computation of standard errors is a very 16

19 Pichler: Forecasting with DSGE Models Table 1: Parameter Estimates Parameter Linear Quadratic True β φ p π z ρ a ρ z σ a σ z ρ i ρ y ρ π τ σ i which found that using nonlinear methods typically has only a small influence on point estimates. In our application, the quadratic model apparently delivers slightly better estimates for some parameters, while generating slightly worse estimates for others. 5.2 Forecasting performance Let us now turn to the evaluation of forecasting performances. Table 2 lists forecast accuracy measures for both the linearized and the quadratic model, as well as the benchmark time series models. Out of space considerations, we limit attention to the forecast horizons h = 1, 2, 4, 8. There are several observations that we would like to emphasize. First and foremost, the quadratic model outperforms its linearized counterpart at all forecast horizons. This finding is independent of the measure of accuracy we consider, as can be seen from the smaller MASE and RMSSE values that are associated with the quadratic model. Although the differences between linear and nonlinear accuracy measures appear small, we consider them indeed relevant. For example, as indicated by the multivariate MASE measures for h = 8, the accuracy gains feasible from choosing the quadratic over the linear DSGE model are of roughly the same size as the accuracy gains associated with choosing the linear DSGE model over a naive random walk. complicated task in the case of the quadratic model because the likelihood function is not continuous with respect to the parameters. See Fernandez-Villaverde and Rubio-Ramirez (2007) for further information. Published by The Berkeley Electronic Press,

20 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 Table 2: Forecast Accuracy Measures - Simulated Data Quarters ahead Output MASE: Linear MASE: Quadratic MASE: RW MASE: VAR(1) MASE: VAR(2) RMSSE: Linear RMSSE: Quadratic RMSSE: RW RMSSE: VAR(1) RMSSE: VAR(2) Inflation MASE: Linear MASE: Quadratic MASE: RW MASE: VAR(1) MASE: VAR(2) RMSSE: Linear RMSSE: Quadratic RMSSE: RW RMSSE: VAR(1) RMSSE: VAR(2) Interest Rate MASE: Linear MASE: Quadratic MASE: RW MASE: VAR(1) MASE: VAR(2) RMSSE: Linear RMSSE: Quadratic RMSSE: RW RMSSE: VAR(1) RMSSE: VAR(2) Multivariate Measures MASE: Linear MASE: Quadratic MASE: RW MASE: VAR(1) MASE: VAR(2) RMSSE: Linear RMSSE: Quadratic RMSSE: RW RMSSE: VAR(1) RMSSE: VAR(2)

21 Pichler: Forecasting with DSGE Models A further interesting implication of Table 2 is that the relative advantage of the quadratic model over the linearized model is bigger for larger forecast horizons. This suggests that nonlinearities are particularly important when using DSGE models for medium and long run forecasting. To understand this result, first note that the linear and nonlinear one-step ahead forecasts differ in only a single way: the (non)linearity of the transition equation that is used to generate the forecast from the state predictor. On the other hand, linear and nonlinear h-step ahead forecasts (with h > 1) differ in a further respect: the latter are not functions of the state predictor alone, but also of the higher moments of the model s underlying shock processes. These two observations together suggest that, for small forecast horizons, linear and nonlinear forecasts will be relatively similar, and hence, that the gains from using the nonlinear model will arguably be small. As the forecast horizon increases, however, the differences between nonlinear and linear forecasts grow, and the gains associated with the nonlinear model become more apparent. 16 Finally, Table 2 reveals that the quadratic DSGE model also outperforms the simple time series models we have included in our forecast comparison exercise. In particular, it beats the random walk at all horizons and independent of the accuracy measure considered. Likewise, the quadratic model always performs better than both VAR models according to the RMSSE measures. Only in the very short run and according to the MASE measures for output and interest rates, the VAR models perform slightly better. However, when analyzing the multivariate measure for h = 1 and h = 2, we find again a better forecasting performance attributed to the quadratic DSGE model. To check whether any differences in predictive abilities are statistically significant, we furthermore perform the tests outlined in Section 4. Table 3 summarizes the results of Diebold-Mariano (DM) tests for each combination of data series and forecast horizon. For every pair of methods, we list the mean loss differentials and indicate whether those are significantly different from zero. The observation we find most interesting is that the quadratic model often performs indeed significantly better than the linearized model. In particular, medium-run forecasts for inflation derived from the nonlinear model are substantially more accurate, such that significance is obtained even at the 1% level. Differences between the theoretical model and the simple reduced form models, on the other hand, are hardly ever significant. This result appears puzzling at first sight, since the differences in the accuracy measures are often 16 In this context, note that, as h, the nonlinear model should outperform the linear model as its forecasts coincide with the unconditional mean of the respective series (i.e., the stochastic steady state), whereas the linear forecasts coincide with the non-stochastic steady state values. Published by The Berkeley Electronic Press,

22 The B.E. Journal of Macroeconomics, Vol. 8 [2008], Iss. 1 (Topics), Art. 20 Table 3: DM Tests - Simulated Data Quarters ahead Output Quadratic vs. Linear ** Quadratic vs. RW Quadratic vs. VAR(1) Quadratic vs. VAR(2) Linear vs. RW * Linear vs. VAR(1) Linear vs. VAR(2) Inflation Quadratic vs. Linear * *** Quadratic vs. RW Quadratic vs. VAR(1) * Quadratic vs. VAR(2) * * Linear vs. RW Linear vs. VAR(1) Linear vs. VAR(2) Interest Rate Quadratic vs. Linear ** * Quadratic vs. RW * Quadratic vs. VAR(1) Quadratic vs. VAR(2) * Linear vs. RW * Linear vs. VAR(1) Linear vs. VAR(2) Note: *, **, and *** denote significance at the 10%, 5% and 1% level. larger as compared to differences between the quadratic and the linear model. However, the intuition behind this finding is readily seen: the series of loss differentials constructed from the linear and quadratic DSGE model generally have a substantially lower volatility as compared to those constructed from the quadratic model and any reduced form model. Consequently, even relatively small differences in means often turn out statistically significant. In Table 4 we present the results of Wilcoxon tests conducted for every data series. We list the average loss differentials, d, together with the Z-statistic and p-value associated with each test. Our results indicate that the quadratic model performs substantially better than the linear model. Significance is obtained at the 1% level for each of the three series considered. Likewise, the quadratic model performs significantly better than all reduced form time series models we consider, whereby in two cases, significance is obtained only at the 5% level. The linearized model, on the other hand, does in general not substantially outperform the reduced form time series models. The differ- 20

23 Pichler: Forecasting with DSGE Models Table 4: Wilcoxon Tests - Simulated Data d Z p-value Output Quadratic vs. Linear Quadratic vs. RW Quadratic vs. VAR(1) Quadratic vs. VAR(2) Linear vs. RW Linear vs. VAR(1) Linear vs. VAR(2) Inflation Quadratic vs. Linear Quadratic vs. RW Quadratic vs. VAR(1) Quadratic vs. VAR(2) Linear vs. RW Linear vs. VAR(1) Linear vs. VAR(2) Interest Rate Quadratic vs. Linear Quadratic vs. RW Quadratic vs. VAR(1) Quadratic vs. VAR(2) Linear vs. RW Linear vs. VAR(1) Linear vs. VAR(2) ences in predictive abilities between the linear model and the VAR models, for example, are typically not significant. This result, we believe, is particularly interesting: although the nonlinear DSGE model corresponds to the true data generating process, its linearized version cannot significantly beat a VAR(1) model in out-of-sample forecasting while the quadratic model can. Finally, let us place the following remark. It may seem obvious that the quadratic model outperforms the linearized one in an environment of simulated data. After all, the quadratic model is a more accurate description of the true data generating process. However, one must keep in mind that, when the quadratic model is taken to the data, both the estimation and the forecasting step require the use of numerical methods (Monte Carlo and numerical integration methods), which is not true for the linearized model. Additional sampling uncertainty, therefore, is introduced by resorting to nonlinear approximations. Even though the quadratic model is a more accurate description of the DGP, this could potentially lead to the first-order accurate model outperforming the second-order accurate model in out-of-sample forecasting exercises. Published by The Berkeley Electronic Press,