Sequential Monte Carlo Methods (for DSGE Models)

Transcription

1 Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017

2 Some References Solution and Estimation of DSGE Models, with J. Fernandez-Villaverde and Juan Rubio-Ramirez, 2016, in: Handbook of Macroeconomics, vol. 2A Bayesian Estimation of DSGE Models, with E. Herbst, 2015, Princeton University Press Sequential Monte Carlo Sampling for DSGE Models, with E. Herbst, 2014, Journal of Econometrics See syllabus for additional references.

3 Some Background DSGE model: dynamic model of the macroeconomy, indexed by θ vector of preference and technology parameters. Used for forecasting, policy experiments, interpreting past events. Bayesian analysis of DSGE models: p(θ Y ) = p(y θ)p(θ) p(y ) p(y θ)p(θ). Computational hurdles: numerical solution of model leads to state-space representation = likelihood approximation = posterior sampler. Standard approach for (linearized) models (Schorfheide, 2000; Otrok, 2001): Model solution: log-linearize and use linear rational expectations system solver. Evaluation of p(y θ): Kalman filter Posterior draws θ i : MCMC

4 Sequential Monte Carlo (SMC) Methods SMC can help to Lecture 1 approximate the posterior of θ: Chopin (2002)... Durham and Geweke (2013)... Creal (2007), Herbst and Schorfheide (2014) Lecture 2 approximate the likelihood function (particle filtering): Gordon, Salmond, and Smith (1993)... Fernandez-Villaverde and Rubio-Ramirez (2007) or both: SMC 2 : Chopin, Jacob, and Papaspiliopoulos (2012)... Herbst and Schorfheide (2015)

5 Lecture 1

6 A Loglinearized New Keynesian DSGE Model Consumption Euler equation: ŷ t = E t [ŷ t+1 ] 1 ( ) ˆR t E t [ˆπ t+1 ] E t [ẑ t+1 ] + ĝ t E t [ĝ t+1 ] τ New Keynesian Phillips curve: where ˆπ t = βe t [ˆπ t+1 ] + κ(ŷ t ĝ t ), κ = τ 1 ν νπ 2 φ Monetary policy rule: ˆR t = ρ R ˆRt 1 + (1 ρ R )ψ 1ˆπ t + (1 ρ R )ψ 2 (ŷ t ĝ t ) + ɛ R,t

7 Exogenous Processes Technology: ln A t = ln γ + ln A t 1 + ln z t, ln z t = ρ z ln z t 1 + ɛ z,t. Government spending / aggregate demand: define g t = 1/(1 ζ t ); assume ln g t = (1 ρ g ) ln g + ρ g ln g t 1 + ɛ g,t. Monetary policy shock ɛ R,t is assumed to be serially uncorrelated.

8 Solving A Linear Rational Expectations System A Simple Example Consider y t = 1 θ E t[y t+1 ] + ɛ t, (1) where ɛ t iid(0, 1) and θ Θ = [0, 2]. Introduce conditional expectation ξ t = E t [y t+1 ] and forecast error η t = y t ξ t 1. Thus, ξ t = θξ t 1 θɛ t + θη t. (2)

9 A Simple Example Determinacy: θ > 1. Then only stable solution: ξ t = 0, η t = ɛ t, y t = ɛ t (3) Indeterminacy: θ 1 the stability requirement imposes no restrictions on forecast error: η t = Mɛ t + ζ t. (4) For simplicity assume now ζ t = 0. Then y t θy t 1 = Mɛ t θɛ t 1. (5)

10 At the End of the Day... We obtain a transition equation for the vector s t : s t = Φ 1 (θ)s t 1 + Φ ɛ (θ)ɛ t. The coefficient matrices Φ 1 (θ) and Φ ɛ (θ) are functions of the parameters of the DSGE model.

11 Measurement Equation Connect model variables s t with observables y t. In NK model: where YGR t = γ (Q) + 100(ŷ t ŷ t 1 + ẑ t ) INFL t = π (A) + 400ˆπ t INT t = π (A) + r (A) + 4γ (Q) ˆR t. γ = 1 + γ(q) 100, β = r (A) /400, π = 1 + π(a) 400. More generically: y t = Ψ 0 (θ) + Ψ 1 (θ)t + Ψ 2 (θ)s t +u }{{} t. optional

12 State-Space Representation and Likelihood Measurement: y t = Ψ 0 (θ) + Ψ 1 (θ)t + Ψ 2 (θ)s t State transition: +u t }{{} optional s t = Φ 1 (θ)s t 1 + Φ ɛ (θ)ɛ t Joint density for the observations and latent states: T p(y 1:T, S 1:T θ) = p(y t, s t Y 1:t 1, S 1:t 1, θ) = t=1 T p(y t s t, θ)p(s t s t 1, θ). t=1 Compute the marginal p(y 1:T θ). We will discuss how in Lecture 2.

13 Where Are We Headed?... Bayesian Inference Ingredients of Bayesian Analysis: Likelihood function p(y θ) Prior density p(θ) Marginal data density p(y ) = p(y θ)p(θ)dφ Bayes Theorem: p(θ Y ) = p(y θ)p(θ) p(y ) p(y θ)p(θ) Implementation: usually by generating a sequence of draws (not necessarily iid) from posterior θ i p(θ Y ), i = 1,..., N Algorithms: direct sampling, accept/reject sampling, importance sampling, Markov chain Monte Carlo sampling, sequential Monte Carlo sampling...

14 Bayesian Inference Posterior Focus on SMC algorithm that generates draws {θ i } N i=1 from posterior distributions of parameters. Draws can then be transformed into objects of interest, h(θ i ), and under suitable conditions a Monte Carlo average of the form h N = 1 N N i=1 h(θ i ) E π [h] h(θ)p(θ Y )dθ. Strong law of large numbers (SLLN), central limit theorem (CLT)...

15 Bayesian Inference Decision Making The posterior expected loss of decision δ( ): ρ ( δ( ) Y ) = L ( θ, δ(y ) ) p(θ Y )dθ. Θ Bayes decision minimizes the posterior expected loss: δ (Y ) = argmin d ρ ( δ( ) Y ). Approximate ρ ( δ( ) Y ) by a Monte Carlo average ρ N ( δ( ) Y ) = 1 N Then compute N L ( θ i, δ( ) ). i=1 δ N(Y ) = argmin d ρ N ( δ( ) Y ).

16 Bayesian Inference Point estimation: Quadratic loss: posterior mean Absolute error loss: posterior median Interval/Set estimation P π {θ C(Y )} = 1 α: highest posterior density sets equal-tail-probability intervals

17 Implementation: Sampling from Posterior DSGE model posteriors are often non-elliptical, e.g., multimodal posteriors may arise because it is difficult to disentangle internal and external propagation mechanisms; disentangle the relative importance of shocks. θ θ Economic Example: is wage growth persistent because 1 wage setters find it very costly to adjust wages? 2 exogenous shocks affect the substitutability of labor inputs and hence markups?

18 Sampling from Posterior If posterior distributions are irregular, standard MCMC methods can be inaccurate (examples will follow). SMC samplers often generate more precise approximations of posteriors in the same amount of time. SMC can be parallelized. SMC = importance sampling on steroids = We will first review importance sampling.

19 Importance Sampling Approximate π( ) by using a different, tractable density g(θ) that is easy to sample from. For more general problems, posterior density may be unnormalized. So we write π(θ) = p(y θ)p(θ) p(y ) = f (θ) f (θ)dθ. Importance sampling is based on the identity E π [h(θ)] = h(θ)π(θ)dθ = (Unnormalized) importance weight: w(θ) = f (θ) g(θ). f (θ) h(θ) Θ Θ g(θ) g(θ)dθ f (θ) g(θ) g(θ)dθ.

20 Importance Sampling 1 For i = 1 to N, draw θ i iid g(θ) and compute the unnormalized importance weights w i = w(θ i ) = f (θi ) g(θ i ). 2 Compute the normalized importance weights W i = 1 N w i N i=1 w i. An approximation of E π [h(θ)] is given by h N = 1 N N W i h(θ i ). i=1

21 Illustration If θ i s are draws from g( ) then E π [h] 1 N N i=1 h(θi )w(θ i ) 1 N N i=1 w(θi ), w(θ) = f (θ) g(θ) f 0.30 g g weights f/g 1 f/g

22 Accuracy Since we are generating iid draws from g(θ), it s fairly straightforward to derive a CLT: N( hn E π [h]) = N ( 0, Ω(h) ), where Ω(h) = V g [(π/g)(h E π [h])]. Using a crude approximation (see, e.g., Liu (2008)), we can factorize Ω(h) as follows: Ω(h) V π [h] ( V g [π/g] + 1 ). The approximation highlights that the larger the variance of the importance weights, the less accurate the Monte Carlo approximation relative to the accuracy that could be achieved with an iid sample from the posterior. Users often monitor ESS = N V π[h] Ω(h) N 1 + V g [π/g].

23 From Importance Sampling to Sequential Importance Sampling In general, it s hard to construct a good proposal density g(θ), especially if the posterior has several peaks and valleys. Idea - Part 1: it might be easier to find a proposal density for π n (θ) = [p(y θ)]φn p(θ) = f n(θ). [p(y θ)] φ np(θ)dθ Z n at least if φ n is close to zero. Idea - Part 2: We can try to turn a proposal density for π n into a proposal density for π n+1 and iterate, letting φ n φ N = 1.

24 Illustration: Tempered Posteriors of θ n θ π n (θ) = [p(y θ)]φn p(θ) = f ( ) λ n(θ) n, φ [p(y θ)] φ np(θ)dθ n = Z n N φ

25 SMC Algorithm: A Graphical Illustration C S M C S M C S M φ 0 φ 1 φ 2 φ 3 π n (θ) is represented by a swarm of particles {θ i n, W i n} N i=1 : h n,n = 1 N N i=1 W i nh(θ i n) a.s. E πn [h(θ n )]. C is Correction; S is Selection; and M is Mutation.

26 SMC Algorithm 1 Initialization. (φ 0 = 0). Draw the initial particles from the prior: iid p(θ) and W i 1 = 1, i = 1,..., N. θ i 1 2 Recursion. For n = 1,..., N φ, 1 Correction. Reweight the particles from stage n 1 by defining the incremental weights w i n = [p(y θ i n 1)] φn φ n 1 (6) and the normalized weights W n i w i = nwn 1 i 1 N N i=1 w, i = 1,..., N. (7) nw i n 1 i An approximation of E πn [h(θ)] is given by h n,n = 1 N 2 Selection. N i=1 W i nh(θ i n 1). (8)

27 SMC Algorithm 1 Initialization. 2 Recursion. For n = 1,..., N φ, 1 Correction. 2 Selection. (Optional Resampling) Let {ˆθ} N i=1 denote N iid draws from a multinomial distribution characterized by support points and weights {θ i n 1, W i n} N i=1 and set W i n = 1. An approximation of E πn [h(θ)] is given by ĥ n,n = 1 N N Wnh(ˆθ i n). i (9) i=1 3 Mutation. Propagate the particles {ˆθ i, W i n} via N MH steps of a MH algorithm with transition density θ i n K n(θ n ˆθ i n; ζ n) and stationary distribution π n(θ). An approximation of E πn [h(θ)] is given by h n,n = 1 N N h(θn)w i n. i (10) i=1

28 Remarks Correction Step: reweight particles from iteration n 1 to create importance sampling approximation of E πn [h(θ)] Selection Step: the resampling of the particles (good) equalizes the particle weights and thereby increases accuracy of subsequent importance sampling approximations; (not good) adds a bit of noise to the MC approximation. Mutation Step: changes particle values adapts particles to posterior π n(θ); imagine we don t do it: then we would be using draws from prior p(θ) to approximate posterior π(θ), which can t be good! n θ

29 More on Transition Kernel in Mutation Step Transition kernel K n (θ ˆθ n 1 ; ζ n ): generated by running M steps of a Metropolis-Hastings algorithm. Lessons from DSGE model MCMC: blocking of parameters can reduces persistence of Markov chain; mixture proposal density avoids getting stuck. Blocking: Partition the parameter vector θ n into N blocks equally sized blocks, denoted by θ n,b, b = 1,..., N blocks. (We generate the blocks for n = 1,..., N φ randomly prior to running the SMC algorithm.) Example: random walk proposal density: ( ) ϑ b (θn,b,m 1, i θn, b,m, i Σ n,b) N θn,b,m 1, i cnσ 2 n,b.

30 Adaptive Choice of ζ n = (Σ n, c n ) Infeasible adaption: Let Σ n = V πn [θ]. Adjust scaling factor according to c n = c n 1f ( 1 R n 1(ζ n 1) ), where R n 1( ) is population rejection rate from iteration n 1 and e16(x 0.25) f (x) = e. 16(x 0.25) Feasible adaption use output from stage n 1 to replace ζ n by ˆζ n : Use particle approximations of E πn [θ] and V πn [θ] based on {θ i n 1, W i n} N i=1. Use actual rejection rate from stage n 1 to calculate ĉ n = ĉ n 1f ( ˆRn 1(ˆζ n 1) ).

31 More on Resampling So far, we have used multinomial resampling. It s fairly intuitive and it is straightforward to obtain a CLT. But: multinominal resampling is not particularly efficient. The Herbst-Schorfheide book contains a section on alternative resampling schemes (stratified resampling, residual resampling...) These alternative techniques are designed to achieve a variance reduction. Most resampling algorithms are not parallelizable because they rely on the normalized particle weights.

32 Application 1: Small Scale New Keynesian Model We will take a look at the effect of various tuning choices on accuracy: Tempering schedule λ: λ = 1 is linear, λ > 1 is convex. Number of stages N φ versus number of particles N.

33 Effect of λ on Inefficiency Factors InEff N [ θ] λ Notes: The figure depicts hairs of InEff N [ θ] as function of λ. The inefficiency factors are computed based on N run = 50 runs of the SMC algorithm. Each hair corresponds to a DSGE model parameter.

34 Number of Stages N φ vs Number of Particles N ρ g σ r ρ r ρ z σ g κ σ z π (A) ψ 1 γ (Q) ψ 2 r (A) τ N φ = 400, N = 250 N φ = 200, N = 500 N φ = 100, N = 1000 N φ = 50, N = 2000 N φ = 25, N = 4000 Notes: Plot of V[ θ]/v π [θ] for a specific configuration of the SMC algorithm. The inefficiency factors are computed based on N run = 50 runs of the SMC algorithm. N blocks = 1, λ = 2, N MH = 1.

35 A Few Words on Posterior Model Probabilities Posterior model probabilities π i,t = π i,0 p(y 1:T M i ) M j=1 π j,0p(y 1:T M j ) where p(y 1:T M i ) = p(y 1:T θ (i), M i )p(θ (i) M i )dθ (i) For any model: ln p(y 1:T M i ) = T t=1 ln p(y t θ (i), Y 1:t 1, M i )p(θ (i) Y 1:t 1, M i )dθ (i) Marginal data density p(y 1:T M i ) arises as a by-product of SMC.

36 Marginal Likelihood Approximation Recall w i n = [p(y θ i n 1 )]φn φn 1. Then 1 N N w nw i n 1 i i=1 = p φn 1 (Y θ)p(θ) [p(y θ)] φn φn 1 dθ p φ n 1(Y θ)p(θ)dθ p(y θ) φ n p(θ)dθ p(y θ) φ n 1p(θ)dθ Thus, N φ ( ) 1 N w i N nwn 1 i n=1 i=1 p(y θ)p(θ)dθ.

37 SMC Marginal Data Density Estimates N φ = 100 N φ = 400 N Mean(ln ˆp(Y )) SD(ln ˆp(Y )) Mean(ln ˆp(Y )) SD(ln ˆp(Y )) (3.18) (0.20) 1, (1.98) (0.14) 2, (1.65) (0.12) 4, (0.92) (0.07) Notes: Table shows mean and standard deviation of log marginal data density estimates as a function of the number of particles N computed over N run = 50 runs of the SMC sampler with N blocks = 4, λ = 2, and N MH = 1.

38 Application 2: Estimation of Smets and Wouters (2007) Model Benchmark macro model, has been estimated many (many) times. Core of many larger-scale models. 36 estimated parameters. RWMH: 10 million draws (5 million discarded); SMC: 500 stages with 12,000 particles. We run the RWM (using a particular version of a parallelized MCMC) and the SMC algorithm on 24 processors for the same amount of time. We estimate the SW model twenty times using RWM and SMC and get essentially identical results.

39 Application 2: Estimation of Smets and Wouters (2007) Model More interesting question: how does quality of posterior simulators change as one makes the priors more diffuse? Replace Beta by Uniform distributions; increase variances of parameters with Gamma and Normal prior by factor of 3.

40 SW Model with DIFFUSE Prior: Estimation stability RWH (black) versus SMC (red) l ι w µ p µ w ρ w ξ w r π

41 A Measure of Effective Number of Draws Suppose we could generate iid N eff ( Ê π [θ] approx N E π [θ], 1 N eff V π [θ] draws from posterior, then ). We can measure the variance of Ê π [θ] by running SMC and RWM algorithm repeatedly. Then, N eff V π[θ] V [ Ê π [θ] ]

42 Effective Number of Draws SMC RWMH Parameter Mean STD(Mean) N eff Mean STD(Mean) N eff σ l l ι p h Φ r π ρ b ϕ σ p ξ p ι w µ p ρ w µ w ξ w

43 A Closer Look at the Posterior: Two Modes Parameter Mode 1 Mode 2 ξ w ι w ρ w µ w Log Posterior Mode 1 implies that wage persistence is driven by extremely exogenous persistent wage markup shocks. Mode 2 implies that wage persistence is driven by endogenous amplification of shocks through the wage Calvo and indexation parameter. SMC is able to capture the two modes.

44 A Closer Look at the Posterior: Internal ξ w versus External ρ w Propagation ρ w ξ w

45 Stability of Posterior Computations: RWH (black) versus SMC (red) P (ξ w > ρ w) P (ρ w > µ w) P (ξ w > µ w) P (ξ p > ρ p) P (ρ p > µ p) P (ξ p > µ p)