Sequential Monte Carlo Methods (for DSGE Models)

Transcription

1 Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017

2 Some References These lectures use material from our joint work: Tempered Particle Filtering, 2016, PIER Working Paper, Bayesian Estimation of DSGE Models, 2015, Princeton University Press Sequential Monte Carlo Sampling for DSGE Models, 2014, Journal of Econometrics

3 Sequential Monte Carlo (SMC) Methods SMC can help to Lecture 1 approximate the posterior of θ: Chopin (2002)... Durham and Geweke (2013)... Creal (2007), Herbst and Schorfheide (2014) Lecture 2 approximate the likelihood function (particle filtering): Gordon, Salmond, and Smith (1993)... Fernandez-Villaverde and Rubio-Ramirez (2007) or both: SMC 2 : Chopin, Jacob, and Papaspiliopoulos (2012)... Herbst and Schorfheide (2015)

4 Lecture 2

5 Filtering - General Idea State-space representation of nonlinear DSGE model Measurement Eq. : y t = Ψ(s t, t; θ) + u t, u t F u ( ; θ) State Transition : s t = Φ(s t 1, ɛ t ; θ), ɛ t F ɛ ( ; θ). Likelihood function: T p(y 1:T θ) = p(y t Y 1:t 1, θ) t=1 A filter generates a sequence of conditional distributions s t Y 1:t. Iterations: Initialization at time t 1: p(s t 1 Y 1:t 1, θ) Forecasting t given t 1: 1 Transition equation: p(s t Y 1:t 1, θ) = p(s t s t 1, Y 1:t 1, θ)p(s t 1 Y 1:t 1, θ)ds t 1 2 Measurement equation: p(y t Y 1:t 1, θ) = p(y t s t, Y 1:t 1, θ)p(s t Y 1:t 1, θ)ds t Updating with Bayes theorem. Once y t becomes available: p(s t Y 1:t, θ) = p(s t y t, Y 1:t 1, θ) = p(yt st, Y1:t 1, θ)p(st Y1:t 1, θ) p(y t Y 1:t 1, θ)

6 Conditional Distributions for Kalman Filter (Linear Gaussian State-Space Model) Distribution s t 1 (Y 1:t 1, θ) N ( s ) t 1 t 1, P t 1 t 1 s t (Y 1:t 1, θ) N ( s t t 1, P t t 1 ) y t (Y 1:t 1, θ) N ( ȳ t t 1, F t t 1 ) s t (Y 1:t, θ) N ( s t t, P t t ) Mean and Variance Given from Iteration t 1 s t t 1 = Φ 1 s t 1 t 1 P t t 1 = Φ 1 P t 1 t 1 Φ 1 + Φ ɛσ ɛ Φ ɛ ȳ t t 1 = Ψ 0 + Ψ 1 t + Ψ 2 s t t 1 F t t 1 = Ψ 2 P t t 1 Ψ 2 + Σ u s t t = s t t 1 + P t t 1 Ψ 2 F 1 t t 1 (y t ȳ t t 1 ) P t t = P t t 1 P t t 1 Ψ 2 F 1 t t 1 Ψ 2P t t 1

7 Approximating the Likelihood Function DSGE models are inherently nonlinear. Sometimes linear approximations are sufficiently accurate... but in other applications nonlinearities may be important: asset pricing; borrowing constraints; zero lower bound on nominal interest rates;... Nonlinear state-space representation requires nonlinear filter: y t = Ψ(s t, t; θ) + u t, u t F u ( ; θ) s t = Φ(s t 1, ɛ t ; θ), ɛ t F ɛ ( ; θ).

8 Particle Filters There are many particle filters... We will focus on three types: Bootstrap PF A generic PF A conditionally-optimal PF

9 Bootstrap Particle Filter 1 Initialization. Draw the initial particles from the distribution s j iid 0 p(s0 ) and set W j 0 = 1, j = 1,..., M. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. Propagate the period t 1 particles {s j t 1, W j t 1 } by iterating the state-transition equation forward: s j t = Φ(s j t 1, ɛj t; θ), ɛ j t F ɛ( ; θ). (1) An approximation of E[h(s t) Y 1:t 1, θ] is given by ĥ t,m = 1 M M j=1 h( s j t )W j t 1. (2)

10 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ). (3) The predictive density p(y t Y 1:t 1, θ) can be approximated by ˆp(y t Y 1:t 1, θ) = 1 M M j=1 w j t W j t 1. (4) If the measurement errors are N(0, Σ u) then the incremental weights take the form { w j t = (2π) n/2 Σ u 1/2 exp 1 ( yt Ψ( s j t, t; θ) ) Σ 1 ( u yt Ψ( s j t, t; θ) )}, 2 where n here denotes the dimension of y t. (5)

11 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ). (6) 3 Updating. Define the normalized weights W j t = 1 M w j t W j t 1 M j=1 w. (7) j t W j t 1 An approximation of E[h(s t) Y 1:t, θ] is given by h t,m = 1 M M j=1 h( s j t ) W j t. (8)

12 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. 3 Updating. 4 Selection (Optional). Resample the particles via multinomial resampling. Let {s j t } M j=1 denote M iid draws from a multinomial distribution characterized by support points and weights { s j t, W j and set W j t = 1 for j =, 1..., M. An approximation of E[h(s t) Y 1:t, θ] is given by t } h t,m = 1 M M j=1 h(s j t )W j t. (9)

13 Likelihood Approximation The approximation of the log likelihood function is given by T ln ˆp(Y 1:T θ) = ln 1 M w t j W j t 1. (10) M t=1 j=1 One can show that the approximation of the likelihood function is unbiased. This implies that the approximation of the log likelihood function is downward biased.

14 The Role of Measurement Errors Measurement errors may not be intrinsic to DSGE model. Bootstrap filter needs non-degenerate p(y t s t, θ) for incremental weights to be well defined. Decreasing the measurement error variance Σ u, holding everything else fixed, increases the variance of the particle weights, and reduces the accuracy of Monte Carlo approximation.

15 Generic Particle Filter 1 Initialization. Same as BS PF 2 Recursion. For t = 1,..., T : 1 Forecasting s t. Draw s j t from density g t( s t s j t 1, θ) and define ω j t = p( sj t s j t 1, θ). (11) g t( s j t s j t 1, θ) An approximation of E[h(s t) Y 1:t 1, θ] is given by ĥ t,m = 1 M h( s j t )ω j M tw j t 1. (12) j=1 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ)ω j t. (13) The predictive density p(y t Y 1:t 1, θ) can be approximated by ˆp(y t Y 1:t 1, θ) = 1 M w j t W j t 1 M. (14) j=1 3 Updating. Same as BS PF 4 Selection. Same as BS PF 3 Likelihood Approximation. Same as BS PF

16 Asymptotics The convergence results can be established recursively, starting from the assumption a.s. h t 1,M E[h(s t 1 ) Y 1:t 1 ], M ( h t 1,M E[h(s t 1 ) Y 1:t 1 ] ) = N ( 0, Ω t 1 (h) ). Forward iteration: draw s t from g t (s t s j t 1 ) = p(s t s j t 1 ). Decompose ĥ t,m E[h(s t ) Y 1:t 1 ] (15) = 1 M ( ) h( s j M t) E p( s j )[h] W j t 1 t M j=1 = I + II, M j=1 ( ) E p( s j )[h]w j t 1 E[h(s t) Y 1:t 1 ] t 1 Both I and II converge to zero (and potentially satisfy CLT).

17 Asymptotics Updating step approximates E[h(s t ) Y 1:t ] = h(st )p(y t s t )p(s t Y 1:t 1 )ds t p(yt s t )p(s t Y 1:t 1 )ds t 1 M M j=1 h( sj t) w t j W j 1 M M j=1 w t j W j t 1 (16) t 1 Define the normalized incremental weights as v t (s t ) = p(y t s t ) p(yt s t )p(s t Y 1:t 1 )ds t. (17) Under suitable regularity conditions, the Monte Carlo approximation satisfies a CLT of the form M ( ht,m E[h(s t ) Y 1:t ] ) (18) = N ( 0, Ω t (h) ), Ω t (h) = ˆΩ t ( vt (s t )(h(s t ) E[h(s t ) Y 1:t ]) ). Distribution of particle weights matters for accuracy! = Resampling!

18 Adapting the Generic PF Conditionally-optimal importance distribution: g t ( s t s j t 1 ) = p( s t y t, s j t 1 ). This is the posterior of s t given s j t 1. Typically infeasible, but a good benchmark. Approximately conditionally-optimal distributions: from linearize version of DSGE model or approximate nonlinear filters. Conditionally-linear models: do Kalman filter updating on a subvector of s t. Example: y t = Ψ 0 (m t ) + Ψ 1 (m t )t + Ψ 2 (m t )s t + u t, u t N(0, Σ u ), s t = Φ 0 (m t ) + Φ 1 (m t )s t 1 + Φ ɛ (m t )ɛ t, ɛ t N(0, Σ ɛ ), where m t follows a discrete Markov-switching process.

19 Nonlinear and Partially Deterministic State Transitions Example: s 1,t = Φ 1 (s t 1, ɛ t ), s 2,t = Φ 2 (s t 1 ), ɛ t N(0, 1). Generic filter requires evaluation of p(s t s t 1 ). Define ς t = [s t, ɛ t] and add identity ɛ t = ɛ t to state transition. Factorize the density p(ς t ς t 1 ) as p(ς t ς t 1 ) = p ɛ (ɛ t )p(s 1,t s t 1, ɛ t )p(s 2,t s t 1 ). where p(s 1,t s t 1, ɛ t ) and p(s 2,t s t 1 ) are pointmasses. Sample innovation ɛ t from g ɛ t (ɛ t s t 1 ). Then ω j t = p( ςj t ς j t 1 ) g t ( ς j t ς j t 1 ) = pɛ j ( ɛ t)p( s j 1,t sj t 1, ɛj t)p( s j 2,t sj t 1 ) gt ɛ ( ɛ j t s j t 1 )p( sj 1,t sj t 1, ɛj t)p( s j 2,t sj t 1 ) = pɛ j ( ɛ t) gt ɛ ( ɛ j t s j t 1 ).

20 Degenerate Measurement Error Distributions Our discussion of the conditionally-optimal importance distribution suggests that in the absence of measurement errors, one has to solve the system of equations y t = Ψ ( Φ(s j t 1, ɛj t) ), to determine ɛ j t as a function of s j t 1 and the current observation y t. Then define ω j t = p ɛ ( ɛ j t) and s j t = Φ(s j t 1, ɛj t). Difficulty: one has to find all solutions to a nonlinear system of equations. While resampling duplicates particles, the duplicated particles do not mutate, which can lead to a degeneracy.

21 Next Steps We will now apply PFs to linearized DSGE models. This allows us to compare the Monte Carlo approximation to the truth. Small-scale New Keynesian DSGE model Smets-Wouters model

22 Illustration 1: Small-Scale DSGE Model Parameter Values For Likelihood Evaluation Parameter θ m θ l Parameter θ m θ l τ κ ψ ψ ρ r ρ g ρ z r (A) π (A) γ (Q) σ r σ g σ z ln p(y θ)

23 Likelihood Approximation ln ˆp(y t Y 1:t 1, θ m ) vs. ln p(y t Y 1:t 1, θ m ) Notes: The results depicted in the figure are based on a single run of the bootstrap PF (dashed, M = 40, 000), the conditionally-optimal PF (dotted, M = 400), and the Kalman filter (solid).

24 Filtered State 4 Ê[ĝ t Y 1:t, θ m ] vs. E[ĝ t Y 1:t, θ m ] Notes: The results depicted in the figure are based on a single run of the bootstrap PF (dashed, M = 40, 000), the conditionally-optimal PF (dotted, M = 400), and the Kalman filter (solid).

25 Distribution of Log-Likelihood Approximation Errors Bootstrap PF: θ m vs. θ l Notes: Density estimate of ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) based on N run = 100 runs of the PF. Solid line is θ = θ m ; dashed line is θ = θ l (M = 40, 000).

26 Distribution of Log-Likelihood Approximation Errors Notes: Density estimate of ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) based on N run = 100 runs of the PF. Solid line is bootstrap particle filter (M = 40, 000); dotted line is conditionally optimal particle filter (M = 400) θ m : Bootstrap vs. Cond. Opt. PF

27 Summary Statistics for Particle Filters Bootstrap Cond. Opt. Auxiliary Number of Particles M 40, ,000 Number of Repetitions High Posterior Density: θ = θ m Bias ˆ StdD ˆ Bias ˆ Low Posterior Density: θ = θ l Bias ˆ StdD ˆ Bias ˆ Notes: ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) and ˆ 2 = exp[ln ˆp(Y 1:T θ) ln p(y 1:T θ)] 1. Results are based on N run = 100 runs of the particle filters.

28 Great Recession and Beyond Mean of Log-likelihood Increments ln ˆp(y t Y 1:t 1, θ m ) Notes: Solid lines represent results from Kalman filter. Dashed lines correspond to bootstrap particle filter (M = 40, 000) and dotted lines correspond to conditionally-optimal particle filter (M = 400). Results are based on N run = 100 runs of the filters.

29 Great Recession and Beyond Mean of Log-likelihood Increments ln ˆp(y t Y 1:t 1, θ m ) Notes: Solid lines represent results from Kalman filter. Dashed lines correspond to bootstrap particle filter (M = 40, 000) and dotted lines correspond to conditionally-optimal particle filter (M = 400). Results are based on N run = 100 runs of the filters.

30 Great Recession and Beyond Log Standard Dev of Log-Likelihood Increments Notes: Solid lines represent results from Kalman filter. Dashed lines correspond to bootstrap particle filter (M = 40, 000) and dotted lines correspond to conditionally-optimal particle filter (M = 400). Results are based on N run = 100 runs of the filters.

31 SW Model: Distr. of Log-Likelihood Approximation Errors BS (M = 40, 000) versus CO (M = 4, 000) Notes: Density estimates of ˆ 1 = ln ˆp(Y θ) ln p(y θ) based on N run = 100. Solid densities summarize results for the bootstrap (BS) particle filter; dashed densities summarize results for the conditionally-optimal (CO) particle filter.

32 SW Model: Distr. of Log-Likelihood Approximation Errors BS (M = 400, 000) versus CO (M = 4, 000) Notes: Density estimates of ˆ 1 = ln ˆp(Y θ) ln p(y θ) based on N run = 100. Solid densities summarize results for the bootstrap (BS) particle filter; dashed densities summarize results for the conditionally-optimal (CO) particle filter.

33 SW Model: Summary Statistics for Particle Filters Bootstrap Cond. Opt. Number of Particles M 40, ,000 4,000 40,000 Number of Repetitions High Posterior Density: θ = θ m Bias ˆ StdD ˆ Bias ˆ Low Posterior Density: θ = θ l Bias ˆ StdD ˆ Bias ˆ Notes: ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) and ˆ 2 = exp[ln ˆp(Y 1:T θ) ln p(y 1:T θ)] 1. Results are based on N run = 100.

34 Embedding PF Likelihoods into Posterior Samplers Likelihood functions for nonlinear DSGE models can be approximated by the PF. We will now embed the likelihood approximation into a posterior sampler: PFMH Algorithm (a special case of PMCMC). The book also discusses SMC 2.

35 Embedding PF Likelihoods into Posterior Samplers Distinguish between: {p(y θ), p(θ Y ), p(y )}, which are related according to: p(y θ)p(θ) p(θ Y ) =, p(y ) = p(y θ)p(θ)dθ p(y ) {ˆp(Y θ), ˆp(θ Y ), ˆp(Y )}, which are related according to: ˆp(Y θ)p(θ) ˆp(θ Y ) =, ˆp(Y ) = ˆp(Y θ)p(θ)dθ. ˆp(Y ) Surprising result (Andrieu, Docet, and Holenstein, 2010): under certain conditions we can replace p(y θ) by ˆp(Y θ) and still obtain draws from p(θ Y ).

36 PFMH Algorithm For i = 1 to N: 1 Draw ϑ from a density q(ϑ θ i 1 ). 2 Set θ i = ϑ with probability { α(ϑ θ i 1 ) = min 1, ˆp(Y ϑ)p(ϑ)/q(ϑ θ i 1 } ) ˆp(Y θ i 1 )p(θ i 1 )/q(θ i 1 ϑ) and θ i = θ i 1 otherwise. The likelihood approximation ˆp(Y ϑ) is computed using a particle filter.

37 Why Does the PFMH Work? At each iteration the filter generates draws s j t from the proposal distribution g t ( s j t 1 ). Let S t = ( s t 1,..., s t M S 1:T 1:M. ) and denote the entire sequence of draws by Selection step: define a random variable A j t that contains this ancestry information. For instance, suppose that during the resampling particle j = 1 was assigned the value s t 10 then A 1 t = 10. Let A t = ( ) A 1 t,..., A N t and use A1:T to denote the sequence of A t s. PFMH operates on an enlarged probability space: θ, S 1:T and A 1:T.

38 Why Does the PFMH Work? Use U 1:T to denote random vectors for S 1:T and A 1:T. U 1:T is an array of iid uniform random numbers. The transformation of U 1:T into ( S 1:T, A 1:T ) typically depends on θ and Y 1:T, because the proposal distribution g t ( s t s j t 1 ) depends both on the current observation y t as well as the parameter vector θ. E.g., implementation of conditionally-optimal PF requires sampling from a N( s j t t, P t t) distribution for each particle j. Can be done using a prob integral transform of uniform random variables. We can express the particle filter approximation of the likelihood function as where ˆp(Y 1:T θ) = g(y 1:T θ, U 1:T ). U 1:T p(u 1:T ) = T p(u t ). t=1

39 Why Does the PFMH Work? Define the joint distribution ( ) p g Y1:T, θ, U 1:T = g(y1:t θ, U 1:T )p ( ) U 1:T p(θ). The PFMH algorithm samples from the joint posterior ( ) p g θ, U1:T Y 1:T g(y θ, U1:T )p ( ) U 1:T p(θ) and discards the draws of ( ) U 1:T. For this procedure to be valid, it needs to be the case that PF approximation is unbiased: E[ˆp(Y 1:T θ)] = g(y 1:T θ, U 1:T )p ( ) U 1:T dθ = p(y1:t θ).

40 Why Does the PFMH Work? We can express acceptance probability directly in terms of ˆp(Y 1:T θ). Need to generate a proposed draw for both θ and U 1:T : ϑ and U 1:T. The proposal distribution for (ϑ, U1:T ) in the MH algorithm is given by q(ϑ θ (i 1) )p(u1:t ). No need to keep track of the draws (U 1:T ). MH acceptance probability: g(y ϑ,u )p(u )p(ϑ) α(ϑ θ i 1 ) = min 1, q(ϑ θ (i 1) )p(u ) g(y θ (i 1),U (i 1) )p(u (i 1) )p(θ (i 1) ) q(θ (i 1) θ )p(u (i 1) ) { ˆp(Y ϑ)p(ϑ) / } q(ϑ θ (i 1) ) = min 1, ˆp(Y θ (i 1) )p(θ (i 1) ) /. q(θ (i 1) ϑ)

41 Small-Scale DSGE: Accuracy of MH Approximations Results are based on N run = 20 runs of the PF-RWMH-V algorithm. Each run of the algorithm generates N = 100, 000 draws and the first N 0 = 50, 000 are discarded. The likelihood function is computed with the Kalman filter (KF), bootstrap particle filter (BS-PF, M = 40, 000) or conditionally-optimal particle filter (CO-PF, M = 400). Pooled means that we are pooling the draws from the N run = 20 runs to compute posterior statistics.

42 Autocorrelation of PFMH Draws τ κ σr ρr ψ ψ Notes: The figure depicts autocorrelation functions computed from the output of the 1 Block RWMH-V algorithm based on the Kalman filter (solid), the conditionally-optimal particle filter (dashed) and the bootstrap particle filter (solid with dots).

43 Small-Scale DSGE: Accuracy of MH Approximations Posterior Mean (Pooled) Inefficiency Factors Std Dev of Means KF CO-PF BS-PF KF CO-PF BS-PF KF CO-PF BS-PF τ κ ψ ψ ρ r ρ g ρ z r (A) π (A) γ (Q) σ r σ g σ z ln ˆp(Y )

44 SW Model: Accuracy of MH Approximations Results are based on N run = 20 runs of the PF-RWMH-V algorithm. Each run of the algorithm generates N = 10, 000 draws. The likelihood function is computed with the Kalman filter (KF) or conditionally-optimal particle filter (CO-PF). Pooled means that we are pooling the draws from the N run = 20 runs to compute posterior statistics. The CO-PF uses M = 40, 000 particles to compute the likelihood.

45 SW Model: Accuracy of MH Approximations Post. Mean (Pooled) Ineff. Factors Std Dev of Means KF CO-PF KF CO-PF KF CO-PF (100β 1 1) π l α σ c Φ ϕ h ξ w σ l ξ p ι w ι p ψ r π ρ r y r y

46 SW Model: Accuracy of MH Approximations Post. Mean (Pooled) Ineff. Factors Std Dev of Means KF CO-PF KF CO-PF KF CO-PF ρ a ρ b ρ g ρ i ρ r ρ p ρ w ρ ga µ p µ w σ a σ b σ g σ i σ r σ p σ w ln ˆp(Y )

47 Conclusion Versatile set of tools... SMC can be used to compute Laplace-type estimators. Particle filters can be used for other nonlinear state-space models. If you are interested in more: Tempered Particle Filtering presentation in Econometrics Workshop on Wednesday.