Bayesian Computations for DSGE Models
|
|
- Nora Ellis
- 6 years ago
- Views:
Transcription
1 Bayesian Computations for DSGE Models Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017
2 This Lecture is Based on Bayesian Estimation of DSGE Models Edward P. Herbst & Frank Schorfheide companion-web-site-bayesian-estimation-of-dsge-models
3 Some Background DSGE model: dynamic model of the macroeconomy, indexed by θ vector of preference and technology parameters. Used for forecasting, policy experiments, interpreting past events. Ingredients of Bayesian Analysis: Likelihood function p(y θ) Prior density p(θ) Marginal data density p(y ) = p(y θ)p(θ)dφ Bayes Theorem: p(θ Y ) = p(y θ)p(θ) p(y ) p(y θ)p(θ)
4 Some Background Implementation: usually by generating a sequence of draws (not necessarily iid) from posterior θ i p(θ Y ), i = 1,..., N Algorithms: direct sampling, accept/reject sampling, importance sampling, Markov chain Monte Carlo sampling, sequential Monte Carlo sampling... Draws can then be transformed into objects of interest, h(θ i ), and under suitable conditions a Monte Carlo average of the form h N = 1 N N i=1 h(θ i ) E π [h] h(θ)p(θ Y )dθ. Strong law of large numbers (SLLN), central limit theorem (CLT)...
5 Bayesian Inference Decision Making The posterior expected loss of decision δ( ): ρ ( δ( ) Y ) = L ( θ, δ(y ) ) p(θ Y )dθ. Θ Bayes decision minimizes the posterior expected loss: δ (Y ) = argmin d ρ ( δ( ) Y ). Approximate ρ ( δ( ) Y ) by a Monte Carlo average ρ N ( δ( ) Y ) = 1 N Then compute N L ( θ i, δ( ) ). i=1 δ N(Y ) = argmin d ρ N ( δ( ) Y ).
6 Bayesian Inference Point estimation: Quadratic loss: posterior mean Absolute error loss: posterior median Interval/Set estimation P π {θ C(Y )} = 1 α: highest posterior density sets equal-tail-probability intervals
7 Computational Challenges Numerical solution of model leads to state-space representation = likelihood approximation = posterior sampler. Standard approach for (linearized) models Model solution: log-linearize and use linear rational expectations system solver. Evaluation of p(y θ): Kalman filter Posterior draws θ i : MCMC Book reviews the standard approach, but also studies more recently developed sequential Monte Carlo (SMC) techniques.
8 Sequential Monte Carlo (SMC) Methods SMC can help to Approximate the likelihood function (particle filtering): Gordon, Salmond, and Smith (1993)... Fernandez-Villaverde and Rubio-Ramirez (2007) approximate the posterior of θ: Chopin (2002)... Durham and Geweke (2013)... Creal (2007), Herbst and Schorfheide (2014) or both: SMC 2 : Chopin, Jacob, and Papaspiliopoulos (2012)... Herbst and Schorfheide (2015)
9 Review Importance Sampling
10 Importance Sampling Approximate π( ) by using a different, tractable density g(θ) that is easy to sample from. For more general problems, posterior density may be unnormalized. So we write π(θ) = p(y θ)p(θ) p(y ) = f (θ) f (θ)dθ. Importance sampling is based on the identity E π [h(θ)] = h(θ)π(θ)dθ = f (θ) h(θ) Θ Θ g(θ) g(θ)dθ f (θ) g(θ) g(θ)dθ. (Unnormalized) importance weight: w(θ) = f (θ) g(θ).
11 Importance Sampling 1 For i = 1 to N, draw θ i iid g(θ) and compute the unnormalized importance weights w i = w(θ i ) = f (θi ) g(θ i ). 2 Compute the normalized importance weights W i = 1 N w i N i=1 w i. An approximation of E π [h(θ)] is given by h N = 1 N N W i h(θ i ). i=1
12 Illustration If θ i s are draws from g( ) then E π [h] 1 N N i=1 h(θi )w(θ i ) 1 N N i=1 w(θi ), w(θ) = f (θ) g(θ) f 0.30 g g weights f/g 1 f/g
13 Accuracy Since we are generating iid draws from g(θ), it s fairly straightforward to derive a CLT: N( h N E π [h]) = N ( 0, Ω(h) ), where Ω(h) = V g [(π/g)(h E π [h])]. Using a crude approximation (see, e.g., Liu (2008)), we can factorize Ω(h) as follows: Ω(h) V π [h] ( 1 + V g [π/g] ). The approximation highlights that the larger the variance of the importance weights, the less accurate the Monte Carlo approximation relative to the accuracy that could be achieved with an iid sample from the posterior. Users often monitor ESS = N V π[h] Ω(h) N 1 + V g [π/g].
14 Likelihood Approximation
15 State-Space Representation and Likelihood Measurement Equation: y t = Ψ(s t ; θ) +u }{{} t optional State transition: s t = Φ(s t 1, ɛ t ; θ) Joint density for the observations and latent states: T T p(y 1:T, S 1:T θ) = p(y t, s t Y 1:t 1, S 1:t 1, θ) = p(y t s t, θ)p(s t s t 1, θ). t=1 Need to compute the marginal p(y 1:T θ). t=1
16 Filtering - General Idea State-space representation of nonlinear DSGE model Measurement Eq. : y t = Ψ(s t, t; θ) + u t, u t F u ( ; θ) State Transition : s t = Φ(s t 1, ɛ t ; θ), ɛ t F ɛ ( ; θ). Likelihood function: p(y 1:T θ) = T p(y t Y 1:t 1, θ) t=1 A filter generates a sequence of conditional distributions s t Y 1:t. Iterations: Initialization at time t 1: p(s t 1 Y 1:t 1, θ) Forecasting t given t 1: 1 Transition equation: p(s t Y 1:t 1, θ) = p(s t s t 1, Y 1:t 1, θ)p(s t 1 Y 1:t 1, θ)ds t 1 2 Measurement equation: p(y t Y 1:t 1, θ) = p(y t s t, Y 1:t 1, θ)p(s t Y 1:t 1, θ)ds t Updating with Bayes theorem. Once y t becomes available: p(s t Y 1:t, θ) = p(s t y t, Y 1:t 1, θ) = p(yt st, Y1:t 1, θ)p(st Y1:t 1, θ) p(y t Y 1:t 1, θ)
17 Conditional Distributions for Kalman Filter (Linear Gaussian State-Space Model) Distribution s t 1 (Y 1:t 1, θ) N ( s ) t 1 t 1, P t 1 t 1 s t (Y 1:t 1, θ) N ( s t t 1, P t t 1 ) y t (Y 1:t 1, θ) N ( ȳ t t 1, F t t 1 ) s t (Y 1:t, θ) N ( s t t, P t t ) Mean and Variance Given from Iteration t 1 s t t 1 = Φ 1 s t 1 t 1 P t t 1 = Φ 1 P t 1 t 1 Φ 1 + Φ ɛσ ɛ Φ ɛ ȳ t t 1 = Ψ 0 + Ψ 1 t + Ψ 2 s t t 1 F t t 1 = Ψ 2 P t t 1 Ψ 2 + Σ u s t t = s t t 1 + P t t 1 Ψ 2 F 1 t t 1 (y t ȳ t t 1 ) P t t = P t t 1 P t t 1 Ψ 2 F 1 t t 1 Ψ 2P t t 1
18 Bootstrap Particle Filter 1 Initialization. Draw the initial particles from the distribution s j iid 0 p(s0 ) and set W j 0 = 1, j = 1,..., M. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. Propagate the period t 1 particles {s j t 1, W j t 1 } by iterating the state-transition equation forward: s j t = Φ(s j t 1, ɛj t; θ), ɛ j t F ɛ( ; θ). (1) An approximation of E[h(s t) Y 1:t 1, θ] is given by ĥ t,m = 1 M M j=1 h( s j t )W j t 1. (2)
19 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ). (3) The predictive density p(y t Y 1:t 1, θ) can be approximated by ˆp(y t Y 1:t 1, θ) = 1 M M j=1 w j t W j t 1. (4) If the measurement errors are N(0, Σ u) then the incremental weights take the form { w j t = (2π) n/2 Σ u 1/2 exp 1 ( yt Ψ( s j t, t; θ) ) Σ 1 ( u yt Ψ( s j t, t; θ) )}, (5) 2 where n here denotes the dimension of y t.
20 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ). (6) 3 Updating. Define the normalized weights W j t = 1 M w j t W j t 1 M j=1 w. j t W j t 1 (7) An approximation of E[h(s t) Y 1:t, θ] is given by h t,m = 1 M M j=1 h( s j t ) W j t. (8)
21 Bootstrap Particle Filter 1 Initialization. 2 Recursion. For t = 1,..., T : 1 Forecasting s t. 2 Forecasting y t. 3 Updating. 4 Selection (Optional). Resample the particles via multinomial resampling. Let {s j t } M j=1 denote M iid draws from a multinomial distribution characterized by support points and weights { s j t, W j t } and set W j t = 1 for j =, 1..., M. An approximation of E[h(s t) Y 1:t, θ] is given by h t,m = 1 M M j=1 h(s j t )W j t. (9)
22 Likelihood Approximation The approximation of the log likelihood function is given by T ln ˆp(Y 1:T θ) = ln 1 M w t j W j t 1. (10) M t=1 j=1 One can show that the approximation of the likelihood function is unbiased. This implies that the approximation of the log likelihood function is downward biased.
23 The Role of Measurement Errors Measurement errors may not be intrinsic to DSGE model. Bootstrap filter needs non-degenerate p(y t s t, θ) for incremental weights to be well defined. Decreasing the measurement error variance Σ u, holding everything else fixed, increases the variance of the particle weights, and reduces the accuracy of Monte Carlo approximation.
24 Generic Particle Filter 1 Initialization. Same as BS PF 2 Recursion. For t = 1,..., T : 1 Forecasting s t. Draw s j t from density g t( s t s j t 1, θ) and define ω j t = p( sj t s j t 1, θ) g t( s j t s j t 1, θ). (11) An approximation of E[h(s t) Y 1:t 1, θ] is given by ĥ t,m = 1 M h( s j t )ω j M tw j t 1. (12) j=1 2 Forecasting y t. Define the incremental weights w j t = p(y t s j t, θ)ω j t. (13) The predictive density p(y t Y 1:t 1, θ) can be approximated by 3 (...) ˆp(y t Y 1:t 1, θ) = 1 M M j=1 w j t W j t 1. (14)
25 Adapting the Generic PF Conditionally-optimal importance distribution: g t ( s t s j t 1 ) = p( s t y t, s j t 1 ). This is the posterior of s t given s j t 1. Typically infeasible, but a good benchmark. Approximately conditionally-optimal distributions: from linearize version of DSGE model or approximate nonlinear filters. Conditionally-linear models: do Kalman filter updating on a subvector of s t. Example: y t = Ψ 0 (m t ) + Ψ 1 (m t )t + Ψ 2 (m t )s t + u t, u t N(0, Σ u ), s t = Φ 0 (m t ) + Φ 1 (m t )s t 1 + Φ ɛ (m t )ɛ t, ɛ t N(0, Σ ɛ ), where m t follows a discrete Markov-switching process.
26 Distribution of Log-Likelihood Approximation Errors Bootstrap PF: θ m vs. θ l Notes: Density estimate of ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) based on N run = 100 runs of the PF. Solid line is θ = θ m ; dashed line is θ = θ l (M = 40, 000).
27 Distribution of Log-Likelihood Approximation Errors θ m : Bootstrap vs. Cond. Opt. PF Notes: Density estimate of ˆ 1 = ln ˆp(Y 1:T θ) ln p(y 1:T θ) based on N run = 100 runs of the PF. Solid line is bootstrap particle filter (M = 40, 000); dotted line is conditionally optimal particle filter (M = 400).
28 Great Recession and Beyond Log Standard Dev of Log-Likelihood Increments Notes: Solid lines represent results from Kalman filter. Dashed lines correspond to bootstrap particle filter (M = 40, 000) and dotted lines correspond to conditionally-optimal particle filter (M = 400). Results are based on N run = 100 runs of the filters.
29 Posterior Sampling
30 Implementation: Sampling from Posterior DSGE model posteriors are often non-elliptical, e.g., multimodal posteriors may arise because it is difficult to disentangle internal and external propagation mechanisms; disentangle the relative importance of shocks. θ θ Economic Example: is wage growth persistent because 1 wage setters find it very costly to adjust wages? 2 exogenous shocks affect the substitutability of labor inputs and hence markups?
31 Sampling from Posterior If posterior distributions are irregular, standard MCMC methods can be inaccurate (examples will follow). SMC samplers often generate more precise approximations of posteriors in the same amount of time. SMC can be parallelized. SMC = importance sampling on steroids For now we assume that the likelihood evaluation is exact.
32 From Importance Sampling to Sequential Importance Sampling In general, it s hard to construct a good proposal density g(θ), especially if the posterior has several peaks and valleys. Idea - Part 1: it might be easier to find a proposal density for π n (θ) = [p(y θ)]φn p(θ) = f n(θ). [p(y θ)] φ np(θ)dθ Z n at least if φ n is close to zero. Idea - Part 2: We can try to turn a proposal density for π n into a proposal density for π n+1 and iterate, letting φ n φ N = 1.
33 Illustration: Tempered Posteriors of θ n θ π n (θ) = [p(y p(θ) θ)]φn = f ( ) λ n(θ) n, φ [p(y θ)] φ np(θ)dθ n = Z n N φ
34 SMC Algorithm: A Graphical Illustration C S M C S M C S M φ 0 φ 1 φ 2 φ 3 π n (θ) is represented by a swarm of particles {θ i n, W i n} N i=1 : h n,n = 1 N N i=1 W i nh(θ i n) a.s. E πn [h(θ n )]. C is Correction; S is Selection; and M is Mutation.
35 Remarks Correction Step: reweight particles from iteration n 1 to create importance sampling approximation of E πn [h(θ)] Selection Step: the resampling of the particles (good) equalizes the particle weights and thereby increases accuracy of subsequent importance sampling approximations; (not good) adds a bit of noise to the MC approximation. Mutation Step: changes particle values adapts particles to posterior π n(θ); imagine we don t do it: then we would be using draws from prior p(θ) to approximate posterior π(θ), which can t be good! n θ1
36 More on Transition Kernel in Mutation Step Transition kernel K n (θ ˆθ n 1 ; ζ n ): generated by running M steps of a Metropolis-Hastings algorithm. Lessons from DSGE model MCMC: blocking of parameters can reduces persistence of Markov chain; mixture proposal density avoids getting stuck. Blocking: Partition the parameter vector θ n into N blocks equally sized blocks, denoted by θ n,b, b = 1,..., N blocks. (We generate the blocks for n = 1,..., N φ randomly prior to running the SMC algorithm.) Example: random walk proposal density: ( ) ϑ b (θn,b,m 1, i θn, b,m, i Σ n,b) N θn,b,m 1, i cnσ 2 n,b.
37 Application: Estimation of Smets and Wouters (2007) Model Benchmark macro model, has been estimated many (many) times. Core of many larger-scale models. 36 estimated parameters. RWMH: 10 million draws (5 million discarded); SMC: 500 stages with 12,000 particles. We run the RWM (using a particular version of a parallelized MCMC) and the SMC algorithm on 24 processors for the same amount of time. We estimate the SW model twenty times using RWM and SMC and get essentially identical results.
38 Application: Estimation of Smets and Wouters (2007) Model More interesting question: how does quality of posterior simulators change as one makes the priors more diffuse? Replace Beta by Uniform distributions; increase variances of parameters with Gamma and Normal prior by factor of 3.
39 SW Model with DIFFUSE Prior: Estimation stability RWH (black) versus SMC (red) l ι w µ p µ w ρ w ξ w r π
40 A Measure of Effective Number of Draws Suppose we could generate iid N eff ( Ê π [θ] approx N E π [θ], 1 N eff V π [θ] draws from posterior, then ). We can measure the variance of Ê π [θ] by running SMC and RWM algorithm repeatedly. Then, N eff V π[θ] V [ Ê π [θ] ]
41 Effective Number of Draws SMC RWMH Parameter Mean STD(Mean) N eff Mean STD(Mean) N eff σ l l ι p h Φ r π ρ b ϕ σ p ξ p ι w µ p ρ w µ w ξ w
42 A Closer Look at the Posterior: Two Modes Parameter Mode 1 Mode 2 ξ w ι w ρ w µ w Log Posterior Mode 1 implies that wage persistence is driven by extremely exogenous persistent wage markup shocks. Mode 2 implies that wage persistence is driven by endogenous amplification of shocks through the wage Calvo and indexation parameter. SMC is able to capture the two modes.
43 A Closer Look at the Posterior: Internal ξ w versus External ρ w Propagation ρ w ξ w
44 Stability of Posterior Computations: RWH (black) versus SMC (red) P (ξ w > ρ w) P (ρ w > µ w) P (ξ w > µ w) P (ξ p > ρ p) P (ρ p > µ p) P (ξ p > µ p)
45 Embedding PF Likelihoods into Posterior Samplers Particle MCMC, SMC 2. Distinguish between: {p(y θ), p(θ Y ), p(y )}, which are related according to: p(y θ)p(θ) p(θ Y ) =, p(y ) = p(y θ)p(θ)dθ p(y ) {ˆp(Y θ), ˆp(θ Y ), ˆp(Y )}, which are related according to: ˆp(Y θ)p(θ) ˆp(θ Y ) =, ˆp(Y ) = ˆp(Y θ)p(θ)dθ. ˆp(Y ) Surprising result (Andrieu, Docet, and Holenstein, 2010): under certain conditions we can replace p(y θ) by ˆp(Y θ) and still obtain draws from p(θ Y ).
Sequential Monte Carlo Methods
University of Pennsylvania Bradley Visitor Lectures October 23, 2017 Introduction Unfortunately, standard MCMC can be inaccurate, especially in medium and large-scale DSGE models: disentangling importance
More informationSequential Monte Carlo Methods (for DSGE Models)
Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 Some References Solution and Estimation of DSGE Models, with J. Fernandez-Villaverde
More informationSequential Monte Carlo Methods (for DSGE Models)
Sequential Monte Carlo Methods (for DSGE Models) Frank Schorfheide University of Pennsylvania, PIER, CEPR, and NBER October 23, 2017 Some References These lectures use material from our joint work: Tempered
More informationSequential Monte Carlo Methods
University of Pennsylvania EABCN Training School May 10, 2016 Introduction Unfortunately, standard MCMC can be inaccurate, especially in medium and large-scale DSGE models: disentangling importance of
More informationSequential Monte Carlo Methods
University of Pennsylvania Econ 722 Part 1 February 13, 2019 Introduction Posterior expectations can be approximated by Monte Carlo averages. If we have draws from {θ s } N s=1 from p(θ Y ), then (under
More informationBayesian Estimation of DSGE Models 1 Chapter 3: A Crash Course in Bayesian Inference
1 The views expressed in this paper are those of the authors and do not necessarily reflect the views of the Federal Reserve Board of Governors or the Federal Reserve System. Bayesian Estimation of DSGE
More informationWORKING PAPER NO SEQUENTIAL MONTE CARLO SAMPLING FOR DSGE MODELS. Edward Herbst Federal Reserve Board
WORKING PAPER NO. 12-27 SEQUENTIAL MONTE CARLO SAMPLING FOR DSGE MODELS Edward Herbst Federal Reserve Board Frank Schorfheide University of Pennsylvania and Visiting Scholar, Federal Reserve Bank of Philadelphia
More informationIntroduction to Bayesian Inference
University of Pennsylvania EABCN Training School May 10, 2016 Bayesian Inference Ingredients of Bayesian Analysis: Likelihood function p(y φ) Prior density p(φ) Marginal data density p(y ) = p(y φ)p(φ)dφ
More informationSequential Monte Carlo Sampling for DSGE Models
Sequential Monte Carlo Sampling for DSGE Models Edward Herbst Federal Reserve Board Frank Schorfheide University of Pennsylvania CEPR, and NBER September 15, 2013 Abstract We develop a sequential Monte
More informationDSGE Model Econometrics
DSGE Model Econometrics Frank Schorfheide University of Pennsylvania, CEPR, NBER, PIER June 2017 Cowles Lunch Papers and software available at https://web.sas.upenn.edu/schorf/ A Potted History 1993-1994:
More informationNBER WORKING PAPER SERIES SEQUENTIAL MONTE CARLO SAMPLING FOR DSGE MODELS. Edward P. Herbst Frank Schorfheide
NBER WORKING PAPER SERIES SEQUENTIAL MONTE CARLO SAMPLING FOR DSGE MODELS Edward P. Herbst Frank Schorfheide Working Paper 19152 http://www.nber.org/papers/w19152 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050
More informationDSGE Model Forecasting
University of Pennsylvania EABCN Training School May 1, 216 Introduction The use of DSGE models at central banks has triggered a strong interest in their forecast performance. The subsequent material draws
More informationThe Metropolis-Hastings Algorithm. June 8, 2012
The Metropolis-Hastings Algorithm June 8, 22 The Plan. Understand what a simulated distribution is 2. Understand why the Metropolis-Hastings algorithm works 3. Learn how to apply the Metropolis-Hastings
More informationPoint, Interval, and Density Forecast Evaluation of Linear versus Nonlinear DSGE Models
Point, Interval, and Density Forecast Evaluation of Linear versus Nonlinear DSGE Models Francis X. Diebold Frank Schorfheide Minchul Shin University of Pennsylvania May 4, 2014 1 / 33 Motivation The use
More informationPiecewise Linear Continuous Approximations and Filtering for DSGE Models with Occasionally-Binding Constraints
The views expressed in this presentation are those of the authors and do not necessarily reflect those of the Board of Governors or the Federal Reserve System. Piecewise Linear Continuous Approximations
More informationSequential Monte Carlo samplers for Bayesian DSGE models
Sequential Monte Carlo samplers for Bayesian DSGE models Drew Creal First version: February 8, 27 Current version: March 27, 27 Abstract Dynamic stochastic general equilibrium models have become a popular
More informationBayesian Inference for DSGE Models. Lawrence J. Christiano
Bayesian Inference for DSGE Models Lawrence J. Christiano Outline State space-observer form. convenient for model estimation and many other things. Preliminaries. Probabilities. Maximum Likelihood. Bayesian
More informationBayesian Inference for DSGE Models. Lawrence J. Christiano
Bayesian Inference for DSGE Models Lawrence J. Christiano Outline State space-observer form. convenient for model estimation and many other things. Bayesian inference Bayes rule. Monte Carlo integation.
More informationExercises Tutorial at ICASSP 2016 Learning Nonlinear Dynamical Models Using Particle Filters
Exercises Tutorial at ICASSP 216 Learning Nonlinear Dynamical Models Using Particle Filters Andreas Svensson, Johan Dahlin and Thomas B. Schön March 18, 216 Good luck! 1 [Bootstrap particle filter for
More informationSMC 2 : an efficient algorithm for sequential analysis of state-space models
SMC 2 : an efficient algorithm for sequential analysis of state-space models N. CHOPIN 1, P.E. JACOB 2, & O. PAPASPILIOPOULOS 3 1 ENSAE-CREST 2 CREST & Université Paris Dauphine, 3 Universitat Pompeu Fabra
More informationState-Space Methods for Inferring Spike Trains from Calcium Imaging
State-Space Methods for Inferring Spike Trains from Calcium Imaging Joshua Vogelstein Johns Hopkins April 23, 2009 Joshua Vogelstein (Johns Hopkins) State-Space Calcium Imaging April 23, 2009 1 / 78 Outline
More informationSequential Monte Carlo Methods for Bayesian Computation
Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter
More informationBayesian Estimation with Sparse Grids
Bayesian Estimation with Sparse Grids Kenneth L. Judd and Thomas M. Mertens Institute on Computational Economics August 7, 27 / 48 Outline Introduction 2 Sparse grids Construction Integration with sparse
More informationSequential Monte Carlo samplers for Bayesian DSGE models
Sequential Monte Carlo samplers for Bayesian DSGE models Drew Creal Department of Econometrics, Vrije Universitiet Amsterdam, NL-8 HV Amsterdam dcreal@feweb.vu.nl August 7 Abstract Bayesian estimation
More informationBayesian Inference for DSGE Models. Lawrence J. Christiano
Bayesian Inference for DSGE Models Lawrence J. Christiano Outline State space-observer form. convenient for model estimation and many other things. Bayesian inference Bayes rule. Monte Carlo integation.
More informationEstimating a Nonlinear New Keynesian Model with the Zero Lower Bound for Japan
Estimating a Nonlinear New Keynesian Model with the Zero Lower Bound for Japan Hirokuni Iiboshi 1, Mototsugu Shintani 2, Kozo Ueda 3 August 218 @ EEA-ESEM 1 Tokyo Metropolitan University 2 University of
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationIntroduction to Bayesian Computation
Introduction to Bayesian Computation Dr. Jarad Niemi STAT 544 - Iowa State University March 20, 2018 Jarad Niemi (STAT544@ISU) Introduction to Bayesian Computation March 20, 2018 1 / 30 Bayesian computation
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee University of Minnesota July 20th, 2008 1 Bayesian Principles Classical statistics: model parameters are fixed and unknown. A Bayesian thinks of parameters
More informationEstimation and Inference on Dynamic Panel Data Models with Stochastic Volatility
Estimation and Inference on Dynamic Panel Data Models with Stochastic Volatility Wen Xu Department of Economics & Oxford-Man Institute University of Oxford (Preliminary, Comments Welcome) Theme y it =
More informationLabor-Supply Shifts and Economic Fluctuations. Technical Appendix
Labor-Supply Shifts and Economic Fluctuations Technical Appendix Yongsung Chang Department of Economics University of Pennsylvania Frank Schorfheide Department of Economics University of Pennsylvania January
More informationMonetary and Exchange Rate Policy Under Remittance Fluctuations. Technical Appendix and Additional Results
Monetary and Exchange Rate Policy Under Remittance Fluctuations Technical Appendix and Additional Results Federico Mandelman February In this appendix, I provide technical details on the Bayesian estimation.
More informationBayesian Estimation of DSGE Models: Lessons from Second-order Approximations
Bayesian Estimation of DSGE Models: Lessons from Second-order Approximations Sungbae An Singapore Management University Bank Indonesia/BIS Workshop: STRUCTURAL DYNAMIC MACROECONOMIC MODELS IN ASIA-PACIFIC
More informationEstimation of DSGE Models under Diffuse Priors and Data- Driven Identification Constraints. Markku Lanne and Jani Luoto
Estimation of DSGE Models under Diffuse Priors and Data- Driven Identification Constraints Markku Lanne and Jani Luoto CREATES Research Paper 2015-37 Department of Economics and Business Economics Aarhus
More informationarxiv: v1 [stat.co] 1 Jun 2015
arxiv:1506.00570v1 [stat.co] 1 Jun 2015 Towards automatic calibration of the number of state particles within the SMC 2 algorithm N. Chopin J. Ridgway M. Gerber O. Papaspiliopoulos CREST-ENSAE, Malakoff,
More informationSequential Bayesian Updating
BS2 Statistical Inference, Lectures 14 and 15, Hilary Term 2009 May 28, 2009 We consider data arriving sequentially X 1,..., X n,... and wish to update inference on an unknown parameter θ online. In a
More informationEstimating Macroeconomic Models: A Likelihood Approach
Estimating Macroeconomic Models: A Likelihood Approach Jesús Fernández-Villaverde University of Pennsylvania, NBER, and CEPR Juan Rubio-Ramírez Federal Reserve Bank of Atlanta Estimating Dynamic Macroeconomic
More informationSequential Monte Carlo Samplers for Applications in High Dimensions
Sequential Monte Carlo Samplers for Applications in High Dimensions Alexandros Beskos National University of Singapore KAUST, 26th February 2014 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Alex
More informationSequential Monte Carlo and Particle Filtering. Frank Wood Gatsby, November 2007
Sequential Monte Carlo and Particle Filtering Frank Wood Gatsby, November 2007 Importance Sampling Recall: Let s say that we want to compute some expectation (integral) E p [f] = p(x)f(x)dx and we remember
More informationComputational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
More informationMCMC for big data. Geir Storvik. BigInsight lunch - May Geir Storvik MCMC for big data BigInsight lunch - May / 17
MCMC for big data Geir Storvik BigInsight lunch - May 2 2018 Geir Storvik MCMC for big data BigInsight lunch - May 2 2018 1 / 17 Outline Why ordinary MCMC is not scalable Different approaches for making
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationAn introduction to Sequential Monte Carlo
An introduction to Sequential Monte Carlo Thang Bui Jes Frellsen Department of Engineering University of Cambridge Research and Communication Club 6 February 2014 1 Sequential Monte Carlo (SMC) methods
More informationControlled sequential Monte Carlo
Controlled sequential Monte Carlo Jeremy Heng, Department of Statistics, Harvard University Joint work with Adrian Bishop (UTS, CSIRO), George Deligiannidis & Arnaud Doucet (Oxford) Bayesian Computation
More informationSUPPLEMENT TO MARKET ENTRY COSTS, PRODUCER HETEROGENEITY, AND EXPORT DYNAMICS (Econometrica, Vol. 75, No. 3, May 2007, )
Econometrica Supplementary Material SUPPLEMENT TO MARKET ENTRY COSTS, PRODUCER HETEROGENEITY, AND EXPORT DYNAMICS (Econometrica, Vol. 75, No. 3, May 2007, 653 710) BY SANGHAMITRA DAS, MARK ROBERTS, AND
More information(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationInference in state-space models with multiple paths from conditional SMC
Inference in state-space models with multiple paths from conditional SMC Sinan Yıldırım (Sabancı) joint work with Christophe Andrieu (Bristol), Arnaud Doucet (Oxford) and Nicolas Chopin (ENSAE) September
More informationKernel Sequential Monte Carlo
Kernel Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) * equal contribution April 25, 2016 1 / 37 Section
More informationESTIMATION of a DSGE MODEL
ESTIMATION of a DSGE MODEL Paris, October 17 2005 STÉPHANE ADJEMIAN stephane.adjemian@ens.fr UNIVERSITÉ DU MAINE & CEPREMAP Slides.tex ESTIMATION of a DSGE MODEL STÉPHANE ADJEMIAN 16/10/2005 21:37 p. 1/3
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation
More informationAmbiguous Business Cycles: Online Appendix
Ambiguous Business Cycles: Online Appendix By Cosmin Ilut and Martin Schneider This paper studies a New Keynesian business cycle model with agents who are averse to ambiguity (Knightian uncertainty). Shocks
More informationLecture 4: Dynamic models
linear s Lecture 4: s Hedibert Freitas Lopes The University of Chicago Booth School of Business 5807 South Woodlawn Avenue, Chicago, IL 60637 http://faculty.chicagobooth.edu/hedibert.lopes hlopes@chicagobooth.edu
More informationCalibration of Stochastic Volatility Models using Particle Markov Chain Monte Carlo Methods
Calibration of Stochastic Volatility Models using Particle Markov Chain Monte Carlo Methods Jonas Hallgren 1 1 Department of Mathematics KTH Royal Institute of Technology Stockholm, Sweden BFS 2012 June
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationLecture 8: Bayesian Estimation of Parameters in State Space Models
in State Space Models March 30, 2016 Contents 1 Bayesian estimation of parameters in state space models 2 Computational methods for parameter estimation 3 Practical parameter estimation in state space
More informationBAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA
BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci
More informationAdvanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering
Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Axel Gandy Department of Mathematics Imperial College London http://www2.imperial.ac.uk/~agandy London
More informationParticle Filtering for Data-Driven Simulation and Optimization
Particle Filtering for Data-Driven Simulation and Optimization John R. Birge The University of Chicago Booth School of Business Includes joint work with Nicholas Polson. JRBirge INFORMS Phoenix, October
More informationHypothesis Testing. Econ 690. Purdue University. Justin L. Tobias (Purdue) Testing 1 / 33
Hypothesis Testing Econ 690 Purdue University Justin L. Tobias (Purdue) Testing 1 / 33 Outline 1 Basic Testing Framework 2 Testing with HPD intervals 3 Example 4 Savage Dickey Density Ratio 5 Bartlett
More informationMONTE CARLO METHODS. Hedibert Freitas Lopes
MONTE CARLO METHODS Hedibert Freitas Lopes The University of Chicago Booth School of Business 5807 South Woodlawn Avenue, Chicago, IL 60637 http://faculty.chicagobooth.edu/hedibert.lopes hlopes@chicagobooth.edu
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationarxiv: v1 [stat.co] 23 Apr 2018
Bayesian Updating and Uncertainty Quantification using Sequential Tempered MCMC with the Rank-One Modified Metropolis Algorithm Thomas A. Catanach and James L. Beck arxiv:1804.08738v1 [stat.co] 23 Apr
More informationThe Hierarchical Particle Filter
and Arnaud Doucet http://go.warwick.ac.uk/amjohansen/talks MCMSki V Lenzerheide 7th January 2016 Context & Outline Filtering in State-Space Models: SIR Particle Filters [GSS93] Block-Sampling Particle
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee and Andrew O. Finley 2 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationEstimation of moment-based models with latent variables
Estimation of moment-based models with latent variables work in progress Ra aella Giacomini and Giuseppe Ragusa UCL/Cemmap and UCI/Luiss UPenn, 5/4/2010 Giacomini and Ragusa (UCL/Cemmap and UCI/Luiss)Moments
More informationPackage RcppSMC. March 18, 2018
Type Package Title Rcpp Bindings for Sequential Monte Carlo Version 0.2.1 Date 2018-03-18 Package RcppSMC March 18, 2018 Author Dirk Eddelbuettel, Adam M. Johansen and Leah F. South Maintainer Dirk Eddelbuettel
More informationInfinite-State Markov-switching for Dynamic. Volatility Models : Web Appendix
Infinite-State Markov-switching for Dynamic Volatility Models : Web Appendix Arnaud Dufays 1 Centre de Recherche en Economie et Statistique March 19, 2014 1 Comparison of the two MS-GARCH approximations
More informationCSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 10 Alternatives to Monte Carlo Computation Since about 1990, Markov chain Monte Carlo has been the dominant
More informationMarkov Chain Monte Carlo
Markov Chain Monte Carlo Recall: To compute the expectation E ( h(y ) ) we use the approximation E(h(Y )) 1 n n h(y ) t=1 with Y (1),..., Y (n) h(y). Thus our aim is to sample Y (1),..., Y (n) from f(y).
More informationLECTURE 15 Markov chain Monte Carlo
LECTURE 15 Markov chain Monte Carlo There are many settings when posterior computation is a challenge in that one does not have a closed form expression for the posterior distribution. Markov chain Monte
More informationLearning the hyper-parameters. Luca Martino
Learning the hyper-parameters Luca Martino 2017 2017 1 / 28 Parameters and hyper-parameters 1. All the described methods depend on some choice of hyper-parameters... 2. For instance, do you recall λ (bandwidth
More informationASYMPTOTICALLY INDEPENDENT MARKOV SAMPLING: A NEW MARKOV CHAIN MONTE CARLO SCHEME FOR BAYESIAN INFERENCE
International Journal for Uncertainty Quantification, 3 (5): 445 474 (213) ASYMPTOTICALLY IDEPEDET MARKOV SAMPLIG: A EW MARKOV CHAI MOTE CARLO SCHEME FOR BAYESIA IFERECE James L. Beck & Konstantin M. Zuev
More informationECO 513 Fall 2008 C.Sims KALMAN FILTER. s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. u t = r t. u 0 0 t 1 + y t = [ H I ] u t.
ECO 513 Fall 2008 C.Sims KALMAN FILTER Model in the form 1. THE KALMAN FILTER Plant equation : s t = As t 1 + ε t Measurement equation : y t = Hs t + ν t. Var(ε t ) = Ω, Var(ν t ) = Ξ. ε t ν t and (ε t,
More informationA Review of Pseudo-Marginal Markov Chain Monte Carlo
A Review of Pseudo-Marginal Markov Chain Monte Carlo Discussed by: Yizhe Zhang October 21, 2016 Outline 1 Overview 2 Paper review 3 experiment 4 conclusion Motivation & overview Notation: θ denotes the
More informationParticle Filtering a brief introductory tutorial. Frank Wood Gatsby, August 2007
Particle Filtering a brief introductory tutorial Frank Wood Gatsby, August 2007 Problem: Target Tracking A ballistic projectile has been launched in our direction and may or may not land near enough to
More informationPrinciples of Bayesian Inference
Principles of Bayesian Inference Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department
More informationMonte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
More informationBayesian Model Comparison:
Bayesian Model Comparison: Modeling Petrobrás log-returns Hedibert Freitas Lopes February 2014 Log price: y t = log p t Time span: 12/29/2000-12/31/2013 (n = 3268 days) LOG PRICE 1 2 3 4 0 500 1000 1500
More informationBayesian Phylogenetics:
Bayesian Phylogenetics: an introduction Marc A. Suchard msuchard@ucla.edu UCLA Who is this man? How sure are you? The one true tree? Methods we ve learned so far try to find a single tree that best describes
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter
More informationOn the conjugacy of off-line and on-line Sequential Monte Carlo Samplers. Working Paper Research. by Arnaud Dufays. September 2014 No 263
On the conjugacy of off-line and on-line Sequential Monte Carlo Samplers Working Paper Research by Arnaud Dufays September 2014 No 263 Editorial Director Jan Smets, Member of the Board of Directors of
More informationMarkov Chain Monte Carlo Methods for Stochastic Optimization
Markov Chain Monte Carlo Methods for Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U of Toronto, MIE,
More informationCS281A/Stat241A Lecture 22
CS281A/Stat241A Lecture 22 p. 1/4 CS281A/Stat241A Lecture 22 Monte Carlo Methods Peter Bartlett CS281A/Stat241A Lecture 22 p. 2/4 Key ideas of this lecture Sampling in Bayesian methods: Predictive distribution
More informationParticle Filters. Outline
Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationTutorial on ABC Algorithms
Tutorial on ABC Algorithms Dr Chris Drovandi Queensland University of Technology, Australia c.drovandi@qut.edu.au July 3, 2014 Notation Model parameter θ with prior π(θ) Likelihood is f(ý θ) with observed
More informationMonte Carlo Approximation of Monte Carlo Filters
Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include: Arnaud Doucet, Axel Finke, Anthony Lee, Nick Whiteley 7th January 2014 Context & Outline Filtering in State-Space
More informationParticle Filtering Approaches for Dynamic Stochastic Optimization
Particle Filtering Approaches for Dynamic Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge I-Sim Workshop,
More informationComputer Intensive Methods in Mathematical Statistics
Computer Intensive Methods in Mathematical Statistics Department of mathematics johawes@kth.se Lecture 16 Advanced topics in computational statistics 18 May 2017 Computer Intensive Methods (1) Plan of
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationNotes on pseudo-marginal methods, variational Bayes and ABC
Notes on pseudo-marginal methods, variational Bayes and ABC Christian Andersson Naesseth October 3, 2016 The Pseudo-Marginal Framework Assume we are interested in sampling from the posterior distribution
More informationBayesian Estimation of DSGE Models
Bayesian Estimation of DSGE Models Stéphane Adjemian Université du Maine, GAINS & CEPREMAP stephane.adjemian@univ-lemans.fr http://www.dynare.org/stepan June 28, 2011 June 28, 2011 Université du Maine,
More informationSession 5B: A worked example EGARCH model
Session 5B: A worked example EGARCH model John Geweke Bayesian Econometrics and its Applications August 7, worked example EGARCH model August 7, / 6 EGARCH Exponential generalized autoregressive conditional
More informationMarkov Chain Monte Carlo Methods for Stochastic
Markov Chain Monte Carlo Methods for Stochastic Optimization i John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U Florida, Nov 2013
More information17 : Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models, Spring 2015 17 : Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Heran Lin, Bin Deng, Yun Huang 1 Review of Monte Carlo Methods 1.1 Overview Monte Carlo
More informationEco517 Fall 2013 C. Sims MCMC. October 8, 2013
Eco517 Fall 2013 C. Sims MCMC October 8, 2013 c 2013 by Christopher A. Sims. This document may be reproduced for educational and research purposes, so long as the copies contain this notice and are retained
More informationDSGE Methods. Estimation of DSGE models: Maximum Likelihood & Bayesian. Willi Mutschler, M.Sc.
DSGE Methods Estimation of DSGE models: Maximum Likelihood & Bayesian Willi Mutschler, M.Sc. Institute of Econometrics and Economic Statistics University of Münster willi.mutschler@uni-muenster.de Summer
More informationBayes Factors, posterior predictives, short intro to RJMCMC. Thermodynamic Integration
Bayes Factors, posterior predictives, short intro to RJMCMC Thermodynamic Integration Dave Campbell 2016 Bayesian Statistical Inference P(θ Y ) P(Y θ)π(θ) Once you have posterior samples you can compute
More information