GMM with Latent Variables

Transcription

1 GMM with Latent Variables A. Rnald Gallant Penn State University Raffaella Giacmini University Cllege Lndn Giuseppe Ragusa Luiss University

2 Cntributin The cntributin f GMM (Hansen and Singletn, 1982) was t allw frequentist inference regarding the parameters f a nnlinear structural mdel withut having t slve the mdel. Prvided there were n latent variables. The cntributin f this paper is the same. With latent variables.

3 The Requirements, 1 f 3 A structural mdel with parameters θ and true value θ Observed variables: X = (X 1, X 2,..., X T ) Latent variables: Λ = (Λ 1,Λ 2,...,Λ T ) Knwn transitin density: Λ t+1 P(Λ t+1 Λ t, θ) Cnditinal mment cnditins: E [ g(x t+1,λ t+1, θ) I t ] = 0 That wuld identify θ if bth X and Λ were bserved. I t = {X,..., X t, Λ,...,Λ t }

4 The Requirements, 2 f 3 Sample mment cnditins g T (X,Λ, θ) = 1 T T t=1 g(x t,λ t, θ) Weighting matrix (May have t use a HAC weighting matrix instead.) Such that Σ(X,Λ, θ) = 1 T T t=1 g(x t,λ t, θ) g(x t,λ t, θ) g(x t,λ t, θ) = g(x t,λ t, θ) 1 T g T (X,Λ, θ) Z = [Σ(X,Λ, θ )] 1/2 g T (X,Λ, θ ) d N(0, I) Hansen and Singletn (1982) Gallant and White (1987)

5 The Requirements, 3 f 3 A sample { θ (i)} R i=1 frm the density { } p(θ) = (2π) M/2 exp 1 2 g T(X,Λ, θ) [Σ(X,Λ, θ)] 1 g T (X,Λ, θ) is a sample frm the asympttic distributin f the GMM estimatr fr large T. Chernzhukv and Hng (2003)

6 Estimatin Strategy Sample { θ (i), Λ (i)} frm the density { } p(θ,λ) = (2π) M/2 exp 1 2 g T(X,Λ, θ) [Σ(X,Λ, θ)] 1 g T (X,Λ, θ) Might multiply p(θ,λ) by a Jacbian term [detσ(x,λ, θ)] M/2 Metrplis within Gibbs algrithm Sample θ (i) given Λ (i 1) and θ (i 1) using Metrplis last draw f MCMC chain f length K. Sample Λ (i) given θ (i) and Λ (i 1) using Gibbs. last particle f a mdified particle filter f size N. Iterate back and frth. (Can view it as an apprximate EM algrithm.) Estimate and scale are mean and standard deviatin f { θ (i)}.

7 Next: Tw Examples A Dynamic Stchastic General Equilibrium Mdel Descriptin Estimates A Stchastic Vlatility Mdel Descriptin Estimates

8 A DSGE Mdel 1 f 4 Frm Del Negr and Schrfheide (2008) simplified t permit an analytic slutin by remving rigidities, investment, etc. Three shcks: z t = ρ z z t 1 + σ z ǫ z,t φ t = ρ φ φ t 1 + σ φ ǫ φ,t λ t = ρ λ λ t 1 + σ λ ǫ λ,t Factr prductivity Cnsumptin/leisure preference Price elasticity f intermediate gds Three utputs: w t y t π t Wages Output Inflatin

9 A DSGE Mdel 2 f 4 First rder cnditins 0 = y t + 1 β π t E t (y t+1 + π t+1 + z t+1 ) 0 = w t + λ t 0 = w t (1 + ν)y t φ t where ν is a labr supply elasticity and β is the discunt rate. The true values f the parameters are θ = (ρ z, ρ φ, ρ λ, σ z, σ φ, σ λ, ν, β) = (0.15, 0.68, 0.56, 0.71, 2.93, 0.11, 0.96, 0.996) We take w t, y t, and π t as measured and z t and φ t as latent s X t = (w t, y t, π t ) Λ t = (z t, φ t ).

10 A DSGE Mdel 3 f 4 A set f cnditins that identify the mdel are g 1 = (w t ρ λ w t 1 ) 2 σ 2 λ g 2 = w t 1 (w t ρ λ w t 1 ) g 3 = [w t 1 (1 + ν)y t 1 ][w t (1 + ν)y t ρ φ (w t 1 (1 + ν)y t 1 )] g 4 = [w t 1 (1 + ν)y t 1 ](φ t ρ φ φ t 1 ) g 5 = [w t (1 + ν)y t ] 2 σ 2 φ g 6 = w t 1 (y t β π t 1 y t π t ρ z z t 1 ) g 7 = y t 1 (y t β π t 1 y t π t ρ z z t 1 ) g 8 = π t 1 (y t β π t 1 y t π t ρ z z t 1 ) g 9 = (y t β π t 1 y t π t ) 2 ρ2 z σ2 z 1 ρ 2 z

11 A DSGE Mdel 4 f 4 An analytic expressin fr the likelihd L(θ) = p(x θ) is available fr this mdel Analysis f the likelihd shws that nly ne f the fur parameters σ z, σ φ, ν, β can be identified Three will have t be calibrated in rder t apply frequentist methds We calibrate σ z, σ φ, ν and leave β as the free parameter.

12 Table 1. Parameter Estimates, DSGE Mdel Parameter True Value Mean Mde Standard Errr With Jacbian ρ z ρ φ ρ λ σ λ β Withut Jacbian ρ z ρ φ ρ λ σ λ β Maximum Likelihd ρ z ρ φ ρ λ σ λ β Data with T = 250 simulated at true values. Gibbs particles are N = 1000; Metrplis draws are K = 50. GMM mean, mde, and standard deviatin are frm MCMC chains f length R = 9637 with stride f 1; fr MLE chain R = , stride is 5.

13 Figure 1. PF Estimate f Λ with Jacbian, DSGE Mdel Remark: The Gibbs draw shuld evaluate the mments in the Metrplis step accurately; nt necessarily apprximate the histry accurately.

14 Figure 2. PF Estimate f Λ withut Jacbian, DSGE Mdel

15 Figure 3. PF Estimate f Λ with Jacbian, DSGE Mdel

16 Figure 4. PF Estimate f Λ withut Jacbian, DSGE Mdel

17 The Chice f Mments Des Matter 1 f 2 It is pssible t perfrm cunter-factual (e.g. impulse-respnse) analysis using mment cnditins alne. Hwever, fr it t wrk, ne must d a much better jb f estimating the histry f the latent variables. T estimate latent variables, it is nt necessary t identify mdel parameters. Only the latent variables need t be identified.

18 The Chice f Mments Des Matter 2 f 2 Mment cnditins fr cunter-factual analysis h 1 = y t β π t 1 y t π t ρ z z t 1 h 2 = w t 1 h 1 h 3 = y t 1 h 1 h 4 = π t 1 h 1 h 5 = w t (1 + ν)y t φ t h 6 = w t 1 h 5 h 7 = y t 1 h 5 h 8 = π t 1 h 5

19 Figure 5. PF Estimate f Λ with Jacbian, DSGE Mdel

20 Figure 6. PF Estimate f Λ withut Jacbian, DSGE Mdel

21 Figure 7. PF Estimate f Λ with Jacbian, DSGE Mdel

22 Figure 8. PF Estimate f Λ withut Jacbian, DSGE Mdel

23 Gibbs and Metrplis Mments Can Differ If we use the mments that identify the mdel used fr Table 1 fr the Metrplis step and the mments designed fr a cunterfactual analysis used fr Figures 5 thrugh 8 fr the Gibbs step, we get slightly better results in the fllwing Table 2.

24 Table 2. Alternative Parameter Estimates, DSGE Mdel Parameter True Value Mean Mde Standard Errr With Jacbian ρ z ρ φ ρ λ σ λ β Withut Jacbian ρ z ρ φ ρ λ σ λ β Maximum Likelihd ρ z ρ φ ρ λ σ λ β Data with T = 250 simulated at true values. Gibbs particles are N = 1000; Metrplis draws are K = 50. GMM mean, mde, and standard deviatin are frm MCMC chains f length R = 9637 with stride f 1; fr MLE chain R = , stride is 5.

25 A Stchastic Vlatility Mdel 1 f 2 X t = ρx t 1 + exp(λ t ) u t (1) Λ t = φλ t 1 + σe t (2) e t N(0,1) (3) u t N(0,1) (4) The true values f the parameters are θ 0 = (ρ 0, φ 0, σ 0 ) = (0.9,0.9,0.5) θ 0 = (ρ 0, φ 0, σ 0 ) = (0.25,0.8,0.1) (plts) (estimatin)

26 A Stchastic Vlatility Mdel 2 f 2 Mment Cnditins h 1 = (X t ρx t ) 2 [exp(λ t )] 2 h 2 = X t ρx t X t 1 ρx t 1. h L+1 = X t ρx t X t L ρx t L h L+2 = X t 1 (X t ρx t 1 ) h L+3 = Λ t 1 (Λ t φλ t 1 ) h L+4 = (Λ t φλ t 1 ) 2 σ 2 ( 2 π ( 2 π ) 2 exp(λ t )exp(λ t 1 ) ) 2 exp(λ t )exp(λ t L )

27 Table 3. Parameter Estimates, SV Mdel Parameter True Value Mean Mde Standard Errr With Jacbian Term ρ φ σ Withut Jacbian ρ φ σ Flury and Shephard Estimatr ρ φ σ Data f length T = 200 was generated frm the SV mdel at true values. In all panels the number f particles is N = The clumns labeled mean, mde, and standard deviatin are the mean, mde, and standard deviatins f an MCMC chain f length

28 Figure 9. PF Estimate f Λ with Jacbian Term, SV Mdel

29 Figure 10. PF Estimate f Λ withut Jacbian, SV Mdel

30 Figure 11. Flury-Shephard Estimate f Λ, SV Mdel

31 Figure 12. PF Estimate f Λ with Jacbian Term, SV Mdel

32 Figure 13. PF Estimate f Λ withut Jacbian, SV Mdel

33 Figure 14. Flury-Shephard Estimate f Λ, SV Mdel

34 Next: The Three Algrithms A particle filter algrithm Input: θ Output: Draws { Λ (i)} R i=1 frm P(Λ X, θ) Gibbs algrithm Input: Draws θ (i 1) and Λ (i 1) Output: A draw Λ (i) frm P(Λ X, θ) Metrplis algrithm Input: Draws θ (i 1) and Λ (i) Output: A draw θ (i) frm P(θ X,Λ)

35 Ntatin X 1:t = (X 1,..., X t ) Λ 1:t = (Λ 1,...,Λ t ) p(x 1:t,Λ 1:t, θ) { } = (2π) M/2 exp 1 2 g t(x 1:t,Λ 1:t, θ) [Σ(X 1:t,Λ 1:t, θ)] 1 g t (X 1:t,Λ 1:t, θ)

36 Particle Filter Algrithm, 1 f 3 1. Initializatin. Input θ (and X ) Set T 0 t the minimum sample size required t cmpute g t (X 1:t,Λ 1:t, θ). Fr i = 1,..., N sample (Λ (i) 1,Λ(i) 2,...,Λ(i) T 0 ) frm p(λ t Λ t 1, θ). Set t t T Set Λ (i) 1:t 1 = (Λ(i) 1,Λ(i) 2,...,Λ(i) T 0 )

37 Particle Filter Algrithm, 2 f 3 2. Imprtance sampling step. Fr i = 1,..., N sample Λ (i) t frm p(λ t Λ (i) t 1 ) and set Λ (i) 1:t = (Λ(i) 0:t 1, Λ (i) t ). Fr i = 1,..., N cmpute weights w (i) t = p(x 1:t, Λ (i) 1:t, θ). Scale the weights t sum t ne.

38 Particle Filter Algrithm, 3 f 3 3. Selectin step. Fr i = 1,..., N sample with replacement particles Λ (i) 1:t frm the set { Λ (i) 1:t } accrding t the weights. 4. Repeat If t < T, increment t and g t Imprtance Sampling Step; else utput { Λ (i) } N 1:T. i=1

39 Gibbs Algrithm, 1 f 3 1. Initializatin. Input Λ (1) 1:T, θ (and X ) Set T 0 t the minimum sample size required t cmpute g t (X 1:t,Λ 1:t, θ). Fr i = 2,..., N sample (Λ (i) 1,Λ(i) 2,...,Λ(i) T 0 ) frm p(λ t Λ t 1, θ). Set t t T Set Λ (i) 1:t 1 = (Λ(i) 1,Λ(i) 2,...,Λ(i) T 0 )

40 Gibbs Algrithm, 2 f 3 2. Imprtance sampling step. Fr i = 2,..., N sample Λ (i) t frm p(λ t Λ (i) t 1 ) and set Λ (i) 1:t = (Λ(i) 0:t 1, Λ (i) t ). Fr i = 1,..., N cmpute weights w (i) t = p(x 1:t, Λ (i) 1:t, θ). Scale the weights t sum t ne.

41 Gibbs Algrithm, 3 f 3 3. Selectin step. Fr i = 2,..., N sample with replacement particles Λ (i) 1:t frm the set { Λ (i) 1:t }N i=1 accrding t the weights. 4. Repeat If t < T, increment t and g t Imprtance Sampling Step; else utput the particle Λ (N) 1:T.

42 Metrplis Algrithm Prpsal density: T(θ here, θ there ) (e.g., mve ne-at-time randm walk) Input: Λ, θ ld (and X ) Prpse: Draw θ prp frm T(θ ld, θ) Accept-Reject: Put θ (i) t θ prp with prbability α = min else put θ (i) t θ ld. [ 1, p(x,λ, θ prp)t(θ prp, θ ld ) p(x,λ, θ ld )T(θ ld, θ prp ) Repeat: If i < K put θ ld = θ (i) and g t Prpse; else utput θ (K). ]

43 Next: Why Des this Wrk? Prve that the particle filter wrks using the ntin f Gallant, A. Rnald, and Han Hng (2007), A Statistical Inquiry int the Plausibility f Recursive Utility, Jurnal f Financial Ecnmetrics, that GMM induces a prbability space; next several slides That the Metrplis algrithm wrks fllws frm Chernzhukv, Victr, and Han Hng (2003), An MCMC Apprach t Classical Estimatin, Jurnal f Ecnmetrics. That the Gibbs algrithm wrks fllws frm Andrieu, C., A. Duced, and R. Hlenstein (2010), Particle Markv Chain Mnte Carl Methds, Jurnal f the Ryal Statistical Sciety, Series B.

44 Jint Density Induced by GMM, Dice Example Table 4. Tssing tw crrelated dice (X,Λ) when the prbability f the difference D = X Λ is the primitive. Preimage d P(D = d) P(D = d Λ = 1) P(D = d Λ = 2) C 5 = {(1,6)} C 4 = {(1,5),(2,6)} C 3 = {(1,4),(2,5),(3,6)} C 2 = {(1,3),(2,4),(3,5),(4,6)} C 1 = {(1,2),(2,3),(3,4),(4,5),(5,6)} -1 4/18 0 4/18 C 0 = {(1,1),(2,2),(3,3),(4,4),(5,5),(6,6)} 0 10/18 10/14 10/18 C 1 = {(2,1),(3,2),(4,3),(5,4),(6,5)} 1 4/18 4/14 4/18 C 2 = {(3,1),(4,2),(5,3),(6,4))} C 3 = {(4,1),(5,2),(6,3)} C 4 = {(5,1),(6,2)} C 5 = {(6,1)} Cnditinal prbability is P(D = d Λ = λ) = P(C d O λ )/P(O λ ), where O λ is the unin f the events that can ccur. Q(Λ = λ) = P(O λ ) is the marginal in the sense that 6 P(D = d) = P(D = d Λ = λ)q(λ = λ) λ=1

45 Cnditinal Density, Dice Example, 1 f 2 Let C be the smallest σ-algebra that cntains the preimages in Table 4. Any C-measurable f must be cnstant n the preimages. Fr such f the frmula E(f Λ = 2) = 6 f(x,2) x=1 5 d= 5 I Cd (x,2)p(d = d Λ = 2) (5) can be used t cmpute cnditinal expectatin because f can be regarded as a functin f d and the right hand side f (5) equals 5 d= 5 f(d)p(d = d Λ = 2).

46 Cnditinal Density, Dice Example, 2 f 2 Equatin (5) implies that we can view P(D = d) as defining a cnditinal density functin P(X = x Λ = λ) = 5 d= 5 I Cd (x, λ)p(d = d Λ = λ) (6) that is a functin f x as lng as we nly use it in cnnectin with C-measurable f. T get an expressin that agrees with the expressins in Gallant and Hng (2007) nte that we can write equatin (6) as P(X = x Λ = λ) = P(D = x λ) 6 x=1 P(D = x λ) (7) Similarly, P(Λ = λ X = x) = P(D = x λ) 6 λ=1 P(D = x λ) (8)

47 Abstractin A GMM criterin Z(X,Λ, θ) defines a prbability space (R dim(x) R dim(λ), C, P θ ) C is the smallest σ-algebra cntaining the preimages f Z On which there are ntins f jint { p(x,λ, θ) = (2π) M/2 exp 1 2 g T(X,Λ, θ) [Σ(X,Λ, θ)] 1 g T (X,Λ, θ), cnditinal p(x Λ, θ), and marginal densities. } If P θ dentes the data generating prcess, then (R dim(x) R dim(λ), C, P θ ) = (R dim(x) R dim(λ), C, P θ )

48 What if One Knws a Marginal?, Dice Example Then ne knws the prbabilities P(R λ ) f the rectangles R λ = D {λ} D = {1,2,3,4,5,6} Let C be the smallest σ-algebra cntaining {C d } 5 d= 5 and {R λ } 6 λ=1 The singletn sets {(x, λ)} are in C s jint prbability P n C and cnditinal densities have their cnventinal definitin P (X = x Λ = λ) = P ({(x,λ)}) P (R λ ) P (Λ = λ X = x) = P ({(x,λ)}) P (R x )

49 Indeterminacy, Dice Example, 1 f 2 Fr P ({(x, λ)}) we have nine equatins in sixteen unknwns: 4 18 = = 4 18 = 5 i=1 6 i=1 5 i=1 P ({(i, i + 1)}) P ({(i, i)}) P ({(i + 1, i)}) 1 6 = P ({(1,1)}) + P ({(2,1)}) 1 6 = P ({(1,2)}) + P ({(2,2)}) + P ({(3,2)}) 1 6 = P ({(2,3)}) + P ({(3,3)}) + P ({(4,3)}) 1 6 = P ({(3,4)}) + P ({(4,4)}) + P ({(5,4)}) 1 6 = P ({(4,5)}) + P ({(5,5)}) + P ({(6,5)}) 1 6 = P ({(5,6)}) + P ({(6,6)}) There is ne linear dependency leaving eight equatins in sixteen unknwns.

50 Indeterminacy, Dice Example, 2 f 2 The fact that fr P ({(x, λ)}) we have nly eight equatins in sixteen unknwns is fatal. We have n lgical basis fr chsing a particular slutin. The particle filter depends n the chice f slutin.

51 A Secnd Example, Mimics Fisher (1930), 1 f 2 P[Z(X,Λ) = z] = 1 p 1 + p p z Z(X,Λ) = X Λ X N Λ N N = {0, ±1, ±2,... } The preimages f Z(x, λ) are C z = {(x, λ) : x = z + λ, λ N} z N which lie n 45 degree lines in the (x, λ) plane. Given λ, fr every z N there is an x N with (x, λ) C z s every C z can ccur. Therefre O λ = z N C z and P(O λ ) = 1 fr every λ N.

52 A Secnd Example, Mimics Fisher (1930), 2 f 2 If P(O λ ) = 1 fr every λ N. Then P(Z = z Λ = λ) = P(C z O λ ) P(O λ ) which des nt depend n λ. = P(C z ) = 1 p 1 + p p z, Cnsequently, P(X = x Λ = λ) = P(Z = x λ) Prvides a ratinale fr chsing a slutin: The cnditinal rbability f X given Λ shuld be the same under P θ and P θ. P (X = x,λ = λ) = P(Z = x λ) P (R λ ) P (X = x Λ = λ) = P(Z = x λ).

53 Abstractin, 1 f 3 A GMM criterin Z(X,Λ, θ) And knwledge f P (R B ) R B = R dim(x) B B R dim(λ) is Brel Defines a prbability space (R dim(x) R dim(λ), C, P θ ) C is the smallest σ-algebra cntaining the preimages f Z and rectangles R B, On which there are ntins f jint p (X,Λ, θ), cnditinal p (X Λ, θ), and marginal densities p (Λ).

54 Abstractin, 2 f 3 (R dim(x) R dim(λ), C, P θ ) = (R dim(x) R dim(λ), C, P θ ) (R dim(x) R dim(λ), C, P θ ) = (Rdim(X) R dim(λ), C, P θ ) (R dim(x) R dim(λ), C, P θ ) = (Rdim(X) R dim(λ), C, P θ )

55 Abstractin, 3 f 3 If we assume that the unin O λ f all sets in C that can ccur if Λ = λ is knwn t have ccurred has prbability ne, then p (X,Λ, θ) = p(x,λ, θ)p (Λ, θ) p (X Λ, θ) = p(x,λ, θ) And we can recver f(x, λ) p(λ X, θ) dλ via a particle filter as lng as we restrict attentin t C- measurable f.

56 Interpretatin If we assume cmpact Θ, then Chernzukv-Hng are Bayes n (R dim(x) R dim(λ), C, P θ ) = (R dim(x) R dim(λ), C, P θ ) And we are Bayes n (R dim(x) R dim(λ), C, P θ ) = (Rdim(X) R dim(λ), C, P θ )

57 Cntributin The cntributin f GMM (Hansen and Singletn, 1982) was t allw frequentist inference regarding the parameters f a nnlinear structural mdel withut having t slve the mdel. Prvided there were n latent variables. The cntributin f this paper is the same. With latent variables.