GMM Estimation and Testing II

Transcription

1 GMM Estimation and Testing II Whitney Newey October 2007

2 Hansen, Heaton, and Yaron (1996): In a Monte Carlo example of consumption CAPM, two-step optimal GMM with with many overidentifying restrictions is biased. Continuously updated GMM estimator (CUE) is much less biased. CUE: Let Ωˆ(β) = P i g i (β)g i (β) 0 /n. LIML analog. βˆcue =argminĝ(β) 0 Ωˆ(β) 1 ĝ(β). β Altonji and Segal (1996): In Monte Carlo examples of minimum distance estimation of variance matrix parameters, two-step optimal GMM with with many overidentifying restrictions is biased. GMM with an identity weighting matrix is much less is biased. Give some theory that explains these results.

3 Higher-order Bias and Variance Stochastic Expansion: Estimators that come from smooth (four times continuously differentiable) moment conditions have an expansion of the form n 1/2 (βˆ β 0 )= ψ + Q 1 ( a)/n 1/2 + Q 2 ( a, ψ, ψ, b)/n + O p (1/n 3/2 ), where Q 1 and Q 2 arelinearineachargument, aresmooth, ψ(z), a(z), b(z), are mean zero random vectors, and ψ = P i ψ(z i )/n 1/2, ã = P i a(z i )/n 1/2, P b = i b(z i )/n 1/2. An approximate bias is given by Bias(βˆ) = E[Q 1 ( ψ, a)]/n = E[Q 1 (ψ(z i ),a(z i ))]/n.

4 Simple example: βˆ = r( z),r(z) smooth, z = nx i=1 z i /n. GMM estimator with g(z, β) =: Expand around μ = E[z i ];with β 0 = r(μ) : n(βˆ β 0 ) = r 0 (μ) n( z μ)+r 00 (μ)n( z μ) 2 /2 n +r 000 (μ)n 3/2 ( z μ) 3 /6n +[r 000 ( z) r 000 (μ)]n 3/2 ( z μ) 3 /n where z is between z and μ. Here ψ(z) = a(z) = b(z) = z μ, Q 1 (ψ, a) = r 00 (μ)ψa/2. Bias is Bias(βˆ) =a 00 (μ)var(z i )/2n.

5 Approximation works quite well in describing how bias depends on number of moment conditions. Breaks down if identification is very, very weak. This is where bias formula for 2SLS comes from. Can also get a variance approximation, though does not work as well. Can interpret as the bias of an approximating distribution, i.e. there is a precise sense in which dropping the higher order terms is OK. Can use this approximation to select moments to minimize higher-order mean square error; Donald and Newey (2001). Can also use it to compare different GMM esitmators, like CUE.

6 Bias of GMM: Recall GMM notation. Let g i = g i (β 0 ),G i = g i (β 0 )/ β, Ω = E[g i g 0 i ],G = E[G i ], Σ = (G 0 Ω 1 G) 1,H = ΣG 0 Ω 1,P = Ω 1 Ω 1 GΣG 0 Ω 1 a = (a 1,...,a p ),a j tr(σe[ 2 g ij (β 0 )/ β β 0 ])/2 Bias for GMM has three parts: Bias(βˆGMM ) = B G + B Ω + B I, B G = ΣE[G 0 i Pg i ]/n, B Ω = HE[g i g 0 i Pg i ]/n, B I = H( a + E[G i Hg i ])/n.

7 Bias(βˆGMM ) = B G + B Ω + B I, B G = ΣE[G i 0 Pg i ]/n, B Ω = HE[g i g i 0 Pg i ]/n, B I = H( a + E[G i Hg Interpretation: B G is bias from estimating G, B Ω is bias from estimating Ω, and B I is bias for GMM estimator with moment functions G 0 Ω 1 g i (β). B G : Comes from correlation of G i and g i ; endogeneity is the source; Example: g i (β 0 ) = Z i (y i X i 0 β),g i = Z i X i 0,g i = Z i ε i ; Ω = σ ε 2 E[Z i Z i 0 ], ³ E[G i 0 Pg i ] = E[X i ε]e[z i 0 PZ i ]=E[X i ε]tr PE[Z i Z i 0 ] = E[X i ε]σ ε 2 (m p), under homoskedasticity. E[G i 0 Pg i ] nonzero due to correlation of X i and ε i.grows at same rate as m. Consistent with large biases in Hansen, Heaton, and Yaron (1996). B Ω : Zero if third moments zero, e.g. E[ε i 3 Z i ] = 0 in IV setting. Nonzero otherwise, generally grows with m (certainly magnitude of g i 0 Pg i does). Consistent

8 with biases in Altonji and Segal (1996) (where G i is constant so B G =0), where g i includes is covariance moment condittions Continuous updating GMM (CUE): Bias of CUE: βˆcue =arg minĝ(β) 0 Ωˆ(β) 1 ĝ(β). β Bias(βˆCUE )= B Ω + B I. Eliminates bias due to endogeneity and estimation of Jacobian in optimal linear combination of moments. Explains Hansen, Heaton, and Yaron (1996). CUE is a Generalized Empirical Likelihood (GEL) estimators. All GEL estimators eliminate B G ; some also eliminate B Ω. GEL: For concave function ρ(v) with domain an open interval containing zero. βˆgel =arg minsup nx β λ i=1 ρ(λ 0 g i (β)). Computation: Concentrate out λ, using analytical derivatives for β.

9 Special cases of GEL: CUE: Any quadratic ρ(v) (i.e. ρ(v) =A + Bv + Cv 2 ), Empirical Likelihood (EL): For ρ(v) =ln(1 v), X n X n X n βˆ =arg max ln π i, π i =1, π i g i (β) =0. β,π 1,...,π n i=1 i=1 i=1 Exponential Tilting (ET): For ρ(v) = exp(v), X n X n X n βˆ =arg min π i ln π i, π i =1, π i g i (β) =0. β,π 1,...,π n i=1 i=1 i=1 Bias for GEL: For ρ j (v) = j ρ(v)/ v j,ρ j = ρ j (0), ρ 0 =0, ρ 1 = 1,ρ 2 = 1, Bias(βˆGEL )=B I +(1+ ρ 3 )BΩ. 2 For EL, ρ 3 = 3 ln(1 v)/ v 3 = 2, so Bias(βˆEL )=B I. Also, ELishigher-orderefficient. Higher-order variance smaller than direct bias correction.

10 Probabilities: For ĝ i = g i (βˆ) and vˆi = λˆ0ĝ i, GEL probabilities for each observation are given by P n πˆi = ρ 1 (ˆv i )/ nx j=1 ρ 1 (ˆv j ), (i = 1,...,n). i=1 πˆia(z i, βˆ) efficiently estimates E[a(z, β 0 )], subject to moment conditions. Can explain bias results by interpreting first-order conditions. Let Ĝ i = g i (βˆ)/ β and kˆi =[ρ 1 (ˆv i )+1]/vˆi (kˆi = 1 for vˆi =0). First-order condition for β: Differentiate i ρ(λ 0 g i (β)) with respect to β X 0= ρ 1 (ˆv i )Ĝ 0 i λˆ. i First-order condition for λ : X X X 0= ρ 1 (ˆv i )ĝ i = [ρ 1 (ˆv i )+1]ĝ i nĝ(βˆ) = kˆiĝ i ĝ 0 i λˆ nĝ(βˆ) i i i Solve for λˆ = n ³ P i kˆiĝ i ĝ i 0 1 ĝ(βˆ). Plug into previous equation and divide, P 1 X X 0= πˆiĝ i kˆiĝ i ĝ 0 i ĝ(βˆ) i i

11 First-order conditions for GEL: 1 X X 0= πˆiĝ i kˆiĝ i ĝ 0 i ĝ(βˆ) i i CUE, kˆi =(ˆv i 1+1)/vˆi =1, so first order conditions are 1 X X 0= πˆiĝ i ĝ i ĝ 0 i /n ĝ(βˆ). i i EL, kˆi =( (1 vˆi) 1 +1)/vˆi =(1 vˆi 1)/vˆi(1 vˆi) = 1/(1 vˆi), so 1 X X 0= πˆiĝ i πˆiĝ i ĝ 0 i ĝ(βˆ). i i All use efficient weighting in Jacobian part of first-order conditions, EL uses efficient weighting in second moment matrix, CUE uses standard weighting. Efficient weighting removes correlation of matrix with moments, i.e. removes bias. All GEL have efficient Jacobian. EL has efficient second moment matrix. In Altonji- Segal Monte Carlo design EL bias not as small as one would have hoped.

12 Higher-Order Efficiency ˆ \ Let B = E[Q 1 (ψ(z i ),a(z i ))]. Bias corrected estimator is Higher order variance of θ is θ = ˆθ ˆB/n. V n = Σ + Ξ/n, Σ = E[ψ(z i )ψ(z i ) 0 ], Ξ = lim n {Var( Q Q 1 + ψ 0 ]+ E[ Q 1 + Q 2) 0 1 )+ E[( n Q 2 ) ψ( n ]} +Acov( n(bˆ B), ψ 0 )+ Acov( ψ, n( Bˆ 0 B 0 )) Famous result, conjectured by Fisher, show later by Rao, Pfanzagl and Wefelmeyer, is that bias corrected MLE is higher order efficient. Extends to GMM. Bias corrected EL is higher-order efficient. With discrete data, bias corrected EL becomes MLE in large samples. Discrete data can approximate any data.

13 Choosing Among Instruments (Donald and Newey (2001)). Linear simultaneous equation, one endogenous explanatory variable: y i = X i 0 β i + ε i,x i =(Y i,z i 0 1 ) 0, Y i = Ȳ i + v i, Ȳ i reduced form for variable Y i. Let j index instrument set, Z(j) a n K j matrix of instrumental variable observations for. Z(j) 0 p ε/n 0 for each j, so each instrument set is valid. Let P j = Z(j)[Z(j) 0 Z(j)] 1 Z(j) 0 be projection matrix on j th instruments. Let X = [X 1,...,X n ] 0,Y =(Y 1,...,Y n ) 0,y =(y 1,...,y n ) 0 j th instrument set the 2SLS and LIML esitmators are respectively βˆj =(X 0 P j X) 1 X 0 P j y, β j =argmin (y Xβ)0 P j (y Xβ). β (y Xβ) 0 (y Xβ)

14 Choose among instrument sets 1,...,J by minimizing a large K j estimate of mean square error (MSE). Let Z be some fixed, large set of instruments that does not vary with j, β the IV estimator (2SLS or LIML), ε = y X β, v = (I Z (Z 0 Z ) 1 Z 0 )Y, σ 2 ε 0 ε/n, σˆ2 v 0 v/n, ˆ 0 v/n, ˆ σ 2 v σˆσ ˆε = = v σ εv = ε γ = ˆ Let vˆj =(I P j )Y, Rˆj =ˆv j 0 vˆj/n + ˆσ 2 vk j /n. 2 εv 2. ˆε Ŝ 2SLS (j) =ˆσ 2 K 2 j εv n +ˆσ 2 ε Rˆj, Ŝ LIML =ˆγσˆε 2 K j +ˆσε 2 Rˆj. n Choose 2SLS estimator βˆj that minimizes Ŝ 2SLS (j). Choose LIML estimator β j that minimizes Ŝ LIML (j). Note: LIMLhas smallermse forlarge K j.

15 Monte Carlo Model is joint normal (Gaussian) with and reduced form coefficients π k = c(k ) y i = β 0 Y i + ε i,y i = Z i 0 π + v i, µ ³ k 4 1 for k = 1,..,K. K +1 K is maximal number of instruments, c(k ) is chosen so π 0 π = Rf 2 /(1 R f 2 ), where R2 f is the reduced form r-squared. Each Z(j) is first j columns of the matrix of all instrumental variables, so here K j = j. Sample sizes are n = 100, where K =20 and 5000 replications, and n = 1000, where K =30 and 1000 replications. Estimators were 2SLS-all and LIML-all which are 2SLS and LIML (respectively) using all K instruments. Estimators using the data driven number of instruments are denoted 2SLS-op and LIML-op. Estimates of σ εv and σ 2 ε were obtained using estimates of the two equations using the number of instruments that were optimal for estimating the first stage based on cross-validation.

16 N =100 N =1000 Med. Med. Dec. Cov. Med. Med. Dec. Cov. σ u Estimator Bias AD Rge Rate Bias AD Rge Rate 0.5 OLS SLS-all SLS-op LIML-all LIML-op

17 Angrist and Krueger (1991) emipirical application. Instrument Set K Cross-Val. Ŝ 2SLS Ŝ LIML Q Q+Q*Y Q+Q*S Q+Q*Y+Q*S

18 Instrument Set K 2SLS LIML Q Q+Q*Y Q+Q*S Q+Q*Y+Q*S Optimal ˆK Fuller delivers similar answers.

19 Many Instrument Asymptotics Estimates from Angrist and Krueger (1991) with optimal instrument set are: Optimal 2SLS ( ˆK = 63) LIML ( ˆK = 230) (.0195) (.0144) 2SLS and LIML estimates similar but LIML (with 180 overidentifying restrictions) seems to have smaller standard errors. What about problem that asymptotic approximation poor and variance larger with many instruments? This problem is accounted for in the standard error reported here. Based on an asymptotic approximation where the number of instruments grows at thesamerateasthesamplesize. Leads to an approximation to the distribution of t-ratios with error rate not depending on the number of instruments.

20 n 1/2 (βˆliml β 0 )= ψ + Q 1 ( a)/n 1/2 ψ, + o p (1), Q 1 (ψ, ã)/n 1/2 is asymptotically normal and uncorrelated with ψ when number of instruments K grows at the same rate as the sample size. Requires homoskedasticity: Under heteroskedasticity βˆliml notconsistentwhen K grows as fast as n; more on this below. First result on this is Bekker (1994). Gives consistent standard errors under Gaussian (ε i,v i ), or when have zero third moments and no kurtosis. Hansen, Hausman, and Newey (2007), Estimation with Many Instrumental Variables, give results for non Gaussian disturbances. Variance matrix formula is different than Bekker (1994) with nonnormality but not much different if K/T is small.

21 To describe the Bekker (1994) variance estimator, let P = Z(Z 0 Z) 1 Z 0 be the projection matrix used to form βˆliml and ˆε = y XβˆLIML, σˆ2 ε =ˆε 0ˆε/T, αˆ =ˆε 0 P ˆε/ˆε 0ˆε, Xˆ = X ˆε(ˆε 0 X)/ˆε 0ˆε, Ĥ = X 0 PX αx ˆ 0X, Σˆ =ˆσ 2 ε[(1 α) ˆ 2Xˆ 0 P Xˆ +ˆα 2 Xˆ 0 (I P )Xˆ] The Bekker variance estimator is Vˆ = Ĥ 1 ΣˆĤ 1 d p n(βˆliml β 0 ) N(0,V ),nvˆ V. Can show Vˆ bigger than usual variance formula and can be substantially so even when K/n is small. Ex: Monte Carlo based on Angrist and Krueger (1991). Bias/β RMSE Size 2SLS LIML Bekker SE s 0.049

22 Many Instruments and Heteroskedasticity 6 LIML is not consistent with heteroskedasticity. "Jackknife" version is, Hausman, Newey, Woutersen, Chao, and Swanson (2007). ˆ rgminqˆ(β), Qˆ(β) = (y X0 β)p (y X 0 β) P i P ii (y i X i 0 β) 2. β HLIM =a β (y X 0 β) 0 (y X 0 β) Heteroskedasticity/many instrument robust standard errors: Let X =[y, X], α be the smallest eigenvalue of (X 0 X ) 1 (X 0 P X P i P ii X ix i0 ), and H = X0PX P i P ii X i X 0 i αx 0 X. Then βˆhlim = H 1 (X 0 Py X For ε = y XβˆHLIM and X = X ε(x 0 ε)/ ε 0 ε, let i P ii X i y i αx 0 y), The variance estimator is nx X X Σ = X ip ik ε 2 k PkjX j0 + i,j=1 k/ {i,j} i=j P 2 ij X i ε i ε j X j0 Ṽ = H 1 Σ H 1, d p n(βˆhlim β 0 ) N(0,V ),nvˆ V.

23 Median Bias μ 2 K LIML HLIM HFUL HFUL 1 k JIV E CUE Nine Decile Range:.05 to.95 μ 2 K LIML HLIM HFUL HFUL 1 k JIV E CUE E