Generalized Method of Moment

Generalized Method of Moment CHUNG-MING KUAN Department of Finance & CRETA National Taiwan University June 16, 2010 C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 1 / 32

Lecture Outline 1 Generalized Method of Moments GMM Estimation Asymptotic Properties of the GMM Estimator Different GMM Estimators Examples 2 Large Sample Tests Over-Identifying Restriction Test Hausman Test Wald Test 3 Conditional Moment Restrictions Estimation of Conditional Moment Restrictions C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 2 / 32

Moment Conditions Given y t = x t β + e t, consider the moment function: IE(x t e t = IE[x t (y t x t β]. When x tβ is correctly specified for the linear projection, we have the moment condition: IE[x t (y t x t β o ] = 0. Its sample counterpart, 1 T T x t (y t x tβ = 0, is also the FOC for OLS estimation. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 3 / 32

Similarly, given y t = f (x t ; β + e t and the moment function: IE[ f (x t ; βe t ] = IE{ f (x t ; β[y t f (x t ; β]}. When f (x t ; β is correctly specified for IE(y t x t, we have the moment condition IE{x t [y t f (x t ; β o ]} = 0. Its sample counterpart is the FOC for NLS estimation: 1 T T f (x t ; β[y t f (x t ; β] = 0. When the quasi-likelihood function f (x t ; θ is correctly specified, the moment condition IE[ ln f (x t ; θ o ] = 0 holds. The sample counterpart of this moment condition is the average of the score functions and yields the QMLE θ. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 4 / 32

Given y t = x t β + e t and the moment function: IE(z t e t = IE[z t (y t x t β]. When the variables z t are proper instrument variables such that IE[z t (y t x t β o ] = 0. The sample counterpart of this moment condition is the FOC for IV estimation: 1 T T z t (y t x tβ = 0. Note that we can not solve for unknown parameters if the number of moment conditions is more than the number of parameters. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 5 / 32

GMM Estimation Consider q moment functions IE[m(z t ; θ], where θ (k 1 is the parameter vector. Suppose there exists unique θ o such that IE[m(z t ; θ o ] = 0. These conditions are exactly identified if q = k and over-identified if q > k. When the conditions are exactly identified, θ o can be estimated by solving their sample counterpart: m T (θ = 1 T T m(z t ; θ = 0. This is the method of moment, which is not applicable when the conditions are over-identified. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 6 / 32

Given a q q symmetric and p.d. weighting matrix W o, note that the following quadratic objective function, Q(θ; W o := IE[m(z t ; θ] W o IE[m(z t ; θ], is minimized at θ = θ o. The generalized method of moments (GMM of Hansen (1982, Econometrica suggests estimating θ o by minimizing the sample counterpart of Q(θ; W o : Q T (θ; W T = [ m T (θ ] WT [ mt (θ ], where W T is a symmetric and p.d. weighting matrix, possibly dependent on the sample, and W T converges to W o in probability. The GMM estimator is thus ˆθ T (W T = arg min θ Θ Q T (θ; W T, which clearly depends on the choice of W T. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 7 / 32

The FOC of GMM estimation contains k equations in k unknowns: G T (θ W T m T (θ = 0, where G T (θ = T 1 T m(z t ; θ. The GMM estimator is then solved using a nonlinear optimization algorithm. For example, in the linear regression case, m T (β = 1 T T x t (y t x t β. When W T = I k, the FOC is ( ( 1 T x T t x 1 T t x T t (y t x t β = 0. Clearly, the resulting GMM estimator is the OLS estimator. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 8 / 32

Consistency By invoking a suitable ULLN, Q T (θ; W T is close to Q(θ; W o uniformly in θ when T becomes large. Hence, the GMM estimator ˆθ T (W T ought to be close to θ o, the minimizer of Q(θ; W o, for sufficiently large T. This is the underlying idea of establishing GMM consistency. This approach is analogous to that for NLS and QMLE consistency. Note that consistency does not depend on the weighting matrix. For example, ˆθ T (I q is consistent for θ o, which is also the minimizer of Q(θ; I q. Consider the mean value expansion: T mt (ˆθT (W T = T m T (θ o + G T (θ T T (ˆθT (W T θ o, where θ T lies between ˆθ T (W T and θ o. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 9 / 32

Asymptotic Distribution Using the FOC of GMM estimation, 0 = G T (ˆθ T (W T WT T mt (ˆθ T (W T = G T (ˆθT (W T WT T mt (θ o + G T (ˆθT (W T WT G T (θ T T (ˆθT (W T θ o. When m(z t ; θ obey a ULLN, so that G T (ˆθT (W T converges to G o = IE[ m(z t ; θ o ]. Then, T (ˆθT (W T θ o = { IE[ m(zt ; θ o ] W o IE[ m(z t ; θ o ] } 1 IE[ m(z t ; θ o ] W o [ T m T (θ o ] + o IP (1 = ( G o W o G 1G o o W o T mt (θ o + o IP (1. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 10 / 32

The asymptotic distribution of T (ˆθT (W T θ o is thus determined by T mt (θ o = 1 T T m(z t ; θ o. When m(z t ; θ o obey a central limit theorem: 1 T T m(z t ; θ o we have T (ˆθT (W T θ o D N (0, Σ o, D N (0, Ωo, where Ω o = ( G o W o G 1G o o W o Σ o W o G ( o G 1. o W o G o When W o = Σ 1 o, it is easy to see that Ω o simplifies to ( G o Σ 1 o G 1G o o Σ 1 o G o ( G o Σ 1 o G o 1 = ( G o Σ 1 o G o 1 =: Ω o. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 11 / 32

Asymptotic Efficiency To compare Ω o and Ω o, note that (Ω o 1 Ω 1 o = G o Σ 1 o G o G o W o G o ( G o W o Σ o W o G 1G o o W o G o = G o Σ 1/2 o [ ( I Σ 1/2 o W o G o G o W o Σ1/2 o Σ 1/2 1G o W o G o o W ] o Σ1/2 o Σ 1/2 o G o, which is p.s.d. because the matrix in the square bracket is symmetric and idempotent. Thus, Ω o Ω o is p.s.d. This shows that, given the weighting matrix whose limit is Σ 1 o, the resulting GMM estimator is asymptotically efficient. Σ 1 o is thus known as the optimal (limiting weighting matrix. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 12 / 32

Two-Step Optimal GMM Estimator Hansen and Singleton (1982, Econometrica: 1 Compute a preliminary, consistent estimator ˆθ 1,T based on the pre-specified weighting matrix W 0,T : [ ˆθ 1,T := arg min mt (θ ] [ W0,T mt (θ ]. θ Θ For example, we may set W 0,T = I q. 2 Compute a consistent estimator for Σ o based on ˆθ 1,T and use its inverse as 1 the optimal weighting matrix, i.e., W T (ˆθ 1,T = Σ T. 3 The two-step optimal GMM estimator is obtained as [ ˆθ 2,T := arg min mt (θ ] [ Σ 1 T mt (θ ], θ Θ which is asymptotically efficient. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 13 / 32

Drawbacks of Two-Step Estimators The finite-sample performance of the two-step estimator depends on the preliminary GMM estimator ˆθ 1,T and hence the initial weighting matrix W 0,T. Note that Σ T is also determined by m(z t ; θ. Hence, the correlation between m T and Σ T results in finite-sample bias in ˆθ 2,T. Computing a consistent estimator of Σ o may not be straightforward, especially when the data are serially correlated and heterogeneously distributed. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 14 / 32

Iterative GMM Estimator 1 At the j th iteration, compute the j th iterative GMM estimator using the weighting matrix W T (ˆθ j 1,T : [ ˆθ j,t := arg min mt (θ ] WT (ˆθ j 1,T [ m T (θ ]. θ Θ At the first iteration, we can set the initial weighting matrix W 0,T, as in the two-step estimator. 2 Use ˆθ j,t to construct the optimal weighting matrix W T (ˆθ j,t and set j = j + 1. 3 Iterate the procedure above till one of the following convergence criteria is satisfied: For some pre-specified ε, Q T (ˆθ j,t Q T (ˆθ j 1,T, ε or ˆθ j,t ˆθ j 1,T ε. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 15 / 32

Continuous Updating GMM Estimator Hansen, Heaton and Yaron (1996, JBES: Continuous updating (CU estimator is based on one-time optimization: [ ˆθ T := arg min mt (θ ] WT (θ [ m T (θ ]. θ Θ The objective function clearly changes when W T depends explicitly on θ. This does not affect the limiting distribution of the resulting estimator, however; see Pakes and Pollard (1989, Econometrica. Remark: The CU estimator is invariant when the moment conditions are re-scaled, even when the scale factor is parameter dependent; the two-stage or iterative GMM estimator is sensitive to such transformation, however. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 16 / 32

Independently Weighted Optimal Estimator Altonji and Segal (1996, JBES: The independently weighted optimal estimator avoids possible correlation between m T and Σ T by splitting the sample and computing m T and Σ T with different sub-samples. Split the sample into l groups, and let m Tj (θ be the sample average of m(z t, θ for t in the j th group with T j observations. 1 Also let Σ T be the optimal weighting matrix based on the observations not j in the j th group. The resulting GMM estimator is obtained by solving: ˆθ T := arg min θ Θ l j=1 A common choice of l is 2. [ mtj (θ ] Σ 1 T j [ mtj (θ ]. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 17 / 32

Example: Regression with Symmetric Error Given the specification y t = x t β + e t, let [ ] x m(y t, x t ; β = t (y t x t β x t (y t x. t β3 The moment condition IE[m(y t, x t ; β o ] = 0 suggests estimating β o while taking into account symmetry of the error term. The gradient vector of m is: [ ] x m(y t, x t ; β = t x t 3x t x t (y t. x t β2 If the data are independent over t, [ ] ɛ 2 t Σ o = IE x t x t ɛ 4 t x t x t ɛ 4 t x t x t ɛ 6 t x. t x t C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 18 / 32

Let ê t = y t x ˆβ t 1,T, where ˆβ 1,t is a first-step GMM estimator based on a preliminary weighting matrix. Then, Σ o may be estimated by the sample counterpart: [ ] Σ T (ˆβ 1,T = 1 T êt 2x t x t êt 4x t x t T êt 4x t x t êt 6x. t x t Note that ˆβ 1,T here may be the OLS estimator (in fact, we only need a consistent estimator for β o. The two-step GMM estimator is computed with [ ΣT (ˆβ 1,T ] 1 the weighting matrix; that is, ˆθ 2,T := arg min θ Θ m T (θ[ ΣT (ˆβ 1,T ] 1 mt (θ. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 19 / 32

Example: Generalized Instrumental Variables Estimator For the specification y t = x t β + e t, consider the moment condition: IE[m t (β o ] = IE[z t (y t x t β o ] = 0, where z t contains q > k instrumental variables. The GMM estimator minimizes ( ( 1 T z T t (y t x t β 1 W T T T z t (y t x t β, and solves ( T ( T x t z t W T z t (y t x t β = (X ZW T [Z (y Xβ] = 0. where Z (T q is the matrix of instrumental variables and X (T k is the matrix of regressors. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 20 / 32

The GMM estimator here is also the generalized instrumental variables estimator: ˆβ(W T = (X ZW T Z X 1 X ZW T Z y. When the data are independent over t and there is no condition heteroskedasticity, ( 1 T T Σ o = var z t (y t x t β o = σ2 o IE(z T T t z t. Ignoring σ 2 o in Σ o, T 1 T IE(z t z t can be estimated by Z Z/T. The two-step GMM estimator now is: ˆβ 2,T = [X Z(Z Z 1 Z X] 1 X Z(Z Z 1 Z y, which is also known as the two-stage least squares (2SLS estimator. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 21 / 32

The 2SLS estimator can be expressed as ˆθ 2,T = [ X X] 1 X y, where X = Z(Z Z 1 Z X is the matrix of fitted values from the OLS regression of X on Z. Hence, ˆθ 2,T is also the OLS estimator of regressing y on X (Theil, 1953. When the errors are not conditionally homoskedastic, the 2SLS estimator remains consistent (why?, but it is no longer the most efficient. A two-step GMM estimator with a properly estimated weighting matrix [ Σ T (ˆβ 1,T ] 1 is more efficient. For example, Σ T (ˆβ 1,T = 1 T T êt 2 z t z t, where ê t = y t x t ˆβ 1,T are the residuals from the first-step GMM estimation. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 22 / 32

Over-Identifying Restrictions Test To test whether the model for IE[m(z t, θ 0 ] = 0 is correctly specified, it is natural to check if m T (ˆθ T is sufficiently close to zero. For example, when the moment conditions are the Euler equations in different capital asset pricing models, this amounts to checking if the pricing errors are zero. The over-identifying restrictions (OIR test of Hansen (1982, also known as the J test, is based on the value of the GMM objective function: J T := T [ m T (ˆθT (W T ] WT [ mt (ˆθT (W T ]. As the test statistic involves the GMM estimator obtained from the same objective function, the limiting distribution is χ 2 (q k. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 23 / 32

To derive its limiting distribution, note that W 1/2 T T mt (ˆθT (W T = W 1/2 T T mt (θ o + W 1/2 T G T (θ T T (ˆθT (W T θ o. As T (ˆθ ( T (W T θ o = G o W o G 1G o o W o T mt (θ o + o IP (1, T mt (ˆθ T (W T = P o W 1/2 W 1/2 T o T mt (θ o + o IP (1 where P o = I W 1/2 ( o G o G o W o G 1G o o W1/2 o which is symmetric and idempotent with rank q k (why?, and W 1/2 D o T mt (θ o N (0, W 1/2 o Σ o W 1/2 o. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 24 / 32

When W o = Σ 1, the limiting distribution simplifies: o W 1/2 D o T mt (θ o N (0, I q. Consequently, J T = T m T (θ o W 1/2 o P o P o W 1/2 D o m T (θ o χ 2 (q k. Remarks: The weighting matrix in the J statistic and the weighting matrix for the GMM estimator in m T must be the same. And the weighting matrix W T for the GMM estimator must converge to W o = Σ 1 o. That is, J test requires the optimal GMM estimator. Lee and Kuan (2010 propose an OIR test that does not requires the weighting matrix to converge to Σ o and hence avoids the optimal GMM estimation. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 25 / 32

Hausman Test Given two estimators ˆθ T and ˇθ T of the parameter θ o, suppose that both are consistent under the null hypothesis but only one, say ˇθ T, is consistent under the alternative. The Hausman test suggests testing the null hypothesis by comparing these two estimators. The Hausman test is particularly useful for testing model specification, which may not be expressed as parameter restrictions. For example, consider testing the null hypothesis that regressors are exogenous against the alternative of endogenous regressors. Under classical conditions, the OLS estimator ˆθ T and the 2SLS estimator ˇθ T are consistent under the null, but only the 2SLS estimator is consistent under the alternative. Thus, we can test for endogeneity by checking if ˆθ T ˇθ T is sufficiently large. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 26 / 32

The Hausman test reads: H T = T (ˆθT ˇθ V 1 T T (ˆθT ˇθ D T χ 2 (k, where V T is a consistent estimator for the asymptotic covariance matrix of T (ˆθT ˇθ T : V (ˆθT ˇθ T = V (ˆθT + V (ˇθT 2 cov (ˆθT, ˇθ T. This asymptotic covariance matrix is simplified when ˆθ T is also asymptotically efficient under the null. In particular, we can show V 1,2 := cov (ˆθT, ˇθ T = V(ˆθT, so that V (ˆθT ˇθ T depends only on the respective asymptotic covariance matrices of these two estimators: V (ˆθT ˇθ T = V (ˇθT V (ˆθT. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 27 / 32

To see this, consider the following estimator which is a linear combination of ˆθ T and ˇθ T : ˆθ T = ˆθ T + [ V (ˆθT V1,2 ] V (ˆθT ˇθ T 1 (ˇθT ˆθ T. It is easy to verify that V (ˆθ T = V (ˆθT [ V (ˆθT V1,2 ] V (ˆθT ˇθ T 1 [ V (ˆθT V1,2 ]. If V (ˆθT is not the same as V1,2, the second term on the RHS is p.d. It follows that ˆθ T is asymptotically more efficient than ˆθ T. But this contradicts the assumption that ˆθ T is asymptotically more efficient. This suggests that V (ˆθ T = V1,2. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 28 / 32

Wald Test Hypothesis: R(θ = 0, where R : R K R r. A mean-value expansion yields R(ˆθ T = R(θ 0 + R(θ (ˆθ T θ 0, where T (ˆθ T θ 0 T [ R(ˆθT R(θ 0 ] D N (0, Ω 0. Therefore, It follows that, under the null hypothesis, D N ( 0, [ R(θo ]Ω 0 [ R(θ o ]. W T := T R (ˆθ T ( [ R (ˆθ T ] ΩT [ R (ˆθ T ] 1 R (ˆθ T D χ 2 (r. where Ω T is a consistent estimator of Ω o. For the linear hypothesis Rθ 0 = r with R a r k matrix, W T = T (Rˆθ T r (R Ω T R 1 (Rˆθ T r D χ 2 (r. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 29 / 32

Estimation of Conditional Moment Restrictions Consider the conditional moment restrictions: IE[h(η t ; θ o F t ] = 0, where h is r 1 and F t is the information set up to time t. Let w t be a set of variables from F t and may contain some variables in η t. The conditional moment restrictions above imply IE[D(w t h(η t ; θ o ] = 0, where D(w t is a (r n matrix of instruments. GMM estimation of θ o is based on the sample moments: 1 T T m t (θ = 1 T T D(w t h(η t ; θ. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 30 / 32

The optimal GMM estimator has asymptotic covariance matrix ( G 1, o Σ 1 o G o with Σ o = IE [ D(w t h(η t ; θ o h(η t ; θ o D(w t ], (n n G o = IE [ D(w t h(η t ; θ o ]. (n k For a smaller set of sample moments obtained by taking a linear transformation: 1 T T C D(w t h(η t ; θ, where C is n p with p < n, the asymptotic covariance matrix of the resulting optimal GMM estimator is [ G o C(C Σ o C 1 C G o ] 1. It is readily seen that G o Σ 1 o which is p.s.d. matrix. G o G o C(C Σ o C 1 C G o = G [ o Σ 1/2 o In Σ 1/2 o C(C Σ 1/2 o Σ 1/2 o C 1 C Σ 1/2 o ] Σ 1/2 o G o, C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 31 / 32

Number of moment conditions: The optimal GMM estimator based on a larger set of moment conditions is asymptotically more efficient, yet it may have a larger bias in finite samples. Optimal instruments: D (w t ; θ = [ var(h(η t ; θ F t ] 1 IE [ h(ηt ; θ F t] =: V 1 t J t, (r k which contains k instruments. In this case, Σ o = IE ( J t V 1 t V t V 1 ( t J t = IE J t V 1 t J t, G o = IE [ J t V 1 t h(η t ; θ o ] = IE ( J t V 1 t J t. The optimal GMM estimator thus has the asymptotic covariance matrix ( G o Σ 1 o G o 1 ( = IE J 1. t V 1 t J t As J t and V t are conditional expectations and depend on unknown parameters, it is not easy to estimate the optimal instruments in practice. C.-M. Kuan (Finance & CRETA, NTU Generalized Method of Moment June 16, 2010 32 / 32