On asymptotic properties of Quasi-ML, GMM and. EL estimators of covariance structure models

Transcription

1 On asymptotic properties of Quasi-ML, GMM and EL estimators of covariance structure models Artem Prokhorov November, 006 Abstract The paper considers estimation of covariance structure models by QMLE, GMM and EL. A general condition is derived under which the GMM (and EL) estimators do not dominate normal QMLE in terms of first-order efficiency. The condition is formulated in terms of the fourth order moments of the true distribution. The second-order asymptotic bias of QMLE is derived and a formal proof is presented of the intuitive result that, under normality, this bias is the same as that of EL. JEL Classification: C13 Keywords: GMM, (Q)MLE, EL, Covariance structures, LISREL, MIMIC. Helpful comments from Emma Iglesias, Peter Schmidt, Jeffrey Wooldridge and participants of the 006 Canadian Econometrics Study Group meeting are gratefully acknowledged. Department of Economics, Concordia University, 1455 de Maisonneuve Blvd W, Montreal QC H4ES1 Canada; artem.prokhorov@concordia.ca

2 1 Introduction This paper considers estimation of covariance structure models, i.e. models formulated in terms of the second moments of the data. One situation when such models arise is when there are unobserved variables whose presence in the model introduces a particular pattern of correlation between observed variables (e.g., linear structural relationship (LISREL) models, multiple indicators multiple causes (MIMIC) models, factor analysis and random effect models). Traditional estimation methods for such models are based on the assumption of multivariate normality (see, e.g., Jöreskog, 1970; Jöreskog and Goldberger, 1975; Jöreskog and Sörbom, 1977). Because central moments of even orders for normally distributed vectors can be expressed in terms of elements of the covariance matrix, no improvements can be made by using higher order moments of the data. The maximum likelihood estimator (MLE) is efficient under normality. If the data are not normal, MLE is still consistent. However, the MLE standard errors are wrong and consequently inference may be incorrect. An obvious way to make inference robust to non-normality is to adjust standard errors using the sandwich form of the variance matrix. The exact form of the variance matrix for normal quasi-mle of covariance structures can be found, e.g., in Chamberlain (1984, p. 195). Although obvious, the QMLE improvement to covariance structure modelling does not seem to be widely implemented. For example, the popular software packages used for covariance structure models do not do that (see, e.g., Jöreskog and Sörbom, 1996). It is well known that the efficient generalized method of moments estimator (GMM) optimally uses the information available in the moment conditions ( efficient here means firstorder efficient ). For covariance structures, this means that GMM makes efficient use of the restrictions on the second moments whether or not the data are in fact normal. Similarly, the family of empirical likelihood (EL) estimators that are first order asymptotic equivalents of GMM, possess the same property. It is therefore intuitive that the GMM estimator of covariance structures should be no worse than normal QMLE asymptotically. This intuition has been noted in Chamberlain (198, 1984); Ahn and Schmidt (1995) and other papers. The

3 common argument is that the GMM estimator attains the lower bound for the asymptotic variance matrix for estimators that use restrictions on the second moments. One of this paper s contributions is that it provides a formal comparison between the estimators to the first order in terms of the relevant variance matrices. It presents an explicit condition for equal relative efficiency of the GMM estimator and normal QMLE. The condition is expressed in terms of the fourth moments of the data and normality is shown to be one case when the condition holds. Such a representation provides a clear form of the efficiency gain and identifies a family of distributions for which the normal QMLE and the efficient GMM are equally efficient. This result is given in Section 3. Section describes the general model and the estimators. Section 4 considers second order asymptotics. It is well known that the GMM estimator has a second order bias that contains more terms than that of the EL estimator. Newey et al. (003); Newey and Smith (004) derive the relevant bias expressions. The extra bias terms in GMM come from the estimation of the optimal weighting matrix and the derivative matrix that are both parts of the GMM first order conditions. It is unknown how the two estimators (GMM and EL) compare to normal QMLE in terms of the second order bias. Intuitively, the answer to the question of second order asymptotic comparisons is clear. If the true distribution of the data is discrete then MLE and EL are identical. Furthermore, the bias expression for EL does not depend on discreteness, only on existence of certain moments. So if the assumed distribution (normal, in the case of Gaussian QMLE) turns out to be correct, we should expect the same bias. However it is still interesting to have an explicit form of the QMLE bias so that comparisons with other distributions can conceivably be made. We first derive the second order bias of normal QMLE expressed in terms of higher moments of the true distribution and then show formally that it is in fact the same as EL if the data are normal. 3

4 Preliminaries.1 Setup and assumptions Consider a family of distributions {P θ, θ Θ R p, Θ compact} and a random vector Z Z R q from P θo, θ o Θ, such that EZ = 0, E{ Z 4 } < and E [ ZZ ] = Σ(θ), if and only if θ = θ o. (1) Expectation is with respect to P θo. The measurable real-valued matrix function Σ(θ) comes from a structural model such as a factor model, a random effects model, a simultaneous equations model, a conditional expectation model. For example, Ahn and Schmidt (1995) show how this setup arises in a dynamic panel setting. For a random sample (Z 1,..., Z N ), where Z i is measured as deviations from the mean, denote S i Z i Z i () and S 1 N N S i. (3) The problem is to estimate θ o given the random sample (Z 1,..., Z N ). It is easy to see that since we assumed that the fourth moments exist, then S satisfies a central limit theorem. Thus we can write vec(s) N(vec(Σ(θ o )), (θ o )), where (θ) = V(vec(S i )) = Evec(S i )vec(s i ) vec(σ(θ))vec(σ(θ)) (4) and vec denotes vertical vectorization of a matrix. To save space we will often omit the argument of matrix-functions and write Σ instead of Σ(θ), Σ o instead of Σ(θ o ), o instead of (θ o ), etc. 4

5 It is well known (see, e.g., Magnus and Neudecker, 1988, p. 53) that for the multivariate normal distribution we have o = (Σ o Σ o )(I q + Π q ) = (I q + Π q )(Σ o Σ o ), (5) where denotes the Kronecker product, I k is the identity matrix of dimension k, Π m is the commutation matrix, i.e. such an m m -matrix that Π m vec(a) = vec(a ), for any m m matrix A. Thus the fourth moments of the multivariate normal distribution are expressed in terms of the second moments. We will also need certain smoothness conditions on Σ. Such conditions combined with the above restrictions on Z are summarized in the following assumption. Assumption.1 (i) Z i Z R q, i = 1,..., N are iid from a distribution P θo, θ o Θ R p, Θ compact; (ii) EZ = 0, E{ Z 4 } < and E [ZZ ] = Σ(θ) if and only if θ = θ o ; (iii) θ o int(θ) and p 1 q(q + 1); (iv) vec(σ) is continuous at each θ Θ; (v) vec(σ) is three times continuously differentiable on a neighborhood N of θ o.. Estimators..1 Normal (Q)MLE The normal QML estimator is where ˆθ QMLE = arg max θ Θ N ln f(z i, θ), (6) f(z i, θ) = 1 e 1 (π) p Z i Σ 1 Z i. Σ It is easy to see that the problem in (6) can be equivalently written as ˆθ QMLE = arg min θ Θ F MLE(θ), 5

6 where F MLE (θ) = log Σ + tr(sσ 1 ). (7) Thus QMLE amounts to finding the value of θ that minimize distance (7) between the sample covariance matrix S and the covariance matrix Σ imposed by the model. It is a standard result (see, e.g., Chamberlain, 1984, p. 189) that, under Assumption.1, the normal QMLE of θ o is consistent and asymptotically normal... GMM The optimal GMM estimator of θ o is based on the distinct elements of (1), i.e. on the moment conditions E[m(Z i ; θ o )] = 0, (8) where m(z i ; θ) = vech(s i ) vech(σ) and vech denotes vertical vectorization of the lower triangle of a matrix. Thus m is a 1 q(q + 1)-vector. The optimal GMM estimator of θ o is obtained as the solution to the following problem: where ˆθ GMM = arg min θ Θ F GMM(θ), F GMM (θ) = m N (θ) Wm N (θ), (9) m N (θ) = 1 N N m(z i ; θ) = vech(s) vech(σ), and W is the appropriate (optimal) weighting matrix. The optimal weighting matrix is W o = {E[m(Z i ; θ o )m(z i ; θ o ) ]} 1. (10) 6

7 But in (9), one would typically use the following consistent estimator of W o based on a preliminary consistent estimate of θ o { 1 1 N Ŵ = [m(z i ; N ˆθ)m(z i ; ]} ˆθ). t=1 Note that there is a connection between W in (10) and in (4). To show the connection we need to define matrices that transform vech into vec and vice versa. Magnus and Neudecker (1988, p. 49) show that, for a symmetric k k matrix A there exists a unique k k(k+1) duplication matrix H k such that H k vech(a) = vec(a). Thus H k transforms vech into vec, while the Moore-Penrose inverse of H k, Hk = (H k H k) 1 H k, transforms vec into vech. Matrices H k and H k have the following properties: (i) H k H k = I k(k+1) ; (ii) Π k H k = H k, where Π k is the commutation matrix defined above; (iii) H k Hk = 1 (I k + Π k ); (iv) (I k + Π k ) H k = H k and H k (I k + Π k ) = H k. Thus, omitting the dimensionality subscript, we can write = V[vec(S i )] = V[H vech(s i )] = HV[vech(S i )]H. But V[vech(S i )] = E[m(Z i ; θ)m(z i ; θ) ]. We can therefore write the optimal weighting matrix in (10) as [ H H o ] 1. It is easy to verify that, under Assumption.1 and with Ŵ p W, the standard conditions for consistency and asymptotically normality of the GMM estimator of θ o hold (see, e.g., Newey and McFadden, 1994, Theorems.6 and 3.4)...3 EL The EL estimator of θ o is obtained as follows: subject to ˆθ EL = arg max θ Θ N ln π i N π i m(z i ; θ) = 0 7

8 and N π i = 1 It can also be shown that Assumption.1 is sufficiently strong to satisfy the conditions for consistency and asymptotic normality of ˆθ EL (see, e.g., Kitamura, 1997; Owen, 001). 3 First Order Analysis 3.1 The first order conditions Let G(θ) denote the Jacobian matrix of the moment functions in (8). Then G G(θ) = m(z i, θ) θ = vech(σ) θ. The following lemmas are used in derivation of the main results of the paper. They are well known and thus given without proof (see, e.g., Chamberlain, 1984; Hansen, 198; Qin and Lawless, 1994, for some relevant proofs). Lemma 3.1 Under Assumption.1, the first order condition for ˆθ QMLE is G H (Σ Σ) 1 H [vech(s) vech(σ)] = 0. (11) Lemma 3. Under Assumption.1, the first order condition for ˆθ GMM is G Ŵ 1 [vech(s) vech(σ)] = 0. (1) Lemma 3.3 Under Assumption.1, the first order condition for ˆθ EL is [ N 1 G π i m i m i] [vech(s) vech(σ)] = 0, (13) where m i = m(z i ; θ). In Section 4, we will use an alternative way of writing the first order conditions that circumvents the need to operate with the inverse. Define λ = [Σ(θ) Σ(θ)] 1 H m N (θ). Then the QMLE first order condition can be written as s N (β) = G(θ) H λ H m N (θ) + [Σ(θ) Σ(θ)]λ = 0 8

9 and we now have a p + q -vector of parameters β = (θ, λ ). A similar representation of the GMM and EL first order conditions was used, for example, by Newey and Smith (004). It is clear from (11)-(13) that the only thing that distinguishes the three estimators is the way in which the empirical moments m N (θ) are weighted. One way to compare the first order variances of GMM and normal QMLE is to note that ˆθ QMLE comes from the GMM problem that employs a suboptimal weighting matrix H (Σ Σ) 1 H and is therefore inferior to ˆθ GMM in terms of first-order relative efficiency. However, this argument cannot be used to derive our equal efficiency condition. 3. Relative efficiency to the first order Theorem 3.1 Suppose Assumption.1 holds. Let V denote the first order asymptotic variance matrix of the relevant estimator, i.e. V = Avar[N 1 (ˆθ θ o )]. Then, V QMLE = [G oh (Σ o Σ o ) 1 HG o ] 1 G oh (Σ o Σ o ) 1 o (Σ o Σ o ) 1 HG o (14) [G oh (Σ o Σ o ) 1 HG o ] 1, V GMM = V EL = [G o( H H o ) 1 G o ] 1. (15) Proof. See Chamberlain (1984, p. 195) for derivation of (14); see Hansen (198, p. 1048) and Qin and Lawless (1994, p. 306) for derivation of (15). If the data are multivariate normal then it is easy to show that the above expressions for variance are the same. More specifically, on using properties of the duplication matrix and equation (5) without the dimensionality subscript, the following simplifications apply: H (Σ Σ) 1 (Σ Σ) 1 H = H (Σ Σ) 1 (I + Π)(Σ Σ)(Σ Σ) 1 H = H (Σ Σ) 1 (I + Π)H = H (Σ Σ) 1 H, H H = H(I + Π)(Σ Σ) H = H(Σ Σ) H. 9

10 To see that [ H (Σ Σ) H] 1 is equal to H (Σ Σ) 1 H, note that H (Σ Σ) 1 H H(Σ Σ) H = 1 H (Σ Σ) 1 (I + Π)(Σ Σ) H = 1 H (Σ Σ) 1 (Σ Σ)(I + Π) H = 1 H (I + Π) H = 1 H H = I. Equation (15) of Theorem 3.1 states that the asymptotic variance matrices of optimal GMM and EL are equal, i.e. the two estimators of θ o are asymptotically equivalent to the first order. It is not immediately clear from only the form of (14) that QMLE is dominated by the other two estimators. The main first-order asymptotic result of this paper is stated in the next theorem. Theorem 3. Let Assumption.1 hold. The estimators ˆθ GMM and ˆθ EL are no less asymptotically efficient to the first order than ˆθ QMLE. Equal first-order efficiency occurs under the following equivalent conditions: (i) G o is in the column space of H o (Σ o Σ o ) 1 HG o ; (ii) There exists a q(q+1) q(q+1) matrix D such that G o = H o (Σ o Σ o ) 1 HG o D. Proof. V QMLE V GMM is positive semidefinite (PSD) if and only if V 1 GMM V 1 QMLE is PSD. Denote H o H by C and H (Σ o Σ o ) 1 H by A. We have V 1 GMM V 1 QMLE = G oc 1 G o G oag o [G oacag o ] 1 G oag o = G oc 1 [I C 1 AGo [G oac 1 C 1 AGo ] 1 G oac 1 ]C 1 Go. This is PSD because the middle part is the idempotent projection matrix onto C 1/ AG o. This proves the first part of the theorem. 10

11 The difference is zero if and only if C 1/ G o is in the column space spanned by C 1/ AG o, or equivalently, G o is in the column space of CAG o. Note that CAG o = H o H H (Σ o Σ o ) 1 HG o = H o 1 (I + Π)(Σ o Σ o ) 1 HG o = H o (Σ o Σ o ) 1 1 (I + Π)HG o = H o (Σ o Σ o ) 1 1 HG o = H o (Σ o Σ o ) 1 HG o. This proves both (i) and (ii). Theorem 3. is novel in that it states the first order efficiency properties of QMLE, GMM and EL explicitly in terms of the fourth moments of Z in. It is clear from the theorem, that GMM and EL dominate QMLE because they make efficient use of the second moment information without imposing restrictions on the fourth moments. Ahn and Schmidt (1995, Appendix ) showed that the GMM estimator of covariance structures reaches the semiparametric efficiency bound of Newey (1990). Theorem 3. provides an explicit expression for the gain attained by GMM over QMLE. Not surprisingly, the conditions of Theorem 3. hold for the multivariate normal distribution. Using (5), one can write H o (Σ o Σ o ) 1 HG o = H(I Π)HG o = HHG o = G o. So condition (ii) trivially holds. However, there may conceivably exist other distributions that satisfy the equal first order efficiency conditions of Theorem 3.. We leave further exploration of this point for future work. 11

12 4 Second Order Analysis 4.1 Stochastic expansions to the second order Higher order stochastic expansions are based on the Taylor approximation of the first order conditions at the true value. The expansions have the following form N( ˆβ βo ) = µ + N 1 τ + Op (N 1 ), (16) where µ and τ are O p (1) random vectors. It is well known that the first order bias can be obtained by taking the expectation of the first term. Since QMLE, GMM, and EL are N consistent, their first order bias is zero. Similarly, the first order variances can be obtained as the expectation of the outer product of first term. The second order bias is based on the expectation of the first two terms in (16). Alternatively, the second order bias can be obtained using the Edgeworth approximation to the distribution as in Rothenberg (1984) and McCullagh (1987). General expressions for µ and τ of extremum and minimum distance estimators with many examples can be found in Rilstone et al. (1996); Bao and Ullah (003); Ullah (004); Kim (005). Specialized expressions for the GMM and (generalized) EL can be found in Newey et al. (003) and Newey and Smith (004). Derivation of higher order stochastic expansions involves higher order derivatives of the objective functions. Rilstone et al. (1996) use a recursive definition of derivatives which is useful in general settings. In our derivation we follow Newey and Smith (004) in using the usual definition because we do not go to the order higher than two and because we wish to compare the QMLE bias to the GMM and EL bias expressions they derive. 1

13 Define s i (β) = G H λ H m i + (Σ Σ)λ, M j = s i (β) β β j, where β j is the j-th element of β, (17) R = [G H (Σ Σ) 1 HG] 1, Q = RG H (Σ Σ) 1, P = (Σ Σ) 1 (Σ Σ) 1 HGQ. Note that M j does not depend on i because derivatives of m i are not random. As before, we will use subscript o to denote matrices evaluated at β o = (θ o, 0 ). Theorem 4.1 Under Assumption.1, the estimator ˆβ QMLE satisfies (16) with µ = Q o P o H 1 N τ = 1/ R o Q o Q o P o N where µ j is the j-th element of µ. [vech(s i ) vech(σ o )], (18) p+q j=1 µ j M jo µ, Proof. See Appendix B. Note that Eµ = 0 and the first order variance of ˆβ QMLE based on (18) can be written as Eµµ = = Q o P o Q o o Q o H E[m(Z i ; θ o )m(z i ; θ o ) ]H P o o Q o Q o o P o P o o P o Q o P o, (19) where the upper left p p block of (19) is identical to the first order asymptotic variance of ˆθ QMLE in (14). Interestingly, the matrix in (19) is not in general block diagonal unlike the EL and GMM analogues (see, e.g., Qin and Lawless, 1994, Theorem 1). However, in the case of multivariate 13

14 normality, the blocks of (19) can be simplified as follows Q Q = R, Q P = 0, (0) P P = (I + Π)P. And thus ˆθ QMLE and ˆλ QMLE are in this case asymptotically uncorrelated. 4. Second order bias of QMLE Let B denote the second order bias of the relevant estimator. Using (16), the bias can be written in terms of the expected value of τ as B = Eτ /N. Thus, an explicit form of the QMLE bias contains Eµ j M jo µ, j = 1,..., p + q. But M jo can be written as M jo = β β j where G j o = = θ j G G H λ H m i + (Σ Σ)λ 0 Gj o H HG j o Ω j o, θ=θ o,λ=0 j = 1,..., p G j p,o 0, j = p + 1,..., p + q Ω j p,o 0, G j p,o = θ=θo θ [G H e j p ] θ=θo, Ω j o = θ j (Σ Σ), Ω j p,o = θ=θo θ [(Σ Σ)e j p] θ=θo, and e j p is a q -vector of zeros with the (j p)-th element equal to 1. Therefore M j is non-random and we can write 0 Gj o H Eµµ e j, j = 1,..., p HG j o Ω j o Eµ j M jo µ = G j p,o 0 Eµµ e j, j = p + 1,..., p + q, Ω j p,o 0 where e k is a p + q -vector of zeros with the k-th element equal to 1. Substituting (19) into (1) and simplifying yields the result of the following theorem. (1) 14

15 Theorem 4. Under Assumption.1, the second order bias of ˆβ QMLE can be written as follows B QMLE = 1 N + p+q j=p+1 R o Q o Q o P o G j p,o Ω j p,o p 0 Gj o H Q o o Q o e j j=1 HG j o Ω j o P o o Q o Q o o P oe j p, () where e k is the zero vector of relevant dimension in which the k-th element is 1. McCullagh (1987) and Linton (1997) give expressions for the second order bias of QMLE in terms of cumulants; we use the higher-moment representation to enable comparison with second order biases derived in Newey and Smith (004). Based on (0), the following simplification applies in the multivariate normal case: B QMLE = 1 R o Q o p 0 Gj o H R o e N Q o P j o j=1 HG j o Ω j o 0 = 1 R o Q o 0 N Q o P o p j=1 HGj or o e j = 1 Q oh p j=1 Gj or o e j N P o H. (3) p j=1 Gj or o e j 4.3 Comparison to GMM and EL Newey and Smith s (004, Theorems 4.1 and 4.6) second order biases of the GMM and EL estimators of θ o are, in our notation, B EL = 1 N QEL o p j=1 B GMM = B EL + 1 N QEL o U o, G j or EL o e j, (4) 15

16 where Q EL = R EL G [Em i m i], R EL = (G [Em i m i] 1 G) 1, U = E[m i m ipm i ]. It is not clear how these compare to B QMLE (θ) in general. However, when Z is multivariate normal, it is easy to show that the upper block of B QMLE is equal to (4) since, under normality, R EL = {G [ H(Σ Σ) H ] 1 G} 1 = [G H (Σ Σ) 1 HG] 1 = R, Q EL = R EL G [ H(Σ Σ) H ] 1 = RG H (Σ Σ) 1 H = QH. An interesting related question is whether the equal bias result can be shown for other distributions than normal. 5 Concluding Remarks The paper examined three estimation methods available for covariance structure models in terms of their first order asymptotic variance and second order asymptotic bias. We have derived a condition for the GMM, EL and normal QML estimator to be equally efficient to the first order. This condition is formulated in terms of the fourth moments and can be used to characterize a class of distributions for which no efficiency gain is obtained by the nonparametric estimation of the optimal weights. We have derived the second order bias for the normal QML estimator of covariance structures and have shown formally that if the distribution is in fact normal the ML bias is the same as the EL bias. Finally, the expressions for the second order bias of (Q)MLE we derive can be used to construct a bias corrected (Q)ML estimator of covariance structure models. 16

17 References Ahn, S. C. and P. Schmidt (1995): Efficient estimation of models for dynamic panel data, Journal of Econometrics, 68, 5 7. Bao, Y. and A. Ullah (003): The Second-Order Bias and Mean Squared Error of Estimators in Time Series Models, Working Paper, University of California Riverside, Chamberlain, G. (198): Multivariate regression models for panel data, Journal of Econometrics, 18, (1984): Panel data, in Handbook of Econometrics, ed. by Z. Griliches and M. D. Intriligator, vol. II, Hansen, L. (198): Large sample properties of generalized method of moments estimators, Econometrica, 50, Jöreskog, K. G. (1970): A general method for analysis of covariance structures, Biometrika, 57, Jöreskog, K. G. and A. S. Goldberger (1975): Estimation of a model with multiple indicators and multiple causes of a single latent variable, Journal of American Statistical Association, 70, Jöreskog, K. G. and D. Sörbom (1977): Statistical models and methods for analysis of longitudinal data, in Latent Variables in Socio-Economic Models, ed. by D. a. A.Goldberger, Amsterdam: Publishing Company, Contributions to Economic Analysis, North-Holland (1996): LISREL 8 User s Reference Guide., SSI Scientific Software. Kim, K.-i. (005): Higher order bias correcting moment equation for M-estimation, UCLA Papers. Kitamura, Y. (1997): Empirical likelihood methods with weakly dependent processes, The Annals of Statistics, 5, Linton, O. (1997): An asymptotic expansion in the GARCH(1,1) model, Econometric Theory, 13, 558. Magnus, J. R. and H. Neudecker (1988): Matrix differential calculus with applications in statistics and econometrics, Wiley Series in Probability and Statistics, Chichester: John Wiley and Sons Ltd. McCullagh, P. (1987): Tensor Methods in Statistics, Monographs on Statistics and Applied Probability, London: Chapman and Hall. Newey, W. (1990): Semiparametric efficiency bounds, Journal of Applied Econometrics, 5,

18 Newey, W. and D. McFadden (1994): Large sample estimation and hypothesis testing, in Handbook of Econometrics, ed. by R. Engle and D. McFadden, vol. IV, Newey, W. K., J. S. Ramalho, and R. J. Smith (003): Asymptotic bias for GMM and GEL estimators with estimated nuisance parameters, CEMMAP working paper CWP05/03. Newey, W. K. and R. J. Smith (004): Likelihood estimators, Econometrica, 7, Higher order properties of GMM and Generalized Empirical Owen, A. B. (001): Empirical likelihood, Monographs on statistics and applied probability; 9, Boca Raton, Fla. : Chapman and Hall. Qin, J. and J. Lawless (1994): Empirical likelihood and general estimating equations, The Annals of Statistics,, Rilstone, P., V. Srivastava, and A. Ullah (1996): The second-order bias and mean squared error of nonlinear estimators, Journal of Econometrics, 75, Rothenberg, T. (1984): Approximating the distributions of econometric estimators and test statictics, in Handbook of Econometrics, ed. by Z. Griliches and M. D. Intriligator, vol. II, Ullah, A. (004): Finite Sample Econometrics, Advanced Texts in Econometrics, Oxford University Press. A Proof of Theorem 4.1 Let M(β) = 1 N P N s i (β), M(β) = E s i(β), M β β j(β) P = 1 N s i (β) N β β j and β be between ˆβ and β o. Note that because s i(β) β is non-random, M(β) = M(β). By the second-order Taylor expansion of (11) around β o, we have s N ( ˆβ) = 0 = s N (β o ) + M(β o )( ˆβ β o ) + 1 p+q X j=1 ( ˆβ j β oj ) M j( β)( ˆβ β o ) = s N (β o ) + M(β o )( ˆβ β o ) + [ M(β o ) M(β o )]( ˆβ β o ) p+q X j=1 p+q X j=1 ( ˆβ j β oj )M j(β o )( ˆβ β o ) + ( ˆβ j β oj )[ M j( β) M j(β o )]( ˆβ β o ). Since M(β o ) = M(β o ), the third term in the last equation is zero. Also note that the last term is O p(n 3/ ). 18

19 Assume that M(β o ) is not singular. Then, ˆβ β o = [M(β o )] 1 s N (β o ) 1 p+q X [M(β o )] 1 ( ˆβ j β oj )M j(β o )( ˆβ β o ) j=1 +O p(n 3/ ). (5) 3 0 G But M(β o ) = 4 oh 5, sn (β o ) = HG o Σ o Σ o thus have ˆβ β o = = 1 N 4 Q o P o 3 5 H 1 N NX 4 0 H m N (θ o) [vech(s i) vech(σ o)] + O p(n 1 ) 3 5 and the second term is Op(N 1 ). We 1 N µ + O p(n 1 ). (6) Substituting (6) into (5), multiplying by N and collecting terms of the same order yields the result. 19