The LIML Estimator Has Finite Moments! T. W. Anderson. Department of Economics and Department of Statistics. Stanford University, Stanford, CA 94305

Transcription

1 The LIML Estimator Has Finite Moments! T. W. Anderson Department of Economics and Department of Statistics Stanford University, Stanford, CA 9435 March 25, 2 Abstract The Limited Information Maximum Likelihood estimator of the vector of coefficients of a structural equation in a simultaneous equation model is the vector that defines the linear combination maximizing the effect variance relative to the error variance. If this eigenvector solution is normalized by setting a designated coefficient equal to, the second-order moment of the estimator may be unbounded. However, the second-order moment is finite if the normalization is setting the sample error variance of the linear combination equal to. Key words: Limited Information Maximum Likelihood, bounded moments, normalization. Thanks go to Naoto Kunitomo for very helpful suggestions.

2 Introduction. The Limited Information Maximum Likelihood (LIML) estimator (Anderson and Rubin, 949) was introduced as the solution of an eigenvalue problem, that is, a solution for b in a homogeneous equation (G d H)b =, () where d is the smallest root of G dh =, (2) and G and H are p p positive semi-definite and positive definite matrices, respectively. A solution b to () can be multiplied by a nonzero number to obtain another solution. The conventional normalization of b has been to require b Φb =, (3) where Φ is a given p p constant positive semi-definite matrix. When Φ =, (4) the normalization is equivalent to setting b =, where b is the first component of b. There is a general perception among econometricians that the LIML estimator does not have finite moments. For example, Fuller (977) wrote the limited information estimator does not possess moments. Mariano and Sawa (972) have found the exact distribution of b 2 when b = (, b 2 ) and showed that Eb 2 2 < is not necessarily true. The purpose of this present paper is to show that the natural normalization of b Hb = (5) T 2

3 yields an estimator with finite moments (if T p + q + 2). While the conventional normalization has some advantages in interpretation, the natural normalization has the advantage of invariance with respect to linear transformations, for example, changes of the units of measurement. Anderson and Rubin considered both normalizations. 2 The model, normalizations, and invariances. Let Y be a T p matrix of observed endogenous or dependent variables, Z a T q matrix of exogenous or independent variables, and V a T p matrix of unobserved random disturbances satisfying Y = ZΠ + V, (6) the reduced form. The rows of V = v. v T (7) are assumed to be independently normally distributed with mean Ev t = and covariance Ev t v t = Ω (assumed to be positive definite). A structural equation may be defined by Yβ = u, (8) where u = Vβ and Πβ =. (9) If rank(π) = p, (9) determines β uniquely except for a multiplicative constant. The natural normalization of the parameter vector β is β Ωβ =. () 3

4 The conventional normalization is β Φβ =, () where Φ is a given p p constant positive semi-definite matrix. It is conventional to take Φ as (4), which is equivalent to setting β =, where β is the first component of β. (Sometimes it is convenient to write β = (, β 2 ).) An important advantage of the natural normalization is that the model is invariant with respect to linear transformations of the dependent variable. If Y = YC, Ω = C ΩC, Π = ΠC, β = C β, V = VC, (2) where C is nonsingular, then Π β =, β Ω β =, (3) corresponding to (9) and (). Regardless of normalization, the model is invariant with respect to linear transformations of the exogenous variables. If Z + = ZD, Π + = D Π, (4) where D is nonsingular, then Π + β = (5) corresponding to (9). The model (6), (9), and () is invariant with respect to nonsingular linear transformations of Y and Z. 4

5 3 Estimation. Theorem. The LIML estimator of β normalized by () is invariant with respect to transformations (2) and (4). A sufficient set of statistics is P = A Z Y, H = Y Y P AP, (6) where A = Z Z. Under the transformations (2) and (4) A transforms to D AD, P transforms to D PC, and H transforms to C HC. Anderson and Rubin derived the maximum likelihood estimates of Π, Ω, and β under the conditions (9) and (). Let b satisfy () where G = P AP, H = Y Y P AP, (7) and b Hb =. (8) T Then the maximum likelihood estimator of β satisfying (9) and () is ˆβ = b ˆΩb b = + d b, (9) where ˆΩ = T H + d ( ( ) T )bb T H H. (2) Note that (8) and (2) imply b ˆΩb = + d (2) and ˆβ ˆΩˆβ =. (22) 5

6 Theorem 2. The LIML estimator ˆβ normalized by () satisfies E ˆβ ˆβ < (23) if T p + q + 2. Proof. By virtue of (8), (9), and (2) Note that E ˆβ ˆβ b b b b = E = E + d ( + d )b T Hb b b E b E T Hb min c min c T c Hc c c. (24) c Hc c c = l, (25) where l is the smallest characteristic root of H, which has a Wishart distribution with T q degrees of freedom and EH = (T q)ω. By the invariance (2) E (/l ) < if and only if E (/m ) <, where m is the smallest characteristic root of H = C HC for Ω = I p. The density of m m 2... m p (the roots of H mi p = ) is where n = T q and f (m,..., m p ; n, p) p = C m 2 (n p ) i e 2 i= P p i= m i i<j (m j m i ), p n, (26) C = π 2 p2 2 2 pn Γ p (n)γ p (p) (27) [Anderson (23), Theorem 3.3.2]. Then E m = m p... m 2 m f (m,..., m p ; n, p)dm... dm p dm p 6

7 C... ( ) = C 2 n p 2 (p2 ) Γ Γ 2 m 2 (n p 3) m 2 (n p+) 2...m 2 (n+p 3) p e 2 P p i= m i dm... dm p ( n p )... Γ ( n + p 2 ). (28) Thus E (/m ) < if n p + 2, that is, if T q p + 2. Q.E.D. Mariano and Sawa (972) pointed out that the LIML estimator for p = 2 with the conventional normalization does not necessarily have finite moments. Their analysis is based on a doubly-infinite series for the density of ˆβ 2 when the conventional normalization is used and ˆβ 2 is a scalar. Theorem 2 shows that the lack of finite moments under conventional normalization is a feature of the normalization, not of the LIML estimator itself. 4 Generalizations. In a more general model the structural equation is Y β = Z γ + u, (29) where Y = (Y,Y 2 ), Z = (Z,Z 2 ), V = (V,V 2 ), u = Vβ = V β, Π = Π Π 2 Π 2 Π 22, β = β. (3) Then Πβ = is Π β Π 2 β = γ. (3) 7

8 Define A = A A 2 A 2 A 22 = Z Z Z Z 2 Z 2 Z Z 2 Z 2, (32) A 22 Z 2 = A 22 A 2 A A 2, (33) = Z 2 Z A A 2, P 2 = A 22 Z 2 Y, (34) H = H H 2 H 2 H 22, Ω = Ω Ω 2 Ω 2 Ω 22. (35) Then G and H in () and (2) are replaced by P 2A 22 P 2 and H = Y Y P A P P 2 A 22 P 2. (36) H (p p ) has the Wishart distribution with T q degrees of freedom and EH = (T q)ω. The condition β Ωβ = is β Ω β =. (37) Then E ˆβ ˆβ = E ˆβ ˆβ < if T q p + 2. It might be pointed out that a LIML estimator can be calculated with one normalization and converted to another. For example, the natural normalization may yield ˆβ = (ˆβ, ˆβ 2 ( as the estimator; then, ˆβ ) 2/ˆβ is an estimator with a conventional normalization; if ˆβ is very small, one can expect that ˆβ 2/ˆβ or ˆβ 2/ˆβ will have a large variance. The argument in (24) and (25) holds whether or not H has a Wishart distribution. Thus E ˆβ ˆβ E T m (38) 8 )

9 if T q p when ˆβ has the natural normalization. Suppose H has a density g (m,..., m p ), where m <... < m p are the characteristic roots of H. Then the density of m,..., m p is [Anderson (23), Theorem 3.3.]. Then E m π 2 p2 Γ p ( 2 p) π 2 p2 g (m,..., m p ) Γ p ( 2 p) Π i>j (m i m j ) (39)... m m 2...m p p g (m,..., m p ) dm... dm p. (4) Fuller (977) suggested a modification of the LIML estimator with the conventional normalization to avoid the features of unbounded moments. Roughly speaking, the modification is a compromise between the two-stage least squares and the LIML estimator, replacing d in () by a quantity smaller than d and using the last p coordinates of (). With the natural normalization there is no need for the Fuller modification. 5 Known covariance matrix. When Ω is known, the estimator is based on G dω =, (G d Ω)b = (4) and is known as the LIMLK estimator. The natural normalization is b Ωb =. Let Ω = Γ Γ, where Γ Γ = I and = diag (δ,..., δ p ). Then = b Γ Γb b Γ Γbmin(δ,..., δ p ) = b bmin(δ,..., δ p ). (42) 9

10 Hence Eb b min(δ,..., δ p ) <. (43) When Ω = Ω = I in (2), the transformation matrix C is orthogonal C C = I. Anderson, Stein, and Zaman (985) have proved an admissibility theorem that demonstrates an advantage of the natural normalization. Let Ω = I 2, β = (cos φ, sinφ), α = ( sin φ, cosφ), and Π = ηα. Consider estimation of φ with the loss function sin 2 (ˆφ φ ) = (ˆα α )2 = (ˆβ β ) 2, (44) where ˆα is an estimator of α and ˆφ is the corresponding estimator of φ. Then the solution of G di 2 =, (G d I 2 )b, (45) and b b = is the best invariant estimator for specified value of η η (Theorem of Anderson, Stein, and Zaman). Note that the risk, E sin 2 (ˆφ φ ), of an invariant procedure depends only on η η. It follows (Corollary of Anderson, Stein and Zaman) that the LIMLK estimator with normalization b b = and loss function (44) is admissible. Roughly speaking, for every η η this LIMLK estimator minimizes E sin 2 (ˆφ φ ). An intuitive understanding of the possible difficulty with the conventional normalization is to consider the case of G = 2 and Ω = I 2. Then in the natural normalization β = cos φ sinφ, ˆβ = cos ˆφ sin ˆφ, (46)

11 while in the conventional normalization β = sinφ cos φ = tan φ, ˆβ = tan ˆφ. (47) If φ is close to π/2, tan φ will be large. (tan φ as φ π/2.) Similarly ˆφ will be close to π/2, tan ˆφ will be large, and ˆφ φ can be expected to have a large variance.

12 References Anderson, T.W. (23), An Introduction to Multivariate Statistical Analysis, Third Edition, John Wiley and Sons, Inc. Anderson, T.W., and Herman Rubin (949), Estimation of the parameters of a single equation in a complete system of stochastic equations, Annals of Mathematical Statistics, 2, Anderson, T.W., Charles Stein, and Asad Zaman (985), Best invariant estimation of a direction parameter, Annals of Statistics, 3, Mariano, Roberto S., and Takamitsu Sawa (972), The exact finite-sample distribution of the limited-information maximum likelihood estimator in the case of two included endogenous variables, Journal of the American Statistical Association, 67, Fuller, Wayne A. (977), Some properties of a modification of the limited information estimator, Econometrica 65,