The equalities of ordinary least-squares estimators and best linear unbiased estimators for the restricted linear model

Transcription

1 The equalities of ordinary least-squares estimators and best linear unbiased estimators for the restricted linear model Yongge Tian a and Douglas P. Wiens b a School of Economics, Shanghai University of Finance and Economics, Shanghai , China b Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, Alberta, Canada T6G 2G1 Abstract. We investigate in this paper a variety of equalities for the ordinary least-squares estimators and the best linear unbiased estimators under the general linear (Gauss-Markov) model {y, Xβ, σ 2 Σ} and the restrained model {y, Xβ Aβ = b, σ 2 Σ}. Keywords: method OLSE; BLUE; general linear model; linear restriction; parametric function; matrix rank 1 Introduction Suppose we are given the following general linear (Gauss-Markov) model y = Xβ + ε, E(ε) = 0 and Cov(ε) = σ 2 Σ, (1) constrained by a consistent linear matrix equation Aβ = b, (2) where X is a known n p matrix with rank(x) p, A is a known m p matrix with rank(a) = m, b is a known m 1 vector, y is an n 1 observable random vector, β is a p 1 vector of parameters to be estimated, 0 < σ 2 is an unknown parameter, and Σ is an n n known nonnegative definite matrix. The model (1) is often written as the following triplet M = {y, Xβ, σ 2 Σ}, (3) the model (1) with the parameters constrained by the equation (2) is written as M c = {y, Xβ Aβ = b, σ 2 Σ}. (4) In the investigation of linear models, parameter constraints are usually handled by transforming an explicitly constrained model into an implicitly constrained model. The most popular transformations are based on model reduction, Lagrangian multipliers, and reparameterization by general solution of the equation (2). Through block matrices, (1) and (2) can be written as {[ y M c = b ], [ X A ] β, σ 2 [ Σ ]}. (5) If the observable vector y in (3) arises from the combination of two sub-samples y 1 and y 2, where y 1 and y 2 are n 1 1 and n 2 1 vectors with n 1 + n 2 = n, then (3) can also be written as a partitioned form M = {[ y1 y 2 ], addresses: yongge@mail.shufe.edu.cn, doug.wiens@ualberta.ca } X(1) β, σ 2 Σ11 Σ 12, (6) X (2) Σ 21 Σ 22 1

2 where X (1) and X (2) are n 1 p, n 2 p matrices, Σ 11 is an n 1 n 1 matrix. In this case, the first sub-sample model in (6) can be written as and the first sub-sample model in (4) can be written as M 1 = {y 1, X (1) β, σ 2 Σ 11 }, (7) M 1c = {y 1, X (1) β Aβ = b, σ 2 Σ 11 }. (8) Because (4) has a linear restriction on β, the estimation of β is more complicated than that of β under (3). Of interest to us is to consider relations between estimators of β under (3) and (4). In particular, it is important to give necessary and sufficient conditions for the estimators of β under (3) and (4) to be equal. If two estimators of β under (3) and (4) are equal, it is natural to use the estimator of β under (3) instead of that of (4). The restricted linear model (4) also has a closed link with the hypothesis test Aβ = b for β under (3). From the equalities of the estimators of β under (3) and (4), one can also check the testability of the hypothesis. Linear models with restrictions occur widely in statistics, see, e.g., Amemiya (1985), Bewley (1986), Chipman and Rao (1964), Dent (1980), Haupt and Oberhofer (2002), Ravikumar, Ray and Savin (2000) and Theil (1971). Throughout this paper, R m n stands for the set of all real m n matrices; M, r(m) and R(M) denote the transpose, the rank and the range (column space) of a matrix M, respectively. For M R m n, M denotes a g-inverse of M, i.e., MM M = M; M + denotes the Moore-Penrose inverse of M, i.e., MM + M = M, M + MM + = M +, (MM + ) = MM + and (M + M) = M + M. Further, let P M = MM +, E M = I m MM + and F M = I n M + M. Let M be as given in (3). The OLSE of the parameter vector β under (3) is defined to be any vector minimizing y Xβ 2 = ( y Xβ ) ( y Xβ ) and is denoted by β, The OLSE of the parameter vector Xβ under (3) is defined to be OLSE(Xβ) = X β. In general, for any l p given matrix K, the OLSE of the vector Kβ under (3) is defined to be OLSE(Kβ) = K β. The BLUE of Xβ under (3) is defined to be a linear estimator Gy such that E(Gy) = Xβ and for any other linear unbiased estimator My of Xβ, Cov(My) Cov(Gy) = MΣM GΣG is nonnegative definite. The two estimators are well known and have been extensively investigated in the literature. Some frequently used OLSEs and BLUEs under the models (3), (4), (6), (7) and (8) are OLSE M (Xβ), OLSE Mc (Xβ), OLSE M (β), OLSE Mc (β), OLSE M1 (β), OLSE M1c (β), BLUE M (Xβ), BLUE Mc (Xβ), BLUE M (β), BLUE Mc (β), BLUE M1 (β), BLUE M1c (β). The OLSEs and BLUEs of Xβ and β under (3) and (4) can be derived through matrix equations, generalized inverses of matrices or the Lagrange multipliers method. A well-known result on the solvability and general solution of the linear matrix equation Ax = b is given below. Lemma 1 The linear matrix equation Ax = b is consistent if and only if AA b = b. In this case, the general solution of Ax = b can be written as x = A b + ( I p A A )v, where the vector v is arbitrary. If A is taken as the Moore-Penrose inverse A +, the general solution of Aβ = b can be written as x = A + b + ( I p A + A )v, (9) where the vector v is arbitrary. If Ax = b is not consistent, then (9) is the general expression of the least-squares solutions of Ax = b. 2

3 and Note that [ X, Σ ][ X, Σ ] + [ X, Σ ] = [ X, Σ ]. Hence if the model in (1) is correct, then E([ X, Σ ][ X, Σ ] + y y) = [ X, Σ ][ X, Σ ] + Xβ Xβ = 0 Cov([ X, Σ ][ X, Σ ] + y y) = σ 2 ([ X, Σ ][ X, Σ ] + I)Σ([ X, Σ ][ X, Σ ] + I) = 0, so that [ X, Σ ][ X, Σ ] + y = y, or equivalently, y [ X, Σ ] holds with probability 1. Definition 1 The linear model in (1) is said to be consistent if y [ X, Σ ] holds with probability 1. The consistency concept for a general linear model was introduced in Rao (1971, 1973), which is a basic assumption in the investigation of the best linear unbiased estimator (BLUE) of Xβ under the general linear model in (1). Definition 2 Suppose M in (3) is consistent. Then two linear estimators L 1 y and L 2 y of Xβ under (3) are said to be equal with probability 1 if (L 1 L 2 )[ X, Σ ] = 0. Lemma 2 Let M be as given in (3), and suppose K 1 y + z 1 and K 2 y + z 2 are two linear unbiased estimators for Xβ under (3). Then K 1 y + z 1 = K 2 y + z 2 holds with probability 1 if and only if K 1 Σ = K 2 Σ holds. Lemma 3 Let M and M c be as given in (3) and (4). Then: (a) The OLSE of Xβ under M can uniquely be written as OLSE M (Xβ) = P X y (10) with E[ OLSE M (Xβ) ] = Xβ and Cov[ OLSE M (Xβ) ] = σ 2 P X ΣP X. (b) The BLUE of Xβ under M can be written as BLUE M (Xβ) = P X Σ y, (11) where P X Σ is a solution of the consistent matrix equation Z[ X, ΣE X ] = [ X, 0 ], where the two partitioned matrices satisfy X X R[ X, ΣE X ] = R[ X, Σ ] and R R. (12) 0 E X Σ The general expression of P X Σ can be written as P X Σ = [ X, 0 ][ X, ΣE X ] + + V(I n [ X, ΣE X ][ X, ΣE X ] + ), (13) where V R n n is arbitrary. Moreover, E[ BLUE M (Xβ) ] = Xβ and Cov[ BLUE M (Xβ) ] = σ 2 P X Σ ΣP X Σ. In particular, BLUE M (Xβ) is unique with probability 1 if (3) is consistent. (c) The OLSE of Xβ under M c can uniquely be written as where X A = XF A. In this case, OLSE Mc (Xβ) = ( I n P XA )XA + b + P XA y, (14) E[ OLSE Mc (Xβ) ] = Xβ and Cov[ OLSE Mc (Xβ) ] = σ 2 P XA ΣP XA. 3

4 (d) The BLUE of Xβ under M c can be written as BLUE Mc (Xβ) = ( I n P XA Σ )XA + b + P XA Σ y (15) with E[ BLUE Mc (Xβ) ] = Xβ and Cov[ BLUE Mc (Xβ) ] = σ 2 P XA ΣΣP X A Σ, where P X A Σ is a solution to the consistent equation Z 1 [ X A, ΣE XA ] = [ X A, 0 ], and the two block matrices satisfy [ X R[ X A, ΣE XA ] = R[ X A, Σ ] and R A X R A 0 F XA Σ The general expression of P XA Σ can be written as ]. (16) P XA Σ = [ X A, 0 ][ X A, ΣF XA ] + + V 1 (I n [ X A, ΣF XA ][ X A, ΣF XA ] + ), (17) X Σ where V 1 R n n is arbitrary. P XA Σ is unique if and only if r = r(a) + n; A 0 BLUE Mc (Xβ) is unique with probability 1 if and only if y X Σ R (18) b A 0 holds with probability 1. Proof The results in (a) and (b) are well known, see, e.g., Puntanen and Styan (1989), Puntanen, Styan and Werner (2000). If (2) is consistent, the general solution of (2) is where the vector γ is arbitrary. Hence, β = A + b + F A γ, (19) OLSE Mc (Xβ) = A + b + OLSE(X A γ) and BLUE Mc (Xβ) = A + b + BLUE(X A γ). (20) Substituting (19) into the equation in (1) yields the following reparameterized model z = X A γ + ε, (21) where z = y XA + b and X A = XF A. The OLSE of γ under (21) can be written as γ = X + A z + F X A u, where the vector u is arbitrary. Substituting this result into (19) gives the OLSE of β under (4) as follows OLSE Mc (β) = A + b + F A X + 1 z + F AF XA u. Hence, OLSE Mc (Xβ) = XA + b + P XA z( I n P XA )XA + b + P XA y. It can be derived from (b) that the BLUE of X A γ under (21) is BLUE M (X A γ) = P XA Σz. Substituting this result into the second equality in (18) gives the BLUE of Xβ under (4) BLUE Mc (Xβ) = XA + b + BLUE Mc (X A γ) = XA + b + P XA Σz, (22) as required for (15). It can be seen from (15) and (22) that BLUE c (Xβ) is unique with probability 1 if and only if r[ z, X A, Σ ] = r[ X A, Σ ] (23) 4

5 holds with probability 1. Applying Lemma 4(b), AA + b = b and elementary block matrix operations to both sides of (23) gives r[ z, X A, Σ ] = r[ y XA + b, XF A, Σ y XA = r + b X Σ r(a) 0 A 0 y X Σ = r r(a), b A 0 X Σ r[ X 1, Σ ] = r[ XF A, Σ ] = r r(a). A 0 y X Σ X Σ Thus, (23) is equivalent to r = r, i.e., (18) holds. b A 0 A 0 For the parametric vector Xβ, the four linear estimators OLSE M (Xβ), BLUE M (Xβ), OLSE Mc (Xβ) and BLUE Mc (Xβ) of Xβ under (3) and (4) are not necessarily equal. It is therefore important to reveal relations among these estimators. In Section 2, we investigate the following six equalities among the four estimators for Xβ under (3) and (4): (a1) OLSE M (Xβ) = BLUE M (Xβ), (b1) OLSE M (Xβ) = OLSE Mc (Xβ), (c1) BLUE M (Xβ) = BLUE Mc (Xβ), (d1) OLSE Mc (Xβ) = BLUE Mc (Xβ), (e1) OLSE M (Xβ) = BLUE Mc (Xβ), (f1) OLSE Mc (Xβ) = BLUE M (Xβ). In Section 3, we investigate the following four equalities for the estimators of an estimable parametric function Kβ under (3) and (4): (a2) OLSE M (Kβ) = BLUE M (Kβ), (b2) OLSE M (Kβ) = OLSE Mc (Kβ), (c2) BLUE M (Kβ) = BLUE Mc (Kβ), (d2) OLSE Mc (Kβ) = BLUE Mc (Kβ). In Section 4, we investigate the following three equalities (a3) OLSE M (β) = OLSE M1 (β), (b3) BLUE M (β) = BLUE M1 (β), (c3) OLSE Mc (β) = OLSE M1c (β). Some useful rank formulas for partitioned matrices due to Masarglia and Styan (1974) are given below. These rank formulas can be used to simplify various matrix equalities involving inverses and Moore-Penrose inverses. Lemma 4 Let A R m n, B R m k C R l n and D R l k. Then: (a) r[ A, B ] = r(a) + r(e A B) = r(b) + r(e B A). 5

6 A (b) r = r(a) + r(cf C A ) = r(c) + r(af C ). A B (c) r = r(b) + r(c) + r(e C 0 B AF C ). (d) r[ A, B ] = r(a) E A B = 0 R(B) R(A). A (e) r = r(a) CF C A = 0 R(C ) R(A ). (f) If R(A) R(B) = {0}, then (E A B) + (E A B) = B + B. (g) If R(A ) R(C ) = {0}, then (CF A )(CF A ) + = CC +. (h) if R(B) R(A) and R(C ) R(A ), then CA B is unique, CA B = CA + B, and A B r = r(a) + r( D CA + B ). C D 2 Relations between the four estimators of Xβ Relations between OLSE M (Xβ) and BLUE M (Xβ) under (3), especially the equality OLSE M (Xβ) = BLUE M (Xβ) have been extensively investigated in the literature, see, e.g., Baltagi (1989), Puntanan and Styan (1989), Puntanan, Styan and Werner (2000), Qian and Schmidt (2003), and Tian and Wiens (2006) among others. A well-known result on the equality OLSE M (Xβ) = BLUE M (Xβ) is summarized below. Theorem 1 Then the following statements are equivalent: (a) BLUE M (Xβ) = OLSE M (Xβ) holds with probability 1. (b) P X ΣE X = 0, i.e., R(ΣX) R(X). (c) P X Σ = ΣP X. Proof Note that both OLSE M (Xβ) and BLUE M (Xβ) are unbiased for Xβ. Then by Lemma 2, OLSE M (Xβ) = BLUE M (Xβ) holds with probability 1 if and only if ( P X Σ P X )Σ = 0. From (13), we obtain P X Σ Σ = [ X, 0 ][ X, ΣE X ] + Σ. (24) A It is easy to verify that any matrix [ A, B ] has a g-inverse [ A, B ] = + A + B(E A B) + (E A EB) +. Applying this to [ X, ΣE X ] gives [ X, ΣE X ] X = + X + ΣE X (E X ΣE X ) + (E X ΣE X ) +. Hence, we obtain by Lemma 4(h) and (24) the following result P X Σ Σ = [ X, 0 ][ X, ΣE X ] + Σ = [ X, 0 ][ X, ΣE X ] Σ = P X Σ P X Σ(E X ΣE X ) + Σ. (25) In this case, so that (24) is equivalent to Simplify (26) by the following equivalence P X Σ(E X ΣE X ) + Σ = 0. (26) MN + = 0 MN + N = 0 MN N = 0 MN = 0 (27) leads to the equivalence of (a), (b) and (c). 6

7 Theorem 2 Let M and M c be as given in (3) and (4). Then: (a) OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if X R Σ X R X. (28) 0 A (b) If R(X ) R(A ) = {0}, then OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1. (c) Under the condition R(X) R(Σ), OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if R(X ) R(A ) = {0}. (d) Under the condition r(x) = p. Then OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if AX + Σ = 0. (e) Under the conditions R(X) R(Σ) and r(x) = p, OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if A = 0. Proof Note that both OLSE M (Xβ) and OLSE Mc (Xβ) are unbiased for Xβ. Then by Lemma 2, OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if P X Σ = P XA Σ. (29) To simplify this equality, we first find the rank of P X Σ P XA Σ. It is easy to see P XA P X = P XA. Hence, we find by Lemma 4(a) and (b) that r( P X Σ P XA Σ ) = r( P X Σ P XA P X Σ ) = r[ P X Σ, X A ] r(x A ) = r[ P X Σ, XF A ] r(xf A ) [ X = r X X Σ X r X A 0 A Hence, OLSE M (Xβ) = OLSE Mc (Xβ) holds with probability 1 if and only if X r X X Σ X = r X, (30) A 0 A which is equivalent to (28) by Lemma 4(d). Results (b) (e) are also derived from (30). Relations between BLUE M (Xβ) and BLUE Mc (Xβ) were also studied by several authors, see e.g., Baksalary and Kala (1979), Hallum, Lewis and Boullion (1973), Mathew (1983), and Yang, Wen, Cui and Sun (1987), and there are some disputes among these papers. Our result for the equality between BLUE M (Xβ) and BLUE Mc (Xβ) is given below. ]. Theorem 3 (a) BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if X Σ X r = r + r[ X, Σ ] r(x). A 0 A (b) Under the condition R(X) R(Σ) = {0}, BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1. (c) Under the condition R(X ) R(A ) = {0}, BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1. (d) Under the condition R(X) R(Σ), BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if R(X ) R(A ) = {0}. 7

8 Proof Because both BLUE M (Xβ) and BLUE Mc (Xβ) are unbiased for Xβ, it follows from Lemma 2 that BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if Note from (13) that the product P XA ΣΣ is P X Σ Σ = P XA ΣΣ. P XA ΣΣ = [ X A, 0 ][ X A, ΣE XA ] + Σ. (31) From (24), (25), (8), (12), Lemma 4(h) and elementary block matrix operations, we find that r( P X Σ Σ P XA ΣΣ ) = r{ [ X, 0 ][ X, ΣE X ] + Σ [ X A, 0 ][ X A, ΣE XA ] + Σ } = r X ΣE X 0 0 Σ 0 0 X A ΣE XA Σ r[ X, ΣE X ] r[ X A, ΣE XA ] X 0 X A 0 0 = r 0 0 X A 0 Σ 0 ΣE X 0 ΣE XA 0 r[ X, Σ ] r[ X A, Σ ] X = r[ X A, Σ ] + r[ ΣE X, ΣE XA ] + r(x) r[ X, Σ ] r[ X A, Σ ] = r(σe XA ) + r(x) r[ X, Σ ] = r[ X A Σ ] r(x A ) + r(x) r[ X, Σ ] X Σ X = r r r[ X, Σ ] + r(x). A 0 A Letting this equality be zero gives (a). Results (b), (c) and (d) follow from (a). Mathew (1983) showed that BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if R(Σ) R(X) R[ X(I n A + A) ]. It can be shown by Lemma 4(a) and (b) that this result is equivalent to Theorem 3(a). Theorem 4 Let M and M c be as given in (3) and (4). Then: (a) OLSE Mc (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if (XF A )(XF A ) + Σ = Σ(XF A )(XF A ) +, or equivalently, ΣX X r 0 A X = r + r(a). A A 0 (b) Under the condition R(X ) R(A ) = {0}, OLSE Mc (Xβ) = BLUE Mc (Xβ) holds with probability 1 f and only if OLSE M (Xβ) = BLUE M (Xβ). Proof The proof is left for the reader. Theorem 5 Let M and M c be as given in (3) and (4). Then: ΣX X (a) OLSE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if R R. 0 A In this case, OLSE M (Xβ) = OLSE Mc (Xβ) = BLUE Mc (Xβ) holds with probability 1. (b) Under the condition R(X ) R(A ) = {0}, OLSE M (Xβ) = BLUE Mc (Xβ) holds if and only if OLSE M (Xβ) = BLUE M (Xβ) holds with probability 1. In this case, holds with probability 1. OLSE M (Xβ) = OLSE Mc (Xβ) = BLUE M (Xβ) = BLUE Mc (Xβ) 8

9 Proof Since both OLSE M (Xβ) and BLUE Mc (Xβ) are unbiased for Xβ, it follows from Lemma 2 that OLSE M (Xβ) = BLUE Mc (Xβ) holds with probability 1 if and only if P X Σ = P XA ΣΣ. From (31), Lemma 4(a), (b) and (h), (15) and elementary block matrix operations, we find that r( P X Σ P XA ΣΣ ) = r( P X Σ [ X A, 0 ][ X A, ΣE XA ] + Σ ) XA ΣE = r XA Σ r[ X X A 0 P X Σ A, ΣE XA ] XA 0 Σ = r r[ X 0 P X ΣE XA 0 A, Σ ] = r(e XA ΣP X ) = r[ X A, ΣX ] r(x A ) = r[ XF A, ΣX ] r(xf A ) X ΣX X = r r. A 0 A Letting this equality be zero yields the desired results. Theorem 6 Let M and M c be as given in (3) and (4). Then OLSE Mc (Xβ) = BLUE M (Xβ) holds with probability 1 if and only if X Σ X ΣX X r = r + r[ X, Σ ] r(x) and r = r(x) + r(a). A 0 A A 0 In this case, OLSE M (Xβ) = BLUE M (Xβ) = BLUE Mc (Xβ) holds with probability 1. Proof The proof is left for the reader. 3 Relations between estimators of parametric functions Let K be an l p known matrix. Then the product Kβ is called a parametric function of β under (3). When K = I p, Kβ = β is the unknown parameter vector under (3). The parametric function Kβ is said to be estimable under (3) if there exists an l n matrix L so that E(Ly) = Kβ holds. It is well known that Kβ is estimable under (3) if and only if R(K ) R(X ), see Alalouf and Styan (1979). Since there is a linear restriction for β under (4), the estimability of Kβ under (4) is defined be below. Definition 3 Let K be an l p known matrix. The vector Kβ is said to be estimable under M c in (4) if there exist an l n constant matrix L and an l 1 constant vector c such that E(Ly + c) = Kβ. Note E(Ly + c) = LXβ + c. Hence E(Ly + c) = Kβ is equivalent to LXβ + c = Kβ. However, we cannot simply say that LX = K and c = 0, because β is not free but subject to the linear equation Aβ = b. A well-known result on the estimability of Kβ under (4) asserts that Kβ is estimable under (4) if and only if R(K ) R[ X, A ]. (32) It can be seen from (32) that if Kβ is estimable under (3), then Kβ is estimable under (4). Let M be as given in (3). If K = K 1 K 2, then OLSE M (Kβ) = K 1 OLSE M (K 2 β), (33) 9

10 see Groß, Trenkler and Werner (2001). Suppose Kβ is estimable under (3), i.e., K = L 1 X for some L 1. Then the BLUE of Kβ is a linear estimator Ly such that E(Ly) = Kβ and for any other linear unbiased estimator My of Kβ, Cov(My) Cov(Ly) = MΣM LΣL is nonnegative. It is easy to verify that if Kβ is estimable under (3), then BLUE M (Kβ) = LBLUE M (Xβ), (34) see, e.g., Groß, Trenkler and Werner (2001). Results (33) and (34) imply that the OLSEs and BLUEs of the estimable parametric function Kβ under (3) and (4) can be derived from the OLSEs and BLUEs of Xβ under (3) and (4). Suppose Kβ is estimable under (4), i.e., there exists a K = [ L 1, L 2 ] such that L = L 1 X+L 2 A. From (33) and (34), we see OLSE Mc (Kβ) = L 1 OLSE Mc (Xβ) + L 2 b, BLUE Mc (Kβ) = L 1 BLUE Mc (Xβ) + L 2 b. (35) From these expressions and Lemma 3, we obtain the following two consequences. Theorem 7 Let M be as given in (3) and suppose Kβ is estimable under M, i.e., K can be written as K = LX. Then: (a) OLSE M (Kβ) = LP X y = KX + y with E[ OLSE M (Kβ) ] = Kβ and Cov[ OLSE M (Kβ) ] = σ 2 KX + Σ(KX + ) = σ 2 (LP X )Σ(LP X ). (b) BLUE M (Kβ) = Sy, where S is the solution of the consistent equation S[ X, VE X ] = [ K, 0 ]. Moreover, BLUE M (Kβ) can be written as BLUE M (Kβ) = LBLUE M (Xβ) = LP X Σ y with E[ BLUE M (Kβ) ] = Kβ and Cov[ BLUE M (Kβ) ] = σ 2 (LP X Σ )Σ(LP X Σ ). Theorem 8 Let M c be as given in (4) and suppose Kβ is estimable under (4), i.e., K can be written as K = L 1 X + L 2 A. (a) The OLSE of Kβ under M c can be written as OLSE Mc (Kβ) = L 1 ( I n P XA )XA + b + L 1 P XA y + L 2 b, where X A = XF A. In this case, E[ OLSE Mc (Kβ) ] = Kβ and Cov[ OLSE Mc (Kβ) ] = σ 2 L 1 P XA ΣP XA L 1. (b) The BLUE of Kβ under M c can be written as BLUE Mc (Kβ) = L 1 ( I n P XA Σ )XA + b + L 1 P XA Σ y + L 2 b with E[ BLUE Mc (Kβ) ] = Kβ and Cov[ BLUE Mc (Kβ) ] = σ 2 (L 1 P XA Σ)Σ(L 1 P XA Σ). Concerning the relations among the four estimators for Kβ in Theorems 7 and 8. We have the following several results. Theorem 9 Let M be as given in (3), and suppose Kβ is estimable under M, i.e., K can be written as K = LX. Then OLSE M (Kβ) = BLUE M (Kβ) holds with probability 1 if and only if R[(KX + Σ) ] R(X). Proof Since both OLSE M (Kβ) and BLUE M (Kβ) are unbiased for Kβ under M, it follows from Lemma 2, Theorem 7(a) and (b) that OLSE M (Kβ) = BLUE M (Kβ) holds with probability 1 if and only if LP X Σ = LP X Σ Σ. From (25), we see LP X Σ LP X Σ Σ = LP X Σ(E X ΣE X ) + Σ. Hence, LP X Σ = LP X Σ Σ is equivalent to LP X Σ(E X ΣE X ) + Σ = 0. It is easy to verify by (27) that this equality is equivalent to LP X ΣE X = 0, which can also be written as KX + Σ = KX + ΣP X. 10

11 Theorem 10 Let M and M c be as given in (3) and (4), and suppose Kβ is estimable under M, i.e., K = LX. Then: (a) OLSE M (Kβ) = OLSE Mc (Kβ) holds with probability 1 if and only if KX + Σ = K(XF A ) + Σ. (b) Under the condition R(X ) R(A ) = {0}, OLSE M (Kβ) = OLSE Mc (Kβ) holds with probability 1. (c) Under the condition [ R(X) ] [ R(Σ), ] OLSE M (Kβ) = OLSE Mc (Kβ) holds with probability 1 K X if and only if R R X. 0 A (d) Under the conditions R(X) R(Σ) and r(x) = p, OLSE M (Kβ) = OLSE Mc (Kβ) holds with probability 1 if and only if A(X X) + K = 0. Proof Since both OLSE M (Kβ) and OLSE Mc (Kβ) are unbiased for Kβ, it follows from Lemma 2, Theorems 7(a) and 8(a) that OLSE M (Kβ) = OLSE Mc (Kβ) if and only if LP X Σ = LP XA Σ, that is, KX + Σ = KE A (XE A ) + Σ. If R(X ) R(A ) = {0}, then (XE A )(XE A ) + = XX +. Hence, (b) follows. The proofs of (c) and (d) are left for the reader. Theorem 11 Let M and M c be as given in (3) and (4), and suppose Kβ is estimable under M, i.e., K = LX. Then: (a) BLUE M (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if r 0 X K X Σ 0 X = r + r[ X, Σ ]. A A 0 0 (b) Under the condition R(X) R(Σ), BLUE M (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if K X R R Σ + X. 0 A (c) Under the conditions that Σ is positive definite and r(x) = p, BLUE M (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if A(X Σ 1 X) 1 K = 0. Proof It can be seen from Lemma 2, Theorems 7(b) and 8(b) that BLUE M (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if LP X Σ Σ = LP XA ΣΣ, this is, L[ X, 0 ][ X, ΣE X ] + Σ L[ X A, 0 ][ X A, ΣE XA ] + Σ = 0. Applying Lemma 4(h) to the left-hand side gives r( L[ X, 0 ][ X, ΣE X ] + Σ L[ X A, 0 ][ X A, ΣE XA ] + Σ ) = r 0 X K [ X Σ 0 X r A A 0 0 ] r[ X, Σ ]. The verification of the rank equality is left for the reader. Theorem 12 Let M c be as given in (4), and suppose Kβ is estimable under M c, i.e., K = L 1 X+L 2 A. Then OLSE Mc (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if KX + A Σ = KX + A ΣP X A. 11

12 Proof Since both OLSE Mc (Kβ) and BLUE Mc (Kβ) are unbiased for Kβ, it follows from Lemma 2, Theorem 8(a) and (b) that OLSE Mc (Kβ) = BLUE Mc (Kβ) holds with probability 1 if and only if LP XA Σ = LP XA ΣΣ, i.e., From (25) LP XA Σ L[ X A, 0 ][ X A, ΣE XA ] + Σ = 0. LP XA Σ LP XA ΣΣ = LP XA Σ(E XA ΣE XA ) + Σ. Hence, LP XA Σ = LP XA,ΣΣ is equivalent to LP XA Σ(E XA ΣE XA ) + Σ = 0. It is easy to verify by (27) that this equality is equivalent to LP XA ΣE XA = 0, which can also be written as KX + A Σ = KX + A ΣP X A. 4 Relations between estimators for the nonsingular linear model and its sub-sample model We have seen that the model (7) is the first part of the model (3). In many situations, a general linear model is the combination of a set of sub-sample models, for example, the fix-effect model for panel data in econometrics, see, Gourieroux and Monfort (1980), Qian and Schmidt (2003). If r(x) = r(x (1) ) = p and r(σ) = n, (3), (4), (7) and (8) are said to be nonsingular models. In such cases, X + X = I p, and β is estimable under (3), (4), (7) and (8). From Lemma 3, we obtain the OLSEs and BLUEs of β under (3), (4), (7) and (8) as follows. Lemma 5 Let M and M c be as given in (3) and (4), and suppose r(x) = p and r(σ) = n. Then: (a) The OLSE of β under M can be written as OLSE M (β) = X + y with E[ OLSE M (β) ] = β and Cov[ OLSE M (β) ] = σ 2 X + Σ(X + ). (b) The BLUE of β under M can be written as BLUE M (β) = (X Σ 1 X) 1 X Σ 1 y with E[ BLUE M (β) ] = β and Cov[ BLUE M (β) ] = σ 2 (X Σ 1 X) 1. (c) The OLSE of β under M c can be written as OLSE Mc (β) = ( I p X + A X )A+ b + X + A y, where X A = XF A. In this case, E[ OLSE Mc (β) ] = β and Cov[ OLSE Mc (β) ] = σ 2 X + A Σ(X+ A ). Lemma 6 Let M 1 and M 1c be as given in (7) and (8), and suppose r(x (1) ) = p and r(σ) = n. Then: (a) The OLSE of β under M 1 can be written as OLSE M1 (β) = X + (1) y 1 with E[ OLSE M1 (β) ] = β and Cov[ OLSE M1 (β) ] = σ 2 X + (1) Σ 11(X + (1) ). (b) The BLUE of β under M 1 can be written as BLUE M1 (β) = (X (1) Σ 1 11 X (1)) 1 X (1) Σ 1 11 y 1 with E[ BLUE M1 (β) ] = β and Cov[ BLUE M1 (β) ] = σ 2 (X (1) Σ 1 11 X (1)) 1. (c) The OLSE of β under M 1c can be written as OLSE M1c (β) = [ I p1 (X (1) ) + A X (1) ]A + b + (X (1) ) + A y 1, where (X (1) ) A = X (1) F A. In this case, E[ OLSE M1c (β) ] = β and Cov[ OLSE M1c (β) ] = σ 2 (X (1) ) + A Σ 11[(X (1) ) + A ]. The following results are derived from Lemmas 6 and 7. 12

13 Theorem 13 Let M and M c be as given in (3) and (4), and suppose r(x (1) ) = p and r(σ) = n. Then OLSE M (β) = OLSE M1 (β) holds with probability 1 if and only if X (2) = 0. Proof Note that both BLUE M (β) and OLSE M1 (β) are unbiased for β. Also note that y 1 = [ I n1, 0 ]y and Σ is nonsingular. Hence we find from Lemmas 4, 6(a) and 7(a) that OLSE M (β) = OLSE M1 (β) holds with probability 1 if and only if X + = [ X + (1), 0 ], which is equivalent to X = X(1). 0 Theorem 14 Let M and M c be as given in (3) and (4), and suppose r(x (1) ) = p and r(σ) = n. Then the following statements are equivalent: (a) BLUE M (β) = BLUE M1 (β) holds with probability 1. (b) X (2) = Σ 21 Σ 1 11 X (1). Σ11 X (c) r (1) = r(σ Σ 21 X 11 ). (2) Proof Note that both OLSE M (β) and BLUE M1 (β) are unbiased for β. Then we see from Lemma 2, Lemmas 6(b) and 7(b) that BLUE M (β) = BLUE M1 (β) holds with probability 1 if and only if This is equivalent to (X Σ 1 X) 1 X = [ (X (1) Σ 1 11 X (1)) 1 X (1) Σ 1 11, 0 ]Σ. (36) X = X Σ 1 X(X (1) Σ 1 11 X (1)) 1 [ X (1), X (1) Σ 1 11 Σ 12 ]. It can be found from Lemma 4(h) that ( ) r X X Σ 1 X(X (1) Σ 1 11 X (1)) 1 [ X (1), X (1) Σ 1 11 Σ 12 ] = r[ X Σ 1 X X (1) Σ 1 11 X (1), X (2) X (1) Σ 1 11 Σ 12 ]. Hence (36) is equivalent to (c). The equivalence of (c) and (d) follows from Lemma 4(h). The verification of the following theorem is left for the reader. Theorem 15 Let M c and M 1c be as given in (4) and (8), and suppose r(x (1) ) = p and r(σ) = n. Then OLSE Mc (β) = OLSE M1c (β) holds with probability 1 if and only if R(X (2) ) R(A ). Remarks. Many problems on relations between estimators for the general linear model can reasonably be investigated. For example, assume that the restricted linear model M c in (3) incorrectly specifies the dispersion matrix Σ as σ 2 Σ 0, where Σ 0 is a given nonnegative definite matrix and σ 2 is a positive parameter (possibly unknown). Then consider relations between the BLUEs of the original model M c and the misspecified model {y, Xβ Aβ = b, σ 2 Σ 0 }. Some results on relations between the BLUEs of two general linear models can be found in Mathew (1983), and Mitra and Moore (1973). Through the matrix rank method, we can also derive some necessary and sufficient conditions for the BLUEs of the original model and the misspecified model to equal with probability 1. This work will be presented in another paper. In addition to the explicit restrictions in (2), the parameter vector β may be subject to some implicit restrictions, represented by adding-up conditions on the endogenous variables By = c, where both the q n matrix B and q 1 vector c are fixed and known. Note that y is a random variable. Hence By = c is equivalent to E(By) = c and Cov(By) = 0 with probability 1, i.e., BXβ = c and BΣ = 0. (37) 13

14 In this case, (3), (4) and (37) can be written as M r = {y, Xβ Aβ = b, BXβ = c, BΣ = 0, Σ}. This model is called the fully restricted linear model, see Haupt and Oberhofer (2002). It may also be of interest to investigate various equalities for the estimators of Xβ in this model. References I.S. Alalouf and G.P.H. Styan (1979). Characterizations of estimability in the general linear model. Ann. Statist. 7, T. Amemiya (1985). Advanced Econometrics, Basil Blackwell, Oxford. J.K. Baksalary and R. Kala (1979). Best linear unbiased estimation in the restricted general linear model. Math. Operationsforsch. Statist. Ser. Statist. 10, J.K. Baksalary, S. Puntanen and G.P.H. Styan (1990). A property of the dispersion matrix of the best linear unbiased estimator in the general Gauss-Markov model. Sankhyā Ser. A 52, B.H. Baltagi (1989). Applications of a necessary and sufficient condition for OLS to be BLUE. Statist. & Probab. Letters 8, R. Bewley (1986). Allocation Models: Specifications, Estimation, and Application. Ballinger Publishing Company. P. Bhimasankaram and R. SahaRay (1997). On a partitioned linear model and some associated reduced models. Linear Algebra Appl. 264, J.S. Chipman, M.M. Rao (1964). The treatment of linear restrictions in regression analysis. Econometrica 32, W.T. Dent (1980). On restricted estimation in linear models. J. Econometrics 12, C. Gourieroux and A. Monfort (1980). Sufficient linear structures: econometric applications. Econometrica 48, J. Groß and S. Puntanen (2000). Estimation under a general partitioned linear model. Linear Algebra Appl. 321, J. Groß, T. Trenkler and H.J. Werner (2001). The equality of linear transformations of the ordinary least squares estimator and the best linear unbiased estimator. Sankhyā 63, C.R. Hallum, T.O. Lewis and T.L. Boullion (1973). Estimation in the restricted general linear model with a positive semidefinite covariance matrix. Comm. Statist. 1, H. Haupt and W. Oberhofer (2002). Fully restricted linear regression: A pedagogical note. Economics Bulletin 3, 1 7. G. Marsaglia and G.P.H. Styan (1974). Equalities and inequalities for ranks of matrices. Linear and Multilinear Algebra 2, T. Mathew (1983). A note on best linear unbiased estimation in the restricted general linear model. Math. Operationsforsch. Statist. Ser. Statist. 14, 3 6. S.K. Mitra and B.J. Moore (1973). Sankhyā Ser. A 35, Gauss-Markov estimation with an incorrect dispersion matrix. M. Nurhonen and S. Puntanen (1992). A properties a partitioned generalized regression. Commu. Statist. A-Theory Meth. 21, S. Puntanen (1996). Some matrix results related to a partitioned singular linear model. Commu. Statist.- Theory Meth. 25, S. Puntanen and A.J. Scott (1996). Some further remarks on the singular linear models. Linear Algebra Appl. 237/238, S. Puntanen and G.P.H. Styan (1989). The equality of the ordinary least squares estimator and the best linear unbiased estimator. With comments by O. Kempthorne, S.R. Searle, and a reply by the authors. Amer. Statist. 43,

15 S. Puntanen, G.P.H. Styan and H.J. Werner (2000). Two matrix-based proofs that the linear estimator Gy is the best linear unbiased estimator. J. Statist. Plan. Infer. 88, H. Qian and P. Schmidt (2003). Partial GLS regression. Economic Letters 79, H. Qian, Y. Tian (2006). Partially superfluous observations. Econometric Theory 22(2006), C.R. Rao (1971). Unified theory of linear estimation. Sankhyā Ser. A 33, C.R. Rao (1973). Representations of best linear unbiased estimators in the Gauss-Markoff model with a singular dispersion matrix. J. Multivariate Anal. 3, C.R. Rao and S.K. Mitra (1971). Generalized Inverse of Matrices and Its Applications. Wiley, New York. B. Ravikumar, S. Ray and N.E. Savin (2000). Robust wald tests in SUR systems with adding up restrictions. Econometrica 68, S.R. Searle (1971). Linear Models. Wiley, New York. H. Theil (1971). Principles of Econometrics. Wiley, New York. Y. Tian and D.P. Wiens (2006). On equality and proportionality of ordinary least-squares, weighted least-squares and best linear unbiased estimators in the general linear model. Statist. Prob. Lett., 76, H.J. Werner and C. Yapar (1995). More on partitioned possibly restricted linear regression. In Proceedings of the 5th Tartu Conference on Multivariate Statistics (E.-M. Tiit et al. eds.), W.-L. Yang, H.-J. Cui and G.-W. Sun (1987). On best linear unbiased estimation in the restricted general linear model. Statistics 18, B.-X. Zhang, B.-S. Liu and C.-Y. Lu (2004). A study of the equivalence of the BLUEs between a partitioned singular Linear model and its reduced singular linear models. Acta Math. Sinica 20,