A CLASS OF THE BEST LINEAR UNBIASED ESTIMATORS IN THE GENERAL LINEAR MODEL

Transcription

1 A CLASS OF THE BEST LINEAR UNBIASED ESTIMATORS IN THE GENERAL LINEAR MODEL GABRIELA BEGANU The existence of the best linear unbiased estimators (BLUE) for parametric estimable functions in linear models is treated in a coordinate-free approach, and some relations between the general linear model and certain particular models are derived. A class of the BLUE is constructed using the maximal covariance operator. It is shown that a BLUE exists for each parametric estimable function and is unique in this class. AMS 2000 Subject Classification: 62H12, 47A05. Key words: best linear unbiased estimator, maximal covariance operator, finitedimensional inner product space, orthogonal projection. 1. INTRODUCTION The coordinate-free approach is used in this paper to treat the existence of the best linear unbiased estimators of parametric estimable functions in the general linear model, thus extending and unifying some results previously obtained [7], [8], [10], [16], [17]. This approach, which may also be called the geometric approach, for linear regression models has added the understanding to the problems of estimation and hypothesis testing and it was used at least since linear algebra and the theory of Hilbert spaces have been developed in operators form rather than in matrix form. One of the initial authors who treated in a coordinate-free approach the problem of the existence of BLUE for the expected mean in the general linear model is Kruskal [12]. He gave a necessary and sufficient condition for BLUE to be equal to the ordinary least squares estimators. This condition became the foundation for the alternative forms, as well as their extensions, proved in [1], [2], [3], [4], [5], [6], [9], [11], [15]. The purpose of the paper is to construct a limited class of the best linear unbiased estimators for each linear parametric function using the coordinatefree approach which allows an atractive and simple computational form. The paper is structurated as follows. The general linear model describing the functional relationship disturbed stochastically is presented from the coordinate-free point of view in Section 2. Some relations between the general MATH. REPORTS 9(59), 4 (2007),

2 328 Gabriela Beganu 2 linear model and particular linear models regarding the existence of BLUE are also deduced. In Section 3, a class of the best linear unbiased estimators of parametric estimable functions is constructed. It is shown that a BLUE in this class exists for each parametric function in the linear model and this BLUE is unique in this class. 2. RELATIONS BETWEEN LINEAR MODELS REGARDING THE EXISTENCE OF BLUE Let (E,S) be a measurable space and P = {P θ θ Θ} a family of associated probability measures. The complete structure of P is not necessarily known, but Θ is assumed to be a given parameter space. A random real-valued element y is a transformation y : E K,where (K, (, )) is a finite-dimensional inner product space. The expectation of y is the unique element µ θ Ksuch that (1) E θ (a, y) =(a, µ θ ) while the covariance operator V of y is the unique symmetric and non-negative mapping from K to K such that (2) cov θ ((a, y), (b, y)) = (a, V b) for all a, b V. The expectation (1) and the covariance (2) of y exist with respect to all P θ P, i.e., for all θ Θ. Let us introduce some notation: R(A) is the image of a linear mapping A : L K, or the column space of the matrix A; spana is the linear manifold spanned by an arbitrary set A; A is the orthogonal complement of a linear manifold A in a finite-dimensional inner product space. Let (L, [, ]) and (K, (, )) be two finite-dimensional inner product spaces. Then A : K L is the adjoint of the linear operator A, i.e., (Al, k) =[l, A k] for all l L, k K; N(A) isthenullspaceofa. If A is the matrix corresponding to a linear operator A, thena is its transpose, A is a generalized inverse of A(AA A = A) anda + is the Moore-Penrose inverse uniquely determined by A. Let Ω K. The statistical model M(Ω,V) is the set of all random elements y with E θ y = µ θ Ωandcov θ y = V for all θ Θ. If Ω is a linear subspace of K, thenm(ω,v) is called a linear model. When Ω = span {E θ y = µ θ θ Θ} and V =span{cov θ y = V θ Θ}, the general linear model is denoted M(Ω, V). In the sequel, the general linear regression model will be considered. Then Ω = R(X), the range of a linear mapping X : L K,where(L, [, ]) is a finite-dimensional Euclidean vector space.

3 3 A class of the best linear unbiased estimators 329 A parametric function g(θ) is said to be estimable if and only if there exists an element a Ksuch that (3) E θ (a, y) =(a, µ θ )=g(θ) for all θ Θ. Let us assume that Ω R(V ) andlet [λ, β] be a linear parametric estimable function. It is well known that a BLUE of [λ, β] is(a, y) iffx a = λ and cov θ (a, y) =(a, V a) has to be minimized in the class of all linear unbiased estimators, i.e., a M = {d K X d = λ}. SinceM M = X 1 (0), (a, y) is a linear unbiased estimator of minimum covariance iff (Va,d) = 0 for all d X 1 (0), or, equivalently, iff Va= Xb for some b L. The main question in the theory of linear models is the existence of minimum covariance linear unbiased estimators for parametric functions. One of the classical results, which will be used, is a theorem of Lehmann and Scheffé [14]: for the linear model M(Ω, V), (a, y) isablueofe θ (a, y) for all θ Θ if and only if Va Θ for all V V. Let V 0 be a maximal element of V, i.e., R(V ) R(V 0 ) for all V V. Such an element always exists ([13]). Using the Lehmann-Scheffé theorem, another proof will be given for the result below obtained in [8]. Proposition 1. Let V 0 be the maximal element of V. A BLUE of a parametric estimable function exists in the model M(Ω, V) if and only if (4) V0 1 (Ω) V 1 (Ω) for all V V. Proof. V 0 being the maximal element of V, we deduce that if b R(V ) for all V V then b R(V 0 ). Assume now that (a, y) is a BLUE in the model M(Θ, V). By the Lehmann-Scheffé theorem this condition is equivalent to b = Va Ωfor all V V. Thus, R(V 0 ) Ω. Since Ω R(V 0 ) for all V V, we deduce relation (4). Conversely, if (4) holds, then V 0 a Ω implies Va Ω for all V V, which is the condition for (a, y) tobeablueinthemodelm(ω, V). Proposition 2. If (a, y) is a BLUE of a parametric estimable function in the model M(Ω,V 0 ),then(a, y) is a BLUE in the model M(Ω, V) provided that a BLUE in this model exists. Proof. By the Lehmann-Scheffé theoreminthemodelm(ω,v 0 )wehave V 0 a Ω. Since a BLUE exists in the model M(Ω, V), this and relation (4) yield a V 1 (Ω) for all V V, which means that (a, y) isablueinthe model M(Ω, V).

4 330 Gabriela Beganu 4 Proposition 3. (a, y) is a BLUE of a parametric estimable function in the model M(Ω,V 0 ) if and only if (a, y) is a BLUE in the model M(Ω,W), where (5) W = V 0 + P is a symmetric non-negative definite operator and P is the orthogonal projection on Ω. Proof. (a, y) is a BLUE of a linear parametric function (3) in the model M(Ω,V 0 )if and only if V 0 a Ω. Since P is the orthogonal projection on Ω, we have Pa Ω. These conditions are equivalent to Wa Ω. Remark. The orthogonal projection P = XX + on Ω in (5) can be replaced by every symmetric and non-negative definite operator T such that R(T ) Ω. A choice of T can be XX. Proposition 4. A BLUE of a parametric estimable function (3) exists in the model M(Ω, V) if and only if (6) VW (Ω) Ω for all V V,whereW is a generalized symmetric inverse of W. Proof. From Propositions 1, 2 and 3 we have that the condition (a, y) is a BLUE of a parametric function g(θ) for all θ Θ in the model M(Ω, V), is equivalent to the condition that (a, y) isablueofg(θ) in the model M(Ω,W). This last statement is equivalent to Wa = b Ω. It follows from (5) that (7) R(V ) R(V 0 ) R(W ) for all V V. Hence V = WW V or, by transposing, V = VW W and we can write Va= VW Wa= VW b VW(Ω) for all V V.SinceVa Ω for all V V, relation (6) follows. 3. ACLASSOFBLUE In the sequel, the question of the existence of the best linear unbiased estimators for g(θ), θ Θ, will be treated in the general linear model imposing a restrictive condition. Theorem 1. If (a, y) is a BLUE of E θ (a, y) in the linear model M(Ω, V), then there exists a 1 R(W ) such that (a 1,y) is a BLUE of E θ (a, y) for all θ Θ.

5 5 A class of the best linear unbiased estimators 331 Proof. Let (a, y) beablueofe θ (a, y),θ Θ, such that a / R(W ). Then a Kcan be uniquely written as a = a 1 + a 2,wherea 1 R(W )and a 2 R(W ). Since Ω R(W ) (by (7)), we have R(W ) Ω,whichmeansa 2 Ω. Hence (8) E θ (a, y) =(a, Xβ) =(a 1,Xβ)=E θ (a 1,y) for all θ Θ. Moreover, by the Farkas-Minkowski theorem and the symmetry of the operators V and W, relation (7) implies R(W ) = N(W ) R(V ) = N(V ) for all V V. Therefore, a 2 N(V ) or, equivalently, Va 2 =0forallV V. This result allows one to write (9) cov θ (a, y) =(a 1 + a 2,V(a 1 + a 2 )) = (a 1,Va 1 )=cov θ (a 1,y) for all V V and θ Θ. Relations (8) and (9) show that (a 1,y)witha 1 R(W )isablueof E θ (a, y), which means that a BLUE of g(θ) witha 1 R(W )existswhena BLUE of g(θ) witha Kexists in the model M(Ω, V) for all θ Θ. If there exist a, b Ksuch that X a = X b = λ, then(a, y) and(b, y) are linear unbiased estimators for [λ, β]. Although both are unbiased estimators for the same parametric function, it is not necessarily true that (a, y) =(b, y), hence the linear unbiased estimator is not in general unique. However, if N(X )={0} then X a = X b implies that a b N(X ), so that a = b and the linear unbiased estimator of [λ, β] is unique. Note that this uniqueness is dependent upon the condition R(X) =Ω=K. The problem of the uniqueness of the BLUE (a, y) witha R(W )is solved by the result below Theorem 2. If (a, y) and (b, y) with a, b R(W ) are best linear unbiased estimators of a parametric function g(θ) in the linear model M(Ω, V) for all θ Θ, thena = b. Proof. (a, y) and(b, y) being unbiased estimators of the same parametric function, we have E θ (a, y) =E θ (b, y) for all θ Θ, which means that (a b, Xβ) = 0, and this relation is equivalent to a b Ω. Since (a, y) and(b, y) are minimum variance estimators in the model M(Ω, V), we have cov θ (a, y) =(a, V a) =(a, V b) =cov θ (b, y) for all V θ and θ Θ. The same equations are verified by V 0, the maximal element of V and, by Proposition 3, by W. Hence (10) (a b, W (a b)) = 0,

6 332 Gabriela Beganu 6 which yields W (a b) = 0 because W is a non-negative definite operator. Now, by the Farkas-Minkowski theorem, the relations W (a b) =0andW (a b) Ω imply a b W 1 (0) W 1 (Ω) = R(W ) [W (Ω )] = =[R(W ) W (Ω )] = R(W ). But a, b R(W ), hence a b R(W ). It follows that a b =0. REFERENCES [1] S.F. Arnold, A coordinate-free approach to finding optimal procedures for repeated measures designs. Ann. Statist. 7 (1979), [2] J.K. Baksalary and A.C. van Eijnsbergen, A comparison of two criteria for ordinary least squares estimators to be best linear unbiased estimators. Amer.Statist.42 (1988), [3] G. Beganu, Estimation of regression parameters in a covariance linear model. Stud. Cerc. Mat. 39 (1987), [4] G. Beganu, The existence conditions of the best linear unbiased estimators of the fixed effects. Econom. Comput. Econom. Cybernet. Stud. Res. 36 (2003), [5] G. Beganu, On Gram-Schmidt orthogonalizing process of design matrices in linear models as estimating procedure of covariance components. Rev. Real Acad. Cienc. Ser. A Mat. 99 (2005), [6] G. Beganu, A two-stage estimator of individual regression coefficients in multivariate linear growth curve models. Rev. Acad. Colombiana Cienc. 30 (2006), 1 6. [7] H. Drygas, Estimation and prediction for linear models in general spaces. Math. Operationsforsch. Statist. Ser. Statist. 8 (1975), [8] H. Drygas, Multivariate linear models with missing observations. Ann. Statist. 4 (1976), [9] M.L. Eaton, Gauss-Markov estimation for multivariate linear models: A coordinate-free approach. Ann. Statist. 2 (1974), [10] S. Gnot, W. Klonecki and R. Zmyslony, Uniformly minimum variance unbiased estimation in Euclidean vector spaces. Math. Operationsforsch. Statist. Ser. Statist. 8 (1975), [11] S.I. Haberman, How much do Gauss-Markov and least squares estimates differ? A coordinate-free approach. Ann. Statist. 3 (1975), [12] W. Kruskal, When are Gauss-Markov and least squares estimators identical? A coordinate-free approach. Ann. Math. Statist. 39 (1968), [13] L.R. Lamotte, Admissibility in linear estimation. Ann. Statist. 10 (1982), [14] E. Lehmann and H. Scheffé, Completeness, similar regions and unbiased estimation. Sankhya 10 (1950), [15] S. Puntanen, G.P.H. Styan and Y. Tian, Three rank formulas associated with the covariance matrices of the BLUE and the OLSE in the general linear model. Econometric Theory 21 (2005), [16] J. Seely, Linear spaces and unbiased estimation. Ann. Math. Statist. 41 (1970),

7 7 A class of the best linear unbiased estimators 333 [17] J. Seely, Linear spaces and unbiased estimation. Application to the mixed linear models. Ann. Math. Statist. 41 (1970), [18] G. Zyskind, On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models. Ann. Math. Statist. 38 (1967), Received 8 February 2007 Academy of Economic Studies Department of Mathematics Piaţa Romanǎ nr Bucharest, Romania gabriela beganu@yahoo.com