Nonnegative-Definite Covariance Structures for. which the Least Squares Estimator is the Best. Linear Unbiased Estimator

Transcription

1 Applied Mathematical Scieces, ol. 1, 2007, o. 24, Noegative-Defiite Covariace Structures for which the Least Squares Estimator is the Best Liear Ubiased Estimator Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. Departmet of Statistical Sciece, Baylor Uiversity Waco TX , USA Abstract For the Gauss-Markov model with E(y) = X β ad ar(y) =, we establish a ew explicit characterizatio of the geeral oegative-defiite covariace structure such that the best liear ubiased estimator ad least squares estimator of X β are idetical. The proof of our represetatio is brief, requirig oly basic properties of real matrices. Furthermore, our characterizatio does ot suffer from the idetermiacies of other represetatios. Mathematics Subject Classificatio: 62J05

2 1158 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. 1 Itroductio We cosider the geeral Gauss-Markov model y = X β + e (1) i which y is a 1 vector of observatios, X is a p kow, fixed, o-ull model (desig) matrix of rak r p, β is a p 1 vector of ukow model parameters, ad e is a 1 vector of radom perturbatios such that E(e) = 0 ad ar(e) =, where is a kow o-ull oegative-defiite (.d.) matrix. We assume that the model (1) is cosistet. That is, we assume y R(X:), the rage space of the partitioed matrix X:. Give the model matrix X, the ordiary least squares estimator of X β ca be expressed as ˆ + Xβ = X( XX ) Xy, (2) LS where X is the traspose of X ad ( XX ) + is the Moore-Perose iverse of ( XX ). The best liear ubiased estimator of X β ca be expressed as ˆ Xβ = X( XT X) XT y, where T = + XUX ad U is ay symmetric matrix such that R(T) = R(X:). Pukelsheim (1974) has give a alterative represetatio of X ˆ β which is ˆ X β = y ( I XX )[( I XX ) ( I XX )] ( I XX ) y. (3) I this paper we give a cocise proof of a explicit characterizatio of the geeral.d. error covariace structure for which X ˆ β = X ˆ β. We defie such covariace matrices to be -LS estimatio-equivalet (e.e.) covariace structures ad use d ad pd to deote the sets of.d. ad positive-defiite (p.d.) -LS e.e. covariace structures, respectively. Thus, more specifically, i this paper we give a cocise LS

3 Noegative-Defiite Covariace Structures 1159 costructive derivatio of a explicit characterizatio of d. Ulike other explicit characterizatios, ours precisely defies certai arbitrary matrices such that covariace matrices cotaied i d are, i fact,.d. Thus, our characterizatio of d is more specific ad complete tha previous -LS e.e. covariace structure represetatios. We also show that ie well-kow implicit characterizatios of d are corollaries of our characterizatio theorem. I our paper, we deote the vector space of all m matrices over the real field R by R m. Let R represet the coe of all symmetric.d. matrices i R. Let R > deote the iterior of R, which is the set of all symmetric p.d. matrices i R. We use I R to represet the set of all symmetric idempotet matrices i R >. There is a rich literature o implicit ad explicit characterizatio of the -LS e.e. covariace structures. See Putae ad Stya [10], Alalouf ad Stya [1], ad Siha ad Drygas [14] for a review of may of these results. At least six differet explicit characterizatios of the geeral -LS e.e. covariace structure have previously bee proposed two for the.d. case ad four for the p.d. case. These characterizatios are essetially equivalet. However, the methods ad the assumptios used to derive the explicit characterizatios are quite differet. Rao [11] first derived a explicit characterizatio of d. Rao s [11] derivatio of employs the idea that a ubiased estimator of X β has a miimum covariace matrix i the.d. Löwer orderig sese amog all ubiased liear estimators if ad oly if its cross-covariace matrix with all liear fuctios havig expectatio zero is itself zero. Rao s [11] characterizatio is R 1 d = { R : = α + + } I XAX ZBZ, (4)

4 1160 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. where Z is a real matrix such that where R(Z) = R(X) = N(X) where R(X) represets the complemetary space orthogoal to R(X) ad N(X) represets the ull space of X. The scalar α ad matrices A ad B are specified oly as a scalar ad two arbitrary symmetric matrices such that R. Zyskid [15] has also derived the characterizatio (1.4). Rao [12] has proposed a secod characterizatio d, which is R 2 d = { R : = Λ 1 + Λ2 } X X Z Z, (5) where Λ 1 ad Λ 2 are arbitrary symmetric matrices ad Z is a matrix of maximum rak such that ZX = 0. Lewis ad Odell [7] have derived a explicit characterizatio of pd that has bee geerally overlooked i the literature. Their represetatio is LO > pd = { R : = α I + XAX + ZBZ }, (6) where rak(x) = p, Z R ( p) with rak(z) = ( p) ad ZX = 0, ad α, A, ad B are restricted such that R >. Norlè [8] has derived aother explicit characterizatio of pd. His represetatio, which is idetical to represetatio (5) but is restricted to the p.d. case, is N pd = { R : 1 2 } > = XΛ X + X Λ X, (7) where rak( X ) = rak( X ) ad Λ i, i = 1, 2, are arbitrary matrices which may vary subject to the costrait that for pd but gives o derivatio. R >. Chikuse [4] has also give the characterizatio (7)

5 Noegative-Defiite Covariace Structures 1161 Graybill [5] has derived a ofte overlooked explicit characterizatio of pd. Graybill s represetatio is G > pd = { R : = α I + XAX + ( I -XX ) B( I -XX )}, (8) where α, A, ad B are restricted such that R > >, α I + XAX R, ad X is a arbitrary but fixed coditioal or {1} iverse of X. Kempthore [6] has also derived a explicit characterizatio of pd uder the assumptio that rak(x) = p. Kempthore s represetatio is { K > 1 1 pd = R : = α I + X( XX ) XAX ( XX ) X, (9) + I-X XX X B I-X XX X 1 1 ( ( ) ) ( ( ) ) where α is a positive scalar ad A ad B are matrices such that R >. Kempthore s derivatio is brief, simple, ad, thus, very appealig, but is restricted to the p.d. case. } We have orgaized the remaider of the paper as follows. I Sectio 2 we give a lemma that is used to derive our mai result. I Sectio 3 we derive our explicit characterizatio for d. We coclude with brief commets i Sectio 4. 2 A Prelimiary Lemma We utilize the followig lemma that gives a represetatio of the geeral.d. solutio to a particular homogeeous matrix equatio. Lemma. For C such that 1 rak( C ) m, a represetatio of the geeral.d. R m solutio to the matrix equatio

6 1162 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. is where U 1, U 2 Proof. R are arbitrary. ( CC ) Z( I -CC ) = 0 (10) Z = CC U CC + ( I-CC ) U ( I -CC ) (11) 1 2 Clearly, Z, as defied i (11), is a.d. solutio to (10). We ow show that expressio (11) is a represetatio of the geeral.d. solutio to (10). Let Z 0 be a arbitrary.d. solutio to equatio (10). Also, ote that there exists a matrix W 0 such that Z = WW, ad let U 1 =U 2 = Z 0 i (11). Usig the fact that ( I - CC ) Z0CC ( CC Z ) 0 I CC =0 we have that Z = CC Z CC + ( I -CC ) Z ( I -CC ) 0 0 = ( I -CC ) W + CC W ( I -CC ) W + CC W = [ WW 0 0 ] = Z Hece, (11) is a represetatio of the geeral.d. solutio to equatio (10). = 3 Mai Result We ow preset a proof of our explicit characterizatio for d i the followig theorem. Oe ca view the theorem as a alterative proof of Rao s [12] explicit characterizatio (5) ad as a extesio of the p.d. characterizatio (7) to the more geeral.d. case. Our proof is more cocise tha those of Rao [11] ad Norlè [8].

7 Noegative-Defiite Covariace Structures 1163 Furthermore, ulike the explicit characterizatios (4) through (9), our represetatio leaves o matrices idetermiate. Theorem. Cosider the Gauss-Markov model (1) where R is o-ull ad X is a model matrix such that 1 rak( X ) p. The, X ˆ β = X ˆ β R p LS if ad oly if, d, where = : = XX + W XX + + ( I -XX ) W ( I -XX ) (12) d { R } such that W 1 { W : W R ad W R( XX + )} ad W 2 R arbitrary ad W 2 { : R ad ( + )} is arbitrary or W 1 R is W W W R I XX. We ote that the restrictios o W 1 ad W 2 give above isure that is o-ull. Proof. Assume X ˆ β ˆ = Xβ LS ad ote that we may re-express X ˆLS β as ˆ + Xβ LS = I ( I- XX ) y, (13) where y R(X:). From (3), (13), ad ad property 1) of the Moore-Perose iverse we have that ˆ ˆ Xβ = Xβ LS ( I -XX ) ( I -XX ) ( I-XX ) ( I-XX ) y= ( I -XX ) y for y R(X:) ( I -XX ) ( I -XX ) ( I -XX ) ( I -XX ) z= ( I -XX ) z for z R ( I -XX ) ( I -XX ) ( I -XX ) ( I-XX ) = ( I -XX ) +

8 1164 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr XX ( I -XX ) ( I -XX ) ( I -XX ) = ( XX ) ( I -XX ) = 0 (14) From the lemma we have that a represetatio of the geeral.d. solutio to (14) is give i (12). Now, assume, d. Albert [2] ad Putae ad Stya [10] have oted the relatioship ˆ ˆ Xβ Xβ LS = XX ( I-XX ) ( I -XX ) ( I-XX ) y, (15) Where y R(X:). For,d we clearly have that the right-had-side of (15) is zero ad, hece, that X ˆ β = X ˆ β. LS I additio, several well-kow implicit characterizatios of d are direct results of the proof of the theorem. We state these i the followig corollary. Corollary. Cosider the Gauss-Markov model (1) where R is o-ull ad X is a model matrix such that 1 rak( X ) p. The, X ˆ β = X ˆ β if ad oly R p LS if d, where i) d = { R + + : XX =XX } ; ii) d = { R : XX=XXXX } ; iii) { } d = R : XX = XX XX ; iv) d { R + + = :( I XX ) XX = 0} ; v) d { R = :( ) 0} vi) d = { R : XX = XX } ; I XX XX = ;

9 Noegative-Defiite Covariace Structures 1165 ad vii) d = { R : } viii) d = { R : } XX = XX XX ; XX =XX XX ; ix) d { R = :( ) ( )} I XX = I XX. Proof. The implicit characterizatios of d umbered (i) through (v) follow directly from the proof of the theorem. Implicit characterizatio (vi) follows from (i) ad (v) ad from Theorem i Campbell ad Meyer [3]. Characterizatios (vii) through (ix) follow immediately from (vi). Oe ca fid these well-kow implicit characterizatios of d i Alalouf ad Stya [1], Putae ad Stya [10], ad Searle [13]. 4 Commets Norlè [8] has stated that his -LS e.e. covariace-structure characterizatio (7) is simpler ad more explicit tha the represetatio (4) derived by Rao [11]. He further states that the structure of the represetatio (7) may further be characterized as a reductio of represetatio (4), where the idetermiacies have bee removed. Our explicit characterizatio d does ot suffer from the idetermiacies i the represetatios (4) through (9), all of which have restrictios o the arbitrary matrices that are ot precisely defied. Also, the derivatio of our explicit characterizatio d, give i (12), is brief ad easily uderstadable if oe kows some basic properties of real matrices.

10 1166 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. Fially, our -LS covariace-structure characterizatio d cotais or is equivalet to the characteriazatios (4) through (9). For istace, cosider Kempthore's [6] -LS e.e. characterizatio K K pd give i (10). Let pd. The, = α I + X XX XAX XX X + I -X XX X B I -X XX X ( ) ( ) ( ( ) ) ( ( ) ) = αi + XX [( αi+ A) αi] XX + ( I-XX ) B( I-XX ) = αi + XX [( αi+ A) αi] XX + ( I-XX ) B( I-XX ) = XX ( αi + A) XX + ( αi -αxx ) + ( I -XX ) B( I -XX ) = XX ( αi + A) XX + ( I -XX )( I -XX )( αi + B)( I -XX ) = XX W XX + ( I-XX ) W ( I -XX ) where W 1 = ( α I+A ), W 2 = ( α I+B ) R. Hece, d ad, therefore, pd K d, where d is defied i (12). Similar argumets ca be made to demostrate that the characterizatios R1 d ad R d 2 are equivalet to ad K pd are cotaied i our explicit characterizatio d ad that d. LO N G pd, pd, pd, Refereces [1] I.S. Alalouf, G.P.H. Stya, Characterizatios of the coditios for the ordiary least squares estimator to be best liear ubiased, Topics i Applied Statistics, (Y.P. Chaubey ad T.D. Dwivedi, eds.). Dept. of Mathematics, Cocordia Uiversity, Motreal, , 1984.

11 Noegative-Defiite Covariace Structures 1167 [2] A. Albert, The Gauss-Markov theorem for regressio models with possibly differet sigular covariaces, SIAM Joural of Applied Mathematics, 24, (1973), [3] S.L. Campbell, C.D. Meyer, Geeralized Iverses of Liear Trasformatios, Dover Publicatios, New York, [4] Y. Chikuse, Represetatios of the covariace matrix for robustess i the Gauss- Markov model, Commuicatios i Statistics - Theory ad Methods, 19, (1981), [5] F.A. Graybill, Theory ad Applicatio of the Liear Model, Duxbury, North Scituate, Massachusetts, [6] O. Kempthore, Commet o the equality of the ordiary least-squares estimator ad best liear ubiased estimator, The America Statisticia, 43, (1989), [7] T.O. Lewis, P.L. Odell, Estimatio i Liear Models, Pretice Hall, Eglewood, New Jersey, [8] U. Norlè, The covariace matrices for which least squares is best liear ubiased, Scadiavia Joural of Statistics, 2, (1975) [9] F. Pukelsheim, Equality of two Es ad ridge type estimates, Commuicatios i Statistics - Theory ad Methods, 6, (1977), [10] S. Putae, G.P.H. Stya, The equality of the ordiary least squares estimator ad the best liear ubiased estimator, The America Statisticia, 43, (1989), [11] C.R. Rao, Least squares theory usig a estimated dispersio matrix ad its applicatios to measuremet of sigals, Proceedigs of the Fifth Berkeley Symposium o Mathematical Statistics ad Probability, (ol. 1), eds. L.M. LeCam ad J. Neyma, Berkeley: Uiversity of Califoria Press, , [12] C.R. Rao, A ote o a previous lemma i the theory of least squares ad some further results, Sakhy -,a, Ser. A, 30, (1968), [13] S.R. Searle, Extedig some results ad proofs for the sigular liear model, Liear Algebra ad Its Applicatios, 210, (1994),

12 1168 Dea M. Youg, James D. Stamey ad Joh W. Seama, Jr. [14] B.K. Siha ad H. Drygas, Robustess i liear models, Proceedigs of the First Tampere Semiar o Liear Models, (1985), [15] G. Zyskid, O caoical forms, oegative covariace matrices ad best ad simple least squares liear estimators i liear models, Aals of Mathematical Statistics, 38, (1967), Received: September 1, 2006