Pseudo-minimax linear and mixed regression estimation of regression coecients when prior estimates are available

Transcription

1 Statistics & Probability Letters Pseudo-minimax linear and mixed regression estimation of regression coecients when prior estimates are available H. Shalabh a, H. Toutenburg b; a Department of Statistics, Panjab University, India b Institute of Statistics, University of Munich, LMU Munchen, Akademiestrasse 1/I, Munchen 80799, Germany Received June 00; received in revised form November 00 Abstract When prior estimates ofregression coecients along with their standard errors or their variance covariance matrix are available, they can be incorporated into the estimation procedure through minimax linear and mixed regression approaches. It is demonstrated that the mixed regression approach provides more ecient estimators, at least asymptotically, in comparison to the minimax linear approach with respect to the criterion ofvariance covariance matrix. c 003 Published by Elsevier Science B.V. MSC: 6J05; 6J99 Keywords: Linear regression; Mixed model 1. Introduction Recent studies, past experience and pilot investigations often provide some useful information in the form ofestimates ofcoecients in a linear regression model along with their standard errors or their variance covariance matrix. Such a prior information can be incorporated into the estimation procedure in two simple ways based on point and interval estimation ofparameters. One is to express it as a set ofstochastic linear restrictions and to apply the method ofmixed regression estimation. And the other is to form a condence ellipsoid for the regression coecients and to apply the method ofminimax linear estimation. The latter approach generally does not lead to a simple form ofthe estimators and iterative procedures are to be followed; see, e.g., Schipp 1990 for an interesting exposition. However, ifwe take the matrix involved in the quadratic loss function to be ofrank Corresponding author. Tel.: ; fax: address: toutenb@stat.uni-muenchen.de H. Toutenburg /03/$ - see front matter c 003 Published by Elsevier Science B.V. doi: /s

2 36 H. Shalabh, H. Toutenburg / Statistics & Probability Letters one, the estimators have a closed form; see, e.g., Rao and Toutenburg Here we restrict our attention to such estimators. Superiority ofestimators arising from both the mixed regression and minimax linear estimation procedures over the conventional estimators ignoring the prior information is well discussed in the literature but explicit attention does not seem to have been paid to the relative eciency ofone approach over the other; see, e.g., Rao and Toutenburg 1999 for the details. This has formed the subject matter ofthis note. Taking the performance criterion to be the variance covariance matrix it is found that the mixed regression approach provides more ecient estimators than the minimax linear approach at least asymptotically.. Main result Consider a linear regression model y = X + u; where y is an n 1 vector of n observations on the study variable, X is an n K matrix of n observations on K explanatory variables, is a K 1 vector ofregression coecients and u is an n 1 vector ofdisturbances following a multivariate normal distribution with null mean vector as 0 and variance covariance matrix as times an identity matrix. It is assumed that the matrix X has full column rank and the scalar is unknown. Further, let us assume to be given the prior information specifying an unbiased estimate b 0 along with the variance covariance matrix s 0 X 0 X 0 based on n 0 observations from some extraneous source. This prior information can be expressed in two forms for the purpose of utilizing it in the estimation of. From the viewpoint ofpoint estimation, we may write it as b 0 = + v; where v is a K 1 random vector following a multivariate normal distribution with mean vector as 0 and variance covariance matrix as s 0 X 0 X 0. Alternatively, from the viewpoint of interval estimation, we can formulate it somewhat weaker as a condence ellipsoid given by.1. b 0 X 0X 0 b 0 6 s 0C;.3 where C denotes the 1 quantile ofthe distribution with n 0 degrees offreedom with 1 indicating the level ofcondence like 95% or 99%. We may note that here is a constant. The least-squares estimator of in.1 is given by b =X X X y; which ignores the prior information all together. Ifwe use formulation. ofthe prior information and employ the method ofmixed regression introduced by Theil and Goldberger 1961, the estimator of is given by ˆ = X y + s s 0.4 X 0X 0 b 0 ;.5

3 where H. Shalabh, H. Toutenburg / Statistics & Probability Letters s = 1 n K y Xb y Xb:.6 Similarly, ifwe consider a quadratic loss function with loss matrix ofrank one and apply the method ofminimax linear estimation using formulation.3 ofthe prior information on the lines of Kuks and Olman 197, the following estimator of is obtained: = C X 0X 0 X y + s C X 0X 0 b 0 ;.7 which can be termed as pseudo-minimax estimator; see Rao and Toutenburg 1999 for the details. Eciency properties of ˆ and in relation to b are well discussed in the literature but a comparison of ˆ and does not seem to have been made; see, e.g., Rao and Toutenburg 1999 for an interesting account. It can be easily seen following Kakwani 1968 that all the three estimators b; ˆ and are unbiased. Further, the variance covariance matrix of b is given by V b=eb b = X X : Exact expressions for the variance covariance matrices of ˆ and can be obtained following Swamy and Mehta 1969 but their comparison fails to provide any clear inference regarding their eciency. We, therefore, consider their large sample approximations. For this purpose, we assume that the explanatory variables are asymptotically cooperative so that the limiting form of the matrix n 1 X X is nite and nonsingular. Using.1 and., we observe that V ˆ=E ˆ ˆ = E [ [ = E X uu X + s4 s 4 0 X X + s4 s 0.8 ] X 0X 0 vv X 0X 0 ] X 0X 0 X X + s X 0X 0.9 by virtue ofmutual independence ofx u; s and v. Ifwe assume that the limiting form ofthe matrix n 1 X X is nite and nonsingular, it is shown in the appendix that V ˆ= X X 4 4 X X X 0X 0 X X +On 3 :.10 Similarly, the variance covariance matrix ofthe estimator is given by 1 C V = X X 4 s 0 C s 0 X X X 0X 0 X X +On 3 :.11

4 38 H. Shalabh, H. Toutenburg / Statistics & Probability Letters From.8,.10 and.11, we observe that both ˆ and are more ecient than b with respect to the criterion ofvariance covariance matrix to order On. Further, we have V 4 V ˆ= 1 1 X X X C 0X 0 X X ;.1 whence it follows that the variance covariance matrix of exceeds the variance covariance matrix of ˆ by a positive denite matrix implying the superiority ofthe mixed regression approach over the minimax linear approach for the estimation of regression coecients. It may be remarked that a similar comparison is made by Toutenburg and Srivastava 1996 but they deal with the case ofinterval constraints and consequently the estimators arising from the frameworks of mixed regression and minimax linear estimators do not have the same eciency properties as ˆ and in the present context. Appendix Let us write =s ; so that is oforder O p n 1=. It may be observed that by virtue ofmultivariate normality ofu, it may be observed that E=0 and E = 4 n K = 4 1 K n n = 1+ 4 Kn n + K n = 4 n +On : If g is any xed scalar, we can express gx 0X 0 [ =X X I K + gx 0X 0 X X + ] 1 gx 0X 0 X X [ =X X I K gx 0X 0 X X + ] gx 0X 0 X X + =X X gx X X 0X 0 X X gx X X 0X 0 X X +O p n 3 :

5 Using it along with X X + s4 g H. Shalabh, H. Toutenburg / Statistics & Probability Letters = X X g + g S0 ; g we nd that E gx 0X 0 X X + s4 g gx 0X 0 = X X 4 s 0 g 1 X X X g 0X 0 X X +On 3 : Substituting g = 1 and C, we obtain expressions.10 and.11. References Kakwani, N.C., Note on the unbiasedness ofmixed regression estimation. Econometrica 36, Kuks, J., Olman, W., 197. Minimax linear estimation ofregression coecients II. Iswestija Akademija Nauk Estonskoj SSR 1, 66 7 in Russian. Rao, C.R., Toutenburg, H., Linear Models: Least Squares and Alternatives. Springer, Berlin. Schipp, B., Minimax Schatzer im Simultanen Gleichungsmodell bei Vollstandiger und Partieller Vorinformation. Hain, Konigstein. Swamy, P.A.V.B., Mehta, J.S., On Theil s mixed regression estimator. J. Amer. Statist. Assoc. 64, Theil, H., Goldberger, A.S., On pure and mixed estimation in econometrics. Internat. Econom. Rev., Toutenburg, H., Srivastava, V.K., Estimation ofregression coecients subject to interval constraints. Sankhya Ser. A 58, 73 8.