Use of prior information in the form of interval constraints for the improved estimation of linear regression models with some missing responses

Transcription

1 Journal of Statistial Planning and Inferene 136 (2006) Use of prior information in the form of interval onstraints for the improved estimation of linear regression models with some missing responses H. Toutenburg a, Shalabh b, C. Heumann a, a Institut für Statistik, Universität Münhen, Münhen, Germany b Department of Mathematis & Statistis, Indian Institute of Tehnology, Kanpur , India Reeived 27 June 2004; aepted 21 Otober 2004 Available online 8 Deember 2004 Abstrat We onsider the estimation of oeffiients in a linear regression model when some responses on the study variable are missing and some prior information in the form of lower and upper bounds for the average values of missing responses is available. Employing the mixed regression framework, we present five estimators for the vetor of regression oeffiients. Their exat as well as asymptoti properties are disussed and superiority of one estimator over the other is examined Elsevier B.V. All rights reserved. MSC: 62J05 Keywords: Linear regression model; Missing data; Prior information; Interval onstraints; Least squares estimates 1. Introdution One of the frequently enountered features of the data olletion proess is the missing of some responses due to a variety of reasons. In suh situations, the standard statistial proedures developed for data with no missing values annot be immediately and straightforwardly applied for deduing inferenes. Consequently various strategies have been evolved Corresponding author. Institut für Statistik der LMU Münhen, Universität Münhen, Ludwigstr. 33, Münhen, Germany. Tel.: ; fax: address: hris@stat.uni-muenhen.de (C. Heumann) /$ - see front matter 2004 Elsevier B.V. All rights reserved. doi: /j.jspi

2 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) to find imputed values for the data set aordingly; see, e.g., Little and Rubin (1987) and Rubin (1987) for an exellent exposition. The thus repaired data resembles the data set without any missing value and permits the appliation of standard statistial proedures for analysis. This proposition, however, generally disturbs the known optimal properties of the standard statistial proedures. Sometimes some prior information about the missing responses may be available in the form of lower and upper bounds of their values. For example, if we are olleting data on various harateristis of households for a soio-eonomi study, some households in the lower stratum of the soiety may not dislose their true inome. Some of them may refuse outrightly, while others may supply figures whih are false or unbelievable. Consequently, suh values are onsidered inappropriate by the data olletor and are therefore treated as missing observations. However, depending upon the observations on other harateristis of suh households and mathing them with omplete observations or otherwise, it may be possible to speify an interval in eah ase suh that it is expeted to ontain the inome of that partiular household. Suh intervals speifying the lower and upper bounds may be derived from various soures like the data base, past experiene of data olletors and users, long assoiation with similar investigations, stability in repetitive studies, relationships with other orrelates and/or some other extraneous onsiderations. In suh ases, the piees of these information an be more preiously expressed in the form of lower and upper bounds rather than an exat value and it may be instrutive to utilize this kind of information in finding the imputed values for missing responses, estimating the regression parameters, onstruting onfidene regions and onduting tests of hypotheses. We may note that suh type of information omes from outside the sample and not from the experiment itself. When the prior information relates to regression oeffiients in the form of interval onstraints, the oeffiients are estimated by method of interval restrited estimation using least-squares proedures. The resulting estimators usually have no losed form exept in some partiular ases and therefore further analytial investigations are diffiult to arry out; see, e.g., Esobar and Skarpness (1986,1987), Judge and Yany (1986), Klemn and Sposito (1980), Ohtani (1987,1991), Srivastava and Ohtani(1995) and Wan (1996). Free from suh a limitation and easy to implement are two alternative proedures forwarded by Toutenburg and Roeder (1978), see also Toutenburg (1982). One proedure involves formulating an ellipsoid enlosing the uboid defined by the interval onstraints and then using the method of minimax linear estimation for the regression oeffiients. The other proedure onsists of deriving a set of linear stohasti restritions by treating the lower and upper limits of the intervals as onfidene limits and then applying the method of mixed regression estimation. Toutenburg and Srivastava (1996) have presented a ritial appraisal of the two proedures for the estimation of regression oeffiients and have disussed their asymptoti properties when disturbanes are small. A general onlusion emerging from their investigations is that the mixed regression formulation yields more effiient estimators, at least asymptotially. In view of the above observations, we propose to employ the mixed regression framework for the estimation of oeffiients in a linear regression model when some responses on the study variable are missing, but prior information in the form of lower and upper bounds for the average values of the missing responses is available. Suh a framework is desribed in Setion 2. The estimation of regression oeffiients is disussed in Setion 3 and five estimators are presented. Exat expressions for the bias vetors and mean squared error

3 2432 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) matries for the three estimators are provided and ompared in Setion 4. Similar expressions in ase of the remaining two estimators an be derived, but they will be suffiiently intriate and will not permit us to draw any lear onlusion. In Setion 5, we present large sample asymptoti approximations for the bias vetors and mean squared matries of all the five estimators and arry out effiieny omparisons. The underlying asymptoti theory assumes that only the number of omplete observations grows large; the number of inomplete observations stays fixed. Suh a speifiation is relaxed in Setion 6 and both are assumed to grow large. Under this supposition, the asymptoti approximations for the bias vetors and mean squared error matries are given and a omparative study is made. In order to study the finite sample properties of the estimators, we onduted a Monte Carlo simulation and its results are reported in Setion 7. Setion 8 presents some onluding remarks. Lastly, the derivation of results stated in theorems is given in the appendix. 2. Speifiation of model Consider the following linear regression model with some missing responses: y = X β + ε, y = X β + ε, (2.1) (2.2) where y is a m 1 vetor of observed responses, X is a m K matrix of m observations on K explanatory variables and is assumed to be of full olumn rank, β is a olumn vetor of K regression oeffiients and ε is a olumn vetor of m disturbanes. Similarly, y denotes a m 1 vetor of missing responses, X is a m K matrix (not neessarily of full olumn rank) of m observations on K explanatory variables without any missing value and ε is a olumn vetor of m disturbanes. The elements of disturbane vetors ε and ε are assumed to be independently and identially distributed following a normal distribution with mean 0 and variane σ 2. Suppose that there is some prior information available regarding the average values of the missing responses in the form of lower and upper bounds: L i E(y i ) U i (i = 1, 2,...,m ), (2.3) where L i and U i denote the lower and upper bounds, respetively, for the expeted value of the ith element of y. In order to utilize these onstraints in the estimation of parameters, let us onsider the framework of mixed regression estimation. Following Toutenburg and Srivastava (1996), let us treat them as p sigma limits so that the value of p is determined by onsiderations like redibility and truthfulness of restritions (2.3). Thus if we write a i = 1 2 (U i + L i ) (2.4) ψ i = 1 4p 2 (U i L i ) 2, (2.5)

4 we an express H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) a i = E(y i ) + ε i (i = 1, 2,...,m ), (2.6) where ε i is a random variable with mean vetor 0 and variane ψ i. Writing ompatly, we find a 1 E(y 1 ) ε 1 a 2.. = E(y 2 ). + ε 2... (2.7) a m E(y m ) ε m or a = E(y ) + ε = X β + ε, (2.8) so that ε is a m 1 random vetor with mean vetor 0 and variane matrix Ψ where Ψ is a diagonal matrix with diagonal elements as ψ 1, ψ 2,...,ψ m. It may be observed that (2.1) and (2.7) together mimis the mixed regression framework beause the vetor a here is truly not a random vetor. Eq. (2.7) serves as a kind of approximate representation of restritions (2.3); see Toutenburg and Srivastava (1996) for details and its impliations on the effiieny properties of the estimators for regression oeffiients. 3. Estimation of regression oeffiients When only the omplete observations are used, an appliation of least-squares method to (2.1) provides the following estimator of β: b = (X X ) 1 X y. (3.1) Obviously, suh an estimator disards the remaining inomplete observations in the available data set. We may therefore replae the elements of y in (2.2) by their predited values as speified by X b. Now applying least-squares method to the thus repaired model, we find the estimator of β as follows: b = (X X + X X ) 1 (X y + X X b ) = b. (3.2) This implies that the use of unbiased predited values does not bring any improvement; see Yates (1933) and Rao and Toutenburg (1999, Chapter 8). Besides it, estimator (3.2) does not utilize the available prior information. If we onsider the mixed regression framework speified by (2.1) and (2.7), an appliation of least-squares proedure gives the following estimator of β: b LS = (X X + X X ) 1 (X y + X a) (3.3) while an appliation of the feasible generalized least-squares proedure provides another estimator of β: b FGLS = (X X + s 2 X Ψ 1 X ) 1 (X y + s 2 X Ψ 1 a), (3.4)

5 2434 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) where s 2 1 = m K (y X b ) (y X b ) (3.5) is an estimator of σ 2. One an take (m + m K) instead of (m K) in (3.5); see Rao and Toutenburg (1999, Setion 8.2). Now we an take X b LS or X b FGLS as imputed values for the missing responses in (2.2); see Toutenburg and Shalabh (1996,2000) for the properties and some additional results. Using these to replae y in (2.2) and then applying least-squares method to the thus obtained equation and (2.1), we find the following two estimators of β: ˆβ 1 = (X X + X X ) 1 (X y + X X b LS ), (3.6) ˆβ 2 = (X X + X X ) 1 (X y + X X b FGLS ) (3.7) whih do not redue to b like (3.2). 4. Comparison of estimators: exat results It is easy to see that b is an unbiased estimator of β while the remaining four estimators b LS, b FGLS, ˆβ 1 and ˆβ 2 are generally biased. The bias vetors of b LS and ˆβ 1 are given by B(b LS ) = (X X + X X ) 1 X (a X β) = δ (say), (4.1) B(ˆβ 1 ) = (X X + X X ) 1 X X (X X + X X ) 1 X (a X β) = (X X + X X ) 1 X X δ. (4.2) Similar expressions for the bias vetors of b FGLS and ˆβ 2 an be derived but their funtional forms will be suffiiently intriate and it will not be possible to draw any lear inferene. From (4.1) and (4.2), we observe that [B(ˆβ 1 )] [B(ˆβ 1 )]=δ X X (X X + X X ) 2 X X δ δ δ =[B(b LS )] [B(b LS )] (4.3) implying that the estimator ˆβ 1 is superior to the estimator b LS aording to the riterion of the length of the bias vetor. The variane ovariane matrix of b is given by V(b ) = σ 2 (X X ) 1 (4.4) while the mean squared error matries of b LS and ˆβ 1 are M(b LS ) = σ 2 [(X X ) 1 Δ]+δδ, (4.5) M(ˆβ 1 ) = σ 2 (I K + W) [(X X ) 1 Δ](I K + W)+ W δδ W, (4.6)

6 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) where W = X X (X X + X X ) 1, (4.7) Δ = (X X ) 1 (X X + X X ) 1 X X (X X + X X ) 1. (4.8) As mentioned earlier, one an derive the exat expressions for the mean squared error matries of the estimators b FGLS and ˆβ 2 but they will be onsiderably omplex in omparison to (4.5) and (4.6). As the matrix (X X + X X )(X X ) 1 (X X + X X ) (X X ) is positive definite, the matrix Δ is too. Consequently, from (4.4) and (4.5), we have D(b ; b LS ) = V(b ) M(b LS ) = σ 2 Δ δδ (4.9) whih is a nonnegative definite matrix by virtue of Rao and Toutenburg (1999, Theorem A.57, p. 370) if and only if δ Δ 1 δ σ 2. (4.10) Similarly, the estimator ˆβ 1 an be ompared with b. It is, however, diffiult to dedue any neat ondition from (4.4) and (4.6) for the superiority of one estimator over the other. Finally, if we ompare expressions (4.5) and (4.6), it may be well appreiated that no lear inferene regarding the superiority of b LS over ˆβ 1 or vie-versa an be drawn. 5. Comparison of estimators: asymptoti approximations when m is large but m stays fixed In this setion, we utilize the large sample theory when the number of omplete observations grows large while the number of inomplete observations stays fixed. For this purpose, we assume the asymptoti ooperativeness of the explanatory variables in the model so that the matrix m 1 X X tends to a finite and nonsingular matrix Q as m grows large. Utilizing the law of large numbers, it an be easily seen that all the five estimators b, b LS, b FGLS, ˆβ 1 and ˆβ 2 are onsistent for β. Further, if we onsider the asymptoti distribution of m 1/2 times the estimation error, all the five estimators are found to have the same asymptoti distribution whih is multivariate normal with mean vetor 0 and variane ovariane matrix σ 2 Q 1. We therefore need higher order approximations so as to disriminate among their performane properties. Let us first introdue the following notation: ( ) 1 1 S = X m X, G= 1 X m X, θ = 1 m X (a X β), θ Ψ = 1 m X Ψ 1 (a X β). (5.1)

7 2436 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) We assume that m 1 X X tends to a finite and nonsingular matrix as m grows large whereas G remains nonstohasti. From (4.1) and (4.2), it is easy to see that the bias vetors of b LS and ˆβ 1 to order O(m 2 ) are given by B(b LS ) = m S θ m2 m m 2 S GS θ, (5.2) B(ˆβ 1 ) = m2 m 2 S GS θ. (5.3) Similar asymptoti results for the estimators b FGLS and ˆβ 2 are derived in the appendix. These are stated below. Theorem 1. The bias vetor of b FGLS to order O(m 1 ) is B(b FGLS ) = m σ 2 S θ Ψ (5.4) m while the estimator ˆβ 2 is unbiased to this order of approximation. However, if we onsider bias vetor of order O(m 2 ), it is given by B(ˆβ 2 ) = m2 σ2 m 2 S GS θ Ψ. (5.5) Comparing the estimators with respet to the bias vetor to order O(m 1 ), it is seen that the estimators ˆβ 1 and ˆβ 2 are asymptotially unbiased to the first order while b LS and b FGLS are generally biased. Comparing b LS and b FGLS with respet to the riterion of bias vetor length to order O(m 2 ), we observe from (5.2) and (5.4) that b FGLS is preferable in omparison to b LS when Ψ i exeeds σ 2 for all i. Suh a ondition is satisfied so long as σ 2 is less than the minimum value of Ψ i, i.e., min (U i L i ) 2 > 4p 2 σ 2. (5.6) i The opposite is true, i.e., the estimator b LS is better than b FGLS with respet to the riterion of the bias vetor length when σ 2 exeeds Ψ i for all i whih is satisfied as long as max i (U i L i ) 2 < 4p 2 σ 2. (5.7) Similarly, we see from (5.3) and (5.5) that the estimator ˆβ 2 is superior to ˆβ 1 aording to the riterion of the length of the bias vetor to order O(m 2 ) when Ψ i exeeds σ 2 for all i whih holds true so long as (5.6) is satisfied. The reverse is true when (5.7) holds good. Summarizing the results, we onlude that the estimator b is unbiased while the estimators ˆβ 1 and ˆβ 2 are almost unbiased; the remaining two estimators b LS and b FGLS are biased. Next, let us ompare the estimators with respet to the riterion of the asymptoti mean squared error matrix.

8 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) From (4.5) and (4.6), we observe that M(b LS ) = σ2 S m m m 2 S (2σ 2 G m θθ )S + O(m 3 ), (5.8) M(ˆβ 1 ) = σ2 S 2σ2 m 2 m m 3 S GS GS + O(m 4 ). (5.9) Similar results for the estimators b FGLS and ˆβ 2 are obtained in the appendix. Theorem 2. The mean squared error matries of the estimators b FGLS and ˆβ 2 to the given order of approximation are given by M(b FGLS ) = σ2 S σ4 m m 2 S (2X Ψ 1 X m 2 θ Ψθ Ψ )S + O(m 3 ), (5.10) M(ˆβ 2 ) = σ2 S σ4 m m m 3 S [GS X Ψ 1 X + X Ψ 1 X S G]S + O(m 4 ). (5.11) Observing that the variane ovariane matrix of b is V(b ) = σ2 S (5.12) m we find from (5.8) that D(b ; b LS ) = 1 m 2 S X [2σ2 I K (a X β)(a X β) ]X S (5.13) whih is nonnegative definite if and only if or (a X β) (a X β) 2σ 2 m [E(y i ) 2 1 (L i + U i )] 2 2σ 2, (5.14) i=1 where use has been made of Rao and Toutenburg (1999, Theorem A.57, p. 370). Thus inequality (5.14) provides a neessary and suffiient ondition for the superiority of b LS over b. Similarly, we observe from (5.10) and (5.12) that the matrix D(b ; b FGLS ) = σ4 m 2 S X Ψ 1 [2Ψ (a X β) (a X β)]ψ 1 X S (5.15) is nonnegative definite if and only if (a X β) Ψ 1 (a X β) 2 (5.16)

9 2438 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) or m [ ] 2E(y i ) (L i + U i ) 2 2 (U i L i ) p 2 (5.17) i=1 whih is the ondition for the superiority of b FGLS over b. It is lear from (5.8) and (5.11) that ˆβ 1 and ˆβ 2 are more effiient than b. Next, let us ompare the estimators b LS, b FGLS, ˆβ 1 and ˆβ 2 with respet to the riterion of mean squared error matrix. From expressions (5.8) (5.11), we see that both the estimators b LS and b FGLS are superior to ˆβ 1 and ˆβ 2 as long as they are more effiient than b. When onditions (5.14) and (5.17) do not hold true, then ˆβ 1 and ˆβ 2 are nearly unbiased and more effiient than b LS and b FGLS. Comparing ˆβ 1 and ˆβ 2, it is observed from (5.9) and (5.11) that D(ˆβ 1 ; ˆβ 2 ) = σ2 m m 3 S [GS X (σ2 Ψ 1 I)X + X (σ2 Ψ 1 I)X S G]S (5.18) whih is nonnegative definite implying the superiority of ˆβ 2 over ˆβ 1, when σ 2 is greater than Ψ i for all i. This ondition is satisfied as long as (5.7) holds good. Similarly, it is seen from (5.17) that ˆβ 1 is superior to ˆβ 2 when σ 2 is less than Ψ i for all i. This ondition is satisfied as long as (5.6) holds true. 6. Comparison of estimators: asymptoti approximations when both m and m are large Now let us analyze the properties of estimators when both the number of omplete observations and the number of inomplete observations grow large. Aordingly, it is assumed that both the matries m 1 X X and m 1 X X tend to finite and nonsingular matries as m and m tend to infinity. We first introdue the following notations: ( ) 1 1 ( ) 1 S = X m X, G= X m X, f = m ( ) 1, G Ψ = X m m Ψ 1 X, (6.1) so that (X X + X X ) 1 = 1 m (I K + fs G) 1 S = 1 m H, (6.2) (X X + σ 2 X Ψ 1 X ) 1 = 1 m (I K + f σ 2 S G Ψ ) 1 S = 1 m H Ψ. (6.3) Here f k(k = 0, ) as m and m grow large, where k is a fixed onstant.

10 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) From the appendix, we have the following results: Theorem 3. The leading terms in the bias vetors are given by B(b LS ) = fhθ, (6.4) B(b FGLS ) = f σ 2 H Ψ θ Ψ, (6.5) B(ˆβ 1 ) = f 2 HGHθ, (6.6) B(ˆβ 2 ) = f 2 σ 2 HGH Ψ θ Ψ. (6.7) It should be mentioned that (6.4) and (6.6) are exat results while (6.5) and (6.7) are asymptoti. These results learly indiate that all the four estimators, viz., b LS, b FGLS, ˆβ 1 and ˆβ 2 are inonsistent for β, while b is a known onsistent estimator. If we ompare the estimators with respet to the length of bias vetor, it is easy to see that [B(ˆβ 1 )] [B(ˆβ 1 )] [B(b LS )] [B(b LS )], (6.8) [B(ˆβ 2 )] [B(ˆβ 2 )] [B(b FGLS )] [B(b FGLS )] (6.9) whih implies the superiority of ˆβ 1 over b LS and ˆβ 2 over b FGLS. As the estimators are not onsistent, we onsider their variane ovariane matries rather than their mean squared error matries for omparing them. Theorem 4. The asymptoti approximations for the variane ovariane matrix to order ) are given by O(m 1 V(b LS ) = σ2 m HS 1 H, (6.10) V(b FGLS ) = σ2 (H Ψ S 1 H Ψ + 2f 2 σ 2 H Ψ S 1 H Ψ θ Ψ θ Ψ m H ΨS 1 H Ψ ), (6.11) V(ˆβ 1 ) = σ2 m (I K + fgh)hs 1 H(I K + fgh), (6.12) V(ˆβ 2 ) = σ2 H [(I K + fgh m Ψ )S 1 (I K + fh Ψ G) + 2f 4 σ 2 H Ψ S 1 H Ψ θ Ψ θ Ψ H ΨS 1 H Ψ G]H. (6.13) It is rather inappropriate, in our opinion, to make a omparison of the onsistent estimator b with the inonsistent estimators b LS, b FGLS, ˆβ 1 and ˆβ 2. We therefore delete it and restrit our attention to inonsistent estimators only. It is interesting to observe from expressions (6.10), (6.12) and (6.13) that b LS is better than ˆβ 1 and ˆβ 2 aording to the riterion of asymptoti variane ovariane matrix.

11 2440 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) Next, we observe that H 1 HΨ 1 = m X (I m σ 2 Ψ 1 )X (6.14) is a positive definite matrix when Ψ 1, Ψ 2,...,Ψ m are all greater than σ 2 whih holds true if Ψ min exeeds σ 2 where Ψ min denotes minimum value in Ψ i s; see (5.6). And then (H Ψ H)is positive definite. The opposite is true, i.e., (H H Ψ ) is positive definite when Ψ max is less than σ 2 whih holds so long as (5.7) holds good. It thus follows from (6.10) and (6.11) that b LS is better than b FGLS when Ψ min is greater than σ 2 whih is satisfied as long as ondition (5.6) holds true. Similarly, we have D(b LS ; b FGLS ) = V(b LS ) V(b FGLS ) = σ2 [HS 1 H H Ψ S 1 H Ψ m + 2f 2 σ 2 H Ψ S 1 H Ψ θ Ψ θ Ψ H ΨS 1 H Ψ ] (6.15) whih is nonnegative definite if and only if θ Ψ H ΨS 1 H Ψ (H S 1 H H Ψ S 1 H Ψ ) 1 H Ψ S 1 H Ψ θ Ψ < 2f 2 σ 2 (6.16) provided that Ψ max is less than σ 2 whih is satisfied so long as (5.7) holds true. Under ondition (5.7) ombined with (6.16), the estimator b FGLS is better than b LS. In a similar manner, we an ompare expressions (6.12) and (6.13), and an dedue onditions for the superiority of ˆβ 1 over ˆβ 2 and vie-versa but suh onditions are not attrative as they are hard to verify in any given appliation. 7. Monte Carlo simulation study The large sample asymptoti approximation theory desribes the behaviour of distribution of the estimator in the entral part of the distribution of the estimator. In order to study it over the whole sample spae, we onduted a Monte Carlo simulation study. We would like to mention that using information on the fully observed data, i.e., X and y, alone to impute the values for the missing y generally annot give any improvement on the estimate of β. Reason being that the observed X of the missing part does not ontain any information about the regression of Y on X. Thus using, e.g., the predited values X b as mid-points of some intervals [L i,u i ], leads to idential estimates b = b LS = b FGLS = ˆβ 1 = ˆβ 2. That means that the prior information about the bounds for the onditional expetations in Eq. (2.3) really has to ome from an external soure of information. This also verifies the assumption made in Setion 1 that suh information omes from outside the sample. We assume that the intervals for prior information are onstruted around the true onditional expetations of the missing y, whih are known in the simulation. We then onstrut symmetri and unsymmetri bounds around that true value. First, let us desribe the simulation setup. We simulated two different sample sizes (N = 100, 500), two perentages of missing values (30%, 50%), the residual varianes were σ 2 = 1, 5 and p = 2, 3. The data was simulated using an interept term and two orrelated

12 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) Table 1 N = 100, 30% missing, σ 2 = 1, p = 2, ρ X1,X 2 = 0.1 Est SF1 SF2 SF3 SF5 SR1 SR2 SR3 SR5 b b LS b FGLS ˆβ ˆβ ovariates X 1 and X 2 using three different orrelation strutures ρ X1,X 2 = 0.1, 0.5, 0.9 aording to y i = β 0 + β 1 x i1 + β 2 x i2 + ε i with β = (β 0, β 1, β 2 ) = (1.0, 1.0, 1.0) and ε i N(0, σ 2 ). The design matrix X was generated only one for eah orrelation oeffiient, the experiment was repeated 5000 times for eah setting with respet to errors ε. Then we ompute the unweighted empirial mean square error for omparing the five estimators based on 5000 repliations. The empirial MSE of an estimator of β where k is the estimate of the kth repliation, is alulated as Emp.MSE = ) {( 0k β 0 ) 2 + ( 1k β 1 ) 2 + ( 2k β 2 ) 2 }. ( 5000 k=1 Now we desribe the onstrution of the bounds. The first method uses the true values X β and onstruts symmetri bounds around these values with equal length for all missing values. We all it SFj (symmetri fixed with length jσ). The seond method uses intervals of different length for eah missing ase. We all it SRj (symmetri random with length j N(0, σ 2 ) ). The seond method uses unsymmetri bounds around the true value. Again we onsider the ase of a ommon interval length and the ase of a random interval length for eah i. UFjk (unsymmetri fixed) means that the lower bound is onstruted as true mean minus jσ and the upper bound is onstruted as true value plus kσ, j = 1, 2, j<k= 2, 3, 3.5, 4, 5. As in the symmetri ase for simulating individual interval length, a random variable Z i is drawn for eah i and the lower bound is omputed as true value minus j Z i and the upper bound as true value plus k Z i. Sine the number of possible parameter settings are high, we restrit ourselves to report the results of three settings only. Espeially, the results for N = 500 are similar (with respet to the omparison of the estimates) to the results of N = 100 and therefore only the latter are reported. First we look at the results of symmetri bounds in Table 1. We see that b LS is always an improvement to the omplete ase estimator b. This is also lear as a is the vetor of true values in this ase and so the estimator is shrinked towards the true vetor β whih is independent of the length of the intervals. Also, b LS is better than ˆβ 1 and ˆβ 2 in every setting. The omparison of ˆβ 1 and ˆβ 2 in every setting shows superior performane of ˆβ 2 if the bounds are random but inferior performane is reported if the bounds have ommon length and are wide enough (SF3, SF5). Finally, if the bounds are narrow with ommon

13 2442 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) Table 2 N = 100, 50% missing, σ 2 = 1, p = 2, ρ X1,X 2 = 0.9 Est UF12 UF13 UF1(3.5) UF15 UF24 UF25 b b LS b FGLS ˆβ ˆβ Table 3 N = 100, 30% missing, σ 2 = 1, p = 2, ρ X1,X 2 = 0.5 Est UR12 UR13 UR1(3.5) UR15 UR24 UR25 b b LS b FGLS ˆβ ˆβ length, or the bounds are random, b FGLS is better than b LS with large differenes. Note that b LS and β 1 does not depend on the interval length in the symmetri ase. Now let us take a look on the unsymmetri ase. Table 2 shows the result for five settings with a ommon interval length for eah missing ase. With the exeption of setting UF12, ˆβ 1 and ˆβ 2 beat their ounterparts b LS and b FGLS. In every setting, ˆβ 2 beats ˆβ 1. But if the bounds are very muh unsymmetri (UF15) then b is the best estimator whereas b LS is best under UF12. The results do not arry over to the random length bounds as this an be seen from Table 3. If the bounds are narrow (UR12, UR13, UR24), b FGLS is the best estimator (same as in the ase of symmetri setup). As the bounds get more unsymmetri, βˆ 2 performs better and b looses its performane to remain as the best estimator. Finally, we an draw the onlusion autiously that b FGLS and ˆβ 2 are the winners while ˆβ 2 has some advantage to give more stable results. On the one hand, b FGLS an be dramatially better than the omplete ase estimator but may fail (UF15, UR15) in the very unsymmetri ases. 8. Some remarks We have onsidered the estimation of the oeffiients in a linear regression model when some of the responses are missing but possibly lower and upper bounds for them are available. The mixed regression framework for the utilization of suh prior information is presented and five estimators for the vetor of regression oeffiients are envisaged. One estimator is the traditional least-squares estimator whih disards the inomplete observations

14 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) and ignores the prior information too. The remaining four estimators use the inomplete observations as well as the prior restritions. Effiieny properties of these estimators are analyzed employing some exat as well as asymptoti results and onditions are obtained for the superiority of one estimator over the other. In order to explore the exat (analyti) properties, some interesting findings are reported arising from the Monte Carlo simulation study. However, some further exploration is required. For example, it will be interesting to arry out a similar investigation for evaluating the performane of estimators arising from the ellipsoidal formulation of restritions (2.3) and use of minimax linear estimation approah onsidered by Toutenburg and Wargowske (1978). Aknowledgements The authors are grateful to the referees for their illuminating omments to improve the paper. Appendix For the derivation of the results in Theorems 1 and 2, let us write u = 1 m 1/2 X ε, w = ( 1 m 1/2 ) ε ε m 1/2 σ 2 so that u and w are of order O p (1). Using these, we observe from (2.1), (3.4) and (3.5) that s 2 = σ 2 + w m 1/2 + Kσ2 u S u m + O p (m 3/2 ), (A.1) [ ] (b FGLS β) = (X X + s 2 X Ψ 1 X ) 1 X ε + s 2 X Ψ 1 (a X β) = 1 m 1/2 S u + m σ 2 m S θ Ψ + 1 m 3/2 S (m wθ Ψ σ 2 X Ψ 1 X S u) + O p (m 2 ). (A.2) Retaining terms to order O(m 1 ) and taking expetation, we get result (5.4) of Theorem 1.

15 2444 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) Similarly, to order O(m 2 ),wehave M(b FGLS ) = 1 m S E(uu )S + σ2 m S [θ Ψ E(u ) + E(u)θ Ψ ]S + 1 m S [σ 4 m 2 θ Ψθ Ψ + m θ Ψ E(wu ) + m E(wu)θ Ψ σ 2 E(uu )S X Ψ 1 X σ 2 X Ψ 1 X S E(uu )]S utilizing the results for normality of disturbanes, we find result (5.10) of Theorem 2. In a similar manner, using (A.2), we an express (ˆβ 2 β) = (X X + X X ) 1 [X ε + X X (b FGLS β)] = 1 m 1/2 S u + m2 σ2 m S GS θ Ψ + m m 5/2 S GS (m wθ Ψ σ 2 X Ψ 1 X S u) + O p (m 3 ). (A.3) Thus the bias vetor to order O(m 2 ) and mean squared error matrix to order O(m 3 ) are obtained whih are results (5.5) of Theorem 1 and (5.11) of Theorem 2. Next, let us onsider the results stated in Theorem 3. It an be easily seen that ( ) (b LS β) = H f θ + 1 m 1/2 S u (A.4) whene we have (ˆβ 1 β) = (X X + X X ) 1 [X X (b LS β) + X ε ] = f 2 HGHθ + H(I K + f GH )u. (A.5) Taking expetation, we find results (6.4) and (6.6). In a similar manner, using (A.1), we an express where (b FGLS β) = f σ 2 H Ψ θ Ψ + 1 m 1/2 ξ = wf H Ψ S 1 H Ψ θ Ψ + H Ψ u. Likewise, utilizing (A.6), we an write (ˆβ 2 β) = f 2 σ 2 HGH Ψ θ Ψ + 1 m 1/2 ξ + O p (m 1 ), (A.6) H(u+ fgξ) + O p (m 1 ). (A.7) Taking expetations on both the sides in (A.6) and (A.7), we find results (6.5) and (6.7) stated in Theorem 3. Finally, we onsider the results of Theorem 4.

16 H. Toutenburg et al. / Journal of Statistial Planning and Inferene 136 (2006) From (A.4) and (A.5), we obtain results (6.10) and (6.11). Next, utilizing that E(w 2 ) = 1 m E(ε ε ) 2 2σ 2 E(ε ε ) + m σ 4 = 2σ 4, we get results (6.11) and V(ˆβ 2 ) = 1 m HE(u+ fgξ)(u + f ξ G)H = σ2 H [S 1 + f (GH Ψ S 1 m + S 1 + 2σ 2 f 4 H Ψ S 1 H Ψ θ Ψ θ Ψ H ΨS 1 H Ψ ]H whih provides result (6.11) and (6.13). H Ψ G) + f 2 GH Ψ S 1 H Ψ G Referenes Esobar, L.A., Skarpness, B., The bias of the least squares estimator over interval onstraints. Eonom. Lett. 20, Esobar, L.A., Skarpness, B., Mean square error effiieny of the least squares estimator over interval onstraints. Comm. Statist. Part A Theory Methods 16, Judge, G.G., Yany, T., Improved Methods Of Inferene In Eonometris. North-Holland, Amsterdam. Klemn, R.J., Sposito, V.A., Least squares solutions over interval restritions. Comm. Statist. Part B Simulation Comput. 9, Little, R.J.A., Rubin, D.B., Statistial Analysis with Missing Data. Wiley, New York. Ohtani, K., The mse of a least squares estimator over an interval onstraint. Eonom. Lett. 25, 351. Ohtani, K., Small sample properties of the interval onstrained least squares estimator when error terms have a multivariate t-distribution. J. Japan Statist. So. 21, Rao, C.R., Toutenburg, H., Linear Models: Least Squares and Alternatives. seond Ed. Springer, New York. Rubin, D.B., Multiple Imputation for Nonresponse in Sample Surveys. Wiley, New York. Srivastava, V.K., Ohtani, K., A omparison of interval onstrained least squares and mixed regression estimators. Commun. Statist. Part A Theory Methods 24, Toutenburg, H., Prior Information in Linear Models. Wiley, New York. Toutenburg, H., Roeder, B., Minimax-linear and Theil estimator for restrained regression oeffiients. Statistis 9, Toutenburg, H., Shalabh, Preditive performane of the methods of restrited and mixed regression estimators. Biometrial J. 38, Toutenburg, H., Shalabh, Improved preditions in linear regression models with stohasti linear onstraints. Biometrial J. 42, Toutenburg, H., Srivastava, V.K., Estimation of regression oeffiients subjet to interval onstraints. Sankhya, Ser. A 58, Toutenburg, H., Wargowske, B., On restrited 2-stage-least-squares (2-SLSE) in a system of strutural equations. Statistis 9, Wan, A.T.K., On the bias and mean squared error in a regression model with two inequality onstraints and multivariate t error terms. Commun. Statist. Part A Theory Methods 25, Yates, F., The analysis of repliated experiments when the field results are inomplete. Empire J. Exp. Agr. 1,