A matrix handling of predictions of new observations under a general random-effects model

Transcription

1 Electronic Journal of Linear Algebra Volume 29 Special volume for Proceedings of the International Conference on Linear Algebra and its Applications dedicated to Professor Ravindra B. Bapat Article A matrix handling of predictions of new observations under a general random-effects model Yongge Tian China Economics and Management Academy, Central University of Finance and Economics, Beijing , China, yongge.tian@gmail.com Follow this and additional works at: Recommended Citation Tian, Yongge. 2015), "A matrix handling of predictions of new observations under a general random-effects model", Electronic Journal of Linear Algebra, Volume 29, pp DOI: This Article is brought to you for free and open access by Wyoming Scholars Repository. It has been accepted for inclusion in Electronic Journal of Linear Algebra by an authorized editor of Wyoming Scholars Repository. For more information, please contact scholcom@uwyo.edu.

2 A MATRIX HANDLING OF PREDICTIONS UNDER A GENERAL LINEAR RANDOM-EFFECTS MODEL WITH NEW OBSERVATIONS YONGGE TIAN Abstract. Linear regression models that include random effects are commonly used to analyze longitudinal and correlated data. Assume that a general linear random-effects model y = Xβ + ε with β = Aα+γ is given, and new observations in the future follow the linear model y f = X f β+ε f. This paper shows how to establish a group of matrix equations and analytical formulas for calculating the best linear unbiased predictor BLUP) of the vector φ = Fα+Gγ +Hε+H f ε f of all unknown parameters in the two models under a general assumption on the covariance matrix among the random vectors γ, ε and ε f via solving a constrained quadratic matrix-valued function optimization problem. Many consequences on the BLUPs of φ and their covariance matrices, as well as additive decomposition equalities of the BLUPs with respect to its components are established under various assumptions. Key words. Linear random-effects model, Quadratic matrix-valued function, Löwner partial ordering, BLUP, BLUE, Covariance matrix. AMS subject classifications. 15A09, 62H12, 62J05. Dedicated to Professor R. B. Bapat on the occasion of his 60th birthday 1. Introduction. Throughout this paper, R m n stands for the collection of all m n real matrices. The symbols A, ra) and RA) stand for the transpose, the rank and the range column space) of a matrix A R m n, respectively. I m denotes the identity matrix of order m. The Moore Penrose inverse of A, denoted by A +, is defined to be the unique solution G satisfying the four matrix equations AGA = A, GAG = G, AG) = AG, and GA) = GA. P A, E A, and F A stand for the three orthogonal projectors symmetric idempotent matrices) P A = AA +, E A = A = I m AA +, and F A = I n A + A, where E A and F A satisfy E A = F A and F A = E A. Two symmetric matrices A and B of the same size are said to satisfy the Löwner partial ordering A B if A B is nonnegative definite. Consider a general Linear Random-effects Model LRM) defined by or marginally, y = Xβ +ε, β = Aα+γ, 1.1) y = XAα+Xγ +ε, 1.2) Received by the editorsonfebruary22, Accepted forpublication onmay28, 2015 Handling Editor: Simo Puntanen. China Economics and Management Academy, Central University of Finance and Economics, Beijing , China yongge.tian@gmail.com). Supported by the National Natural Science Foundation of China Grant No ). 30

3 Predictions under LRM 31 where in the first-stage model, y R n 1 is a vector of observable response variables, X R n p is a known matrix of arbitrary rank, ε R n 1 is a vector of unobservable random errors, while in the second-stage model, β R p 1 is a vector of unobservable random variables, A R p k is known matrix of arbitrary rank, α R k 1 is a vector of fixed but unknown parameters fixed effects), γ R p 1 is a vector of unobservable random variablesrandom effects). Concerning the expectation and covariance matrix of random vectors γ and ε in 1.1), we adopt the following general assumption E [ γ ε ] = 0, Cov [ γ ε ] = Σ11 Σ 12 := Σ, 1.3) Σ 21 Σ 22 where Σ 11 R p p, Σ 12 = Σ 21 Rp n, and Σ 22 R n n are known, while Σ R p+n) p+n) is non-negative definite nnd) matrix of arbitrary rank. One of the ultimate goals of statistical modelling is to be able to predict future observations based on currently available information. Assume that new observations of response variables in the future follow the model y f = X f β +ε f = X f Aα+X f γ +ε f, 1.4) where X f R nf p is a known model matrix associated with the new observations, β is the same vector of unknown parameters as in 1.1), and ε f R nf 1 is a vector of measurement errors associated with new observations. Combining 1.2) and 1.4) yields the following marginal model [ ỹ = XAα+ Xγ y X ε + ε, ỹ =, X = ], ε =. 1.5) yf X f ε f In order to establish some general results on prediction analysis of 1.5), we assume that the expectation and covariance matrix of γ, ε, and ε f are given by E γ ε = 0, Cov γ ε = Σ 11 Σ 12 Σ 13 Σ 21 Σ 22 Σ 23 := Σ, 1.6) ε f ε f Σ 31 Σ 32 Σ 33 where we don t attach any further restrictions to the patterns of the submatrices Σ ij in 1.6), although they are usually taken as certain prescribed forms for a specified LRM in the statistical literature. In particular, the covariances among the residual or error vector with other random factors in the model are usually assumed to be zero. This assumption is ordinarily applied to most practical applications in the biological sciences when the assumption is invalid. Linear regression models that include random effects are commonly used to analyze longitudinal and correlated data, which are available to account for the variability of model parameters due to different factors that influence a response variable. The LRM in 1.1) is also called nested model or two-level model in the statistical literature, where the two equations are called the first-stage model and the second-stage model, respectively. Statistical inference on LRMs is now an important part in data analysis,

4 32 Y. Tian and a huge amount of literature spreads in statistics and other disciplines. As usual, a main task in the investigation of LRM is to establish predictors/estimators of all unknown parameters in the model. Recall that the Best Linear Unbiased Predictors BLUPs) of unknown random parameters and the Best Linear Unbiased Estimators BLUEs) of fixed but unknown parameters in LRMs are fundamental concepts in current regression analysis, which are defined directly from the requirement of both unbiasedness and minimum covariance matrices of predictors/estimators of the unknown parameters. In fact, BLUPs/BLUEs are primary choices in all possible predictors/estimators due to their simple and optimality properties, and have wide applications in both pure and applied disciplines of statistical inferences. The theory of BLUPs/BLUEs under linear regression models belongs to the classical methods of mathematical statistics. Along with recent development of optimization methods in matrix theory, it is now easy to deal with various complicated matrix operations occurring in the statistical inference of 1.5). In [25], the present author established a group of fundamental matrix equations and analytical formulas for calculating the BLUPs/BLUEs of all unknown parameters in1.1) via solving a constrained quadratic matrix-valued function optimization problem, and also formulated an open problem of establishing matrix equations and formulas for calculating the BLUPs/BLUEs of the future y f, X f β, X f Aα, X f γ, and ε f in 1.4) from the observed response vector y in 1.1). For convenience of representation, let R = R1 = R 2 X In 0 = [ X f 0 I X, I n+nf ]. 1.7) nf Under the general assumptions in 1.3) and 1.6), the covariance matrix of the combined random vector ỹ in 1.5) is given by Covy) Cov{y,yf } Covỹ) = = R ΣR Cov{y f,y} Covy f ) := V, 1.8) where Covy) = R 1 ΣR 1 := V 11 Cov{y, y f } = R 1 ΣR 2 := V 12, 1.9) Cov{y f, y} = R 2 ΣR 1 := V 21, Covy f ) = R 2 ΣR 2 := V ) They are all known matrices under the assumptions in 1.1) 1.6), and will occur in the statistical inference of 1.5). Assumptions in 1.1) 1.6) are so general that they include almost all LRMs with different structures of covariance matrices as their special cases. Note from 1.5) that under the general assumptions in 1.1) 1.6), y and y f are correlated. Hence, it is desirable to give predictions of the future observations y f, as well as X f β and ε f in 1.4) from the original observation vector y in 1.1) under the assumptions in 1.1) 1.6). It is of great practical interest to simultaneously identify the important predictors that correspond to both the fixed- and random-effects components in LRM. Some previous and recent work on simultaneous estimations and predictions of combined unknown parameters under regression models can be found in [3, 17, 21, 25, 26].

5 Predictions under LRM 33 In order to predict/estimate all unknown parameters in 1.5) simultaneously, we construct a vector containing the fixed effects, random effects, and error terms in 1.5) as follows φ = Fα+Gγ +Hε+H f ε f, 1.11) where F R s k, G R s p, H R s n, and H f R s n f are known matrices. In this case, Eφ) = Fα, Covφ) = J ΣJ, Cov{φ, y} = J ΣR 1, J = [G, H, H f ]. 1.12) Eq.1.11) contains all possible matrix and vector operations in 1.1) 1.5) as its special cases. For instance, If F = T XA, G = T X, and [H, H f ] = T, then 1.11) becomes φ = T XAα+T Xγ +T ε = Tỹ, 1.13) whichcontainsy, y f, andỹasitsspecialcasesfordifferentchoicesoft. Anotherwellknown form of φ in 1.11) is the following target function discussed in [3, 4, 21, 26], which allows the prediction of both y f and Ey f ), τ = λy f +1 λ)ey f ) = X f Aα+λX f γ +λε f, 1.14) whereλ0 λ 1)isanon-stochasticscalarassigningweightstoactualandexpected values of y f. Clearly, the problem of predicting a linear combination of the fixed- and random-effects can be formulated as a special case of the general prediction problem on φ in 1.11). Thus, the simultaneous statistical inference of all unknown parameters in 1.11) is a comprehensive work, and will play prescriptive role for various special statistical inference problems under 1.1) from both theoretical and applied points of view. Note that there are 13 given matrices in 1.1) 1.6) and 1.11). Hence, statistical inference of φ in 1.11) is not easy task, we will encounter many tedious matrix operations for the given 13 matrices, as demonstrated in Section 3 below. The paper is organized as follows. Section 2 introduces the definitions of the BLUPs/BLUEs of all unknown parameters in 1.5) and 1.6), and gives a variety of matrix formulas needed to establish the BLUPs/BLUEs. Section 3 derives I) equations and formulas for the BLUPs/BLUEs of φ and its components; II) additive decompositions the BLUPs of φ with respect to its components; III) various formulas for the covariance matrix operations of the BLUPs/BLUEs of φ and its components. Section 4 formulates some further work on statistical inferences of LRMs, and describes how to do prediction analysis from a given linear equation of some random vectors. 2. Preliminaries. We first introduce definitions of the BLUPs/BLUEs of all unknown parameters in 1.5). A linear statistic Ly under 1.1), where L R s n, is

6 34 Y. Tian said to have the same expectation with φ in 1.11) if and only if ELy φ) = 0 holds. If there exists an L 0 y φ such that EL 0 y φ) = 0 and CovLy φ) CovL 0 y φ) s.t. ELy φ) = 0 2.1) hold, where CovLy φ) = E[Ly φ)ly φ) ] is the matrix mean squared error MMSE) of φ under ELy φ) = 0, then the linear statistic L 0 y is defined to be the BLUP of φ in 1.11), and is denoted by L 0 y = BLUPφ) = BLUPFα+Gγ +Hε+H f ε f ). 2.2) If G = 0, H = 0, and H f = 0, in 1.11), the L 0 y satisfying 2.1) is called the BLUE of Fα under 1.1), and is denoted by L 0 y = BLUEFα). 2.3) It should be pointed out that 2.1) can equivalently be converted to certain constrained matrix-valued function optimization problem in the Löwner partial ordering. This kind of equivalences between covariance matrix minimization problems and matrix-valued function minimization problems were firstly characterized in [17]; see also [19]. When A = I p and Σ 11 = 0, 1.1) is the well-known general linear fixedeffects model. In this instance, the work on predictions of new observations was widely considered since 1970s; see, e.g., [5, 7, 8, 9, 10, 11, 12]. On the other hand, 1.1) is a special case of general Linear Mixed-effects Models LMMs), and some previous results on BLUPs/BLUEs under LMMs can be found in the literature; see, e.g., [1, 6, 16, 18, 19]. The following lemma is well known; see [15]. Lemma 2.1. The linear matrix equation AX = B is consistent if and only if r[a, B] = ra), or equivalently, AA + B = B. In this case, the general solution of the equation can be written in the following parametric form X = A + B+I A + A)U, where U is an arbitrary matrix. We also need the following known formulas on ranks of matrices; see [13, 22]. Lemma 2.2. Let A R m n, B R m k, and C R l n. Then r[a, B] = ra)+re A B) = rb)+re B A), 2.4) A r = ra)+rcf C A ) = rc)+raf C ), 2.5) AA B r B = r[a, B]+rB). 2.6) 0

7 Predictions under LRM 35 If RA 1 ) RB 1 ), RA 2) RB 1 ), RA 2 ) RB 2 ) and RA 3) RB 2 ), then ra 1 B + 1 A B1 A 2) = r 2 rb A ), 0 B 2 A 3 2.7) ra 1 B + 1 A 2B + 2 A 3) = r B 1 A 2 0 rb 1 ) rb 2 ). 2.8) A The following result on analytical solutions of a constrained matrix-valued function optimization problem was given in [25]. Lemma 2.3. Let fl) = LC+D)MLC+D) s.t. LA = B, 2.9) where A R p q, B R n q, C R p m, and D R n m are given, M R m m is nnd, and the matrix equation LA = B is consistent. Then, there always exists a solution L 0 of L 0 A = B such that fl) fl 0 ) 2.10) holds for all solutions of LA = B. In this case, the matrix L 0 satisfying 2.10) is determined by the following consistent matrix equation L 0 [A, CMC A ] = [B, DMC A ], 2.11) while the general expression of L 0 and the corresponding fl 0 ) are given by L 0 = [B, DMC A ][A, CMC A ] + +U[A, CMC ], 2.12) fl 0 ) = KMK KMC A CMC A ) + CMK, 2.13) where K = BA + C+D, and U R n p is arbitrary. Many optimization problems in parametric statistical inferences, as demonstrated below, can be converted to the minimization of 2.9) in the Löwner partial ordering, while analytical solutions to these optimization problems in statistics can be derived from Lemma 2.3. More results on constrained) quadratic matrix-valued function optimization problems in the Löwner partial ordering can be found in [23, 24]. 3. Equations and formulas for BLUPs/BLUEs of all unknown parameters in LRM. In what follows, we assume that 1.1) is consistent, i.e., y R[XA, V 11 ] holds with probability 1. In this section, we first show how to derive the BLUP of the vector φ in 1.11), and then give some direct consequences under different assumptions. Lemma 3.1. The vector φ in 1.11) is predictable by y in 1.1) if and only if there exists L R s n such that LXA = F, or equivalently, R[XA) ] RF ). 3.1)

8 36 Y. Tian Proof. It is obvious that ELy φ) = 0 LXAα Fα = 0 for all α LXA = F. From Lemma 2.1, the matrix equation is consistent if and only if 3.1) holds. Theorem 3.2. Assume that φ in 1.11) is predictable by y in 1.1), namely, 3.1) holds, and let X, R 1, V ij, and J be as given in 1.5), 1.7), 1.9), 1.10), and 1.12). Also denote X = XA. Then ELy φ) = 0 and CovLy φ) = min L[ X, Covy) X ] = [F, Cov{φ,y} X ]. 3.2) The matrix equation in 3.2), called the fundamental equation for BLUP, is consistent as well under 3.1). In this case, the general solution of L and BLUPφ) can be written as BLUPφ) = Ly = [F, J ΣR 1 X ][ X, V 11 X ] + +U[ X, V 11 X ] ) y, 3.3) where U R s n is arbitrary. In particular, BLUPXβ) = [ X, [X, 0, 0] ΣR 1 X ][ X, V 11 X ] + +U 1 [ X, V 11 X ] ) y, 3.4) BLUPX f β) = [X f A, [X f, 0, 0] ΣR 1 X ][ X, V 11 X ] + +U 2 [ X, V 11 X ] ) y, 3.5) BLUEXAα) = [ X, 0][ X, V 11 X ] + +U 3 [ X, V 11 X ] ) y, 3.6) BLUEX f Aα) = [X f A, 0][ X, V 11 X ] + +U 4 [ X, V 11 X ] ) y, 3.7) BLUPXγ) = [0, [X, 0, 0] ΣR 1 X ][ X, V 11 X ] + +U 5 [ X, V 11 X ] ) y, 3.8) BLUPX f γ) = [0, [X f, 0, 0] ΣR 1 X ][ X, V 11 X ] + +U 6 [ X, V 11 X ] ) y, 3.9) BLUPε) = [0, [0, I n, 0] ΣR 1 X ][ X, V 11 X ] + +U 7 [ X, V 11 X ] ) y, 3.10) BLUPε f ) = [0, [0, 0, I nf ] ΣR 1 X ][ X, V 11 X ] + +U 8 [ X, V 11 X ] ) y, 3.11) where U i are arbitrary matrices of appropriate sizes, i = 1,2,...,8. Further, the following results hold. a) r[ X, V 11 X ] = r[ X, R 1 Σ], R[ X, V 11 X ] = R[ X, R 1 Σ], and R X) RV 11 X ) = {0}. b) L 0 is unique if and only if r[ X, V 11 ] = n. c) BLUPφ) is unique with probability 1 if and only if y R[ X, V 11 ], i.e., 1.1) is consistent.

9 Predictions under LRM 37 d) BLUPφ) satisfies Cov[BLUPφ)] = [F, J ΣR 1 X ][ X, V 11 X ] + V 11 [F, J ΣR 1 X ][ X, V 11 X ] + ), 3.12) Cov{BLUPφ), φ} = [F, J ΣR 1 X ][ X, V 11 X ] + R 1 ΣJ, 3.13) Covφ) Cov[BLUPφ)] = J ΣJ [F, J ΣR 1 X ][ X, V 11 X ] + V 11 [F, J ΣR 1 X ][ X, V 11 X ] + ), 3.14) Cov[φ BLUPφ)] = [F, J ΣR 1 X ][ X, V 11 X ] + R 1 J) Σ[F, J ΣR 1 X ][ X, V 11 X ] + R 1 J). Proof. By noticing that Ly φ = L Xα+LXγ +Lε Fα Gγ Hε H f ε f = L X F)α+LX G)γ +L H)ε H f ε f, we see that the covariance matrix of Ly φ can be written as CovLy φ) = Cov[LX G)γ +L H)ε H f ε f ] = [LX G, L H, H f ] Σ[LX G, L H, H f ] = L[X, I n, 0] [G, H, H f ]) ΣL[X, I n, 0] [G, H, H f ]) 3.15) = LR 1 J) ΣLR 1 J) := fl). 3.16) In this setting, we see from Lemma 2.3 that the first part of 3.2) is equivalent to finding a solution L 0 of the consistent matrix equation L 0 X = F such that fl) fl 0 ) s.t. L X = F 3.17) holds in the Löwner partial ordering. Further from Lemma 2.3, there always exists s solution L 0 of L 0 X = F such that 3.17) holds, and the L0 is determined by the matrix equation L 0 [ X, V 11 X ] = [F, J ΣR 1 X ], 3.18) establishing the matrix equation in 3.2). Solving the equation by Lemma 2.1 gives the L 0 in 3.3). Also from 2.13), fl 0 ) = CovL 0 y φ) = [F, J ΣR 1 X ][ X, V 11 X ] + R 1 J) Σ as required for 3.15). Result a) is well known. Results b) and c) follow directly from 3.3). [F, J ΣR 1 X ][ X, V 11 X ] + R 1 J),

10 38 Y. Tian Taking the covariance matrix of 3.3), and simplifying by 1.9) and RV 11 ) R[ X, V 11 X ] yield 3.12). From 1.11) and 3.3), Cov{BLUPφ), φ} = Cov{L 0 y, φ} = Cov { [F, J ΣR 1 X ][ X, V 11 X ] + R 1 [ β ε = [F, J ΣR 1 X ][ X, V 11 X ] + R 1 ΣJ, establishing 3.13). Eq. 3.14) follows from 1.12) and 3.12). ], J } β ε The matrix equation in 3.2) shows that the BLUPs/BLUEs of all unknown parameters in 1.5) generally depend on the covariance matrix of the observed random vector y, and the covariance matrix between φ and y. Because the matrix equation in 3.2) and the formulas in 3.3) 3.15) are presented by common operations of the given matrices and their generalized inverses, Theorem 3.2 and its proof in fact provide a standard procedure of handling matrix operations that occurr in the theory of BLUPs/BLUEs under general LRMs. From the fundamental equations and formulas in Theorem 3.2, we are now able to derive many new and valuable consequences on properties of BLUPs/BLUEs under various conditions. Corollary 3.3. Let φ be as given in 1.11). Then, the following results hold. a) If φ is predictable by y in 1.1), then Tφ is predictable by y in 1.1) as well for any matrix T R t s, and BLUPTφ) = TBLUPφ) 3.19) holds. b) If φ is predictable by y in 1.1), then Fα is estimable by y in 1.1) as well, and the BLUP of φ can be decomposed as the sum BLUPφ) = BLUEFα)+BLUPGγ)+BLUPHε)+BLUPH f ε f ), 3.20) and the following formulas for covariance matrices hold Cov{BLUEFα), BLUPGγ +Hε+H f ε f )} = 0, 3.21) Cov[BLUPφ)] = Cov[BLUEFα)]+Cov[BLUPGγ +Hε+H f ε f )]. 3.22) c) If α in 1.1) is estimable by y in 1.1), i.e., rxa) = k, then φ is predictable by y in 1.1) as well. In this case, the following BLUP/BLUE decomposition equalities BLUP α γ ε ε f = BLUEα) BLUPγ) BLUPε) BLUPε f ), 3.23) BLUPφ) = FBLUEα)+GBLUPγ)+HBLUPε)+H f BLUPε f ) 3.24) hold.

11 Predictions under LRM 39 Proof. The predictability of Tφ follows from R[XA) ] RF ) RF T ). Also from 3.3), BLUPTφ) = [TF, TJ ΣR 1 X ][ X, V 11 X ] + +U[ X, V 11 X ] ) y = T [F, J ΣR 1 X ][ X, V 11 X ] + +U 1 [ X, V 11 X ] ) y = TBLUPφ), where U = TU 1, establishing 3.19). where and Note that Ly in 3.3) can be decomposed as Ly = S 1 +S 2 +S 3 +S 4 )y = S 1 y+s 2 y+s 3 y+s 4 y, S 1 = [F, 0][ X, V 11 X ] + +U 1 [ X, V 11 X ], S 2 = [0, [G, 0, 0] ΣR 1 X ][ X, V 11 X ] + +U 2 [ X, V 11 X ], S 3 = [0, [0, H, 0] ΣR 1 X ][ X, V 11 X ] + +U 3 [ X, V 11 X ], S 4 = [0, [0, 0, H f ] ΣR 1 X ][ X, V 11 X ] + +U 4 [ X, V 11 X ], BLUEFα) = S 1 y, BLUPGγ) = S 2 y, BLUPHε) = S 3 y, BLUPH f ε f ) = S 4 y, establishing 3.20). We also obtain from 3.3) that Cov{BLUEFα), BLUPGγ +Hε+H f ε f )} { = Cov [F, 0][ X, V 11 X ] + +U 1 [ X, V 11 X ] )y, [0, J ΣR 1 X ][ X, V 11 X ] + +U 2 [ X, } V 11 X ] )y = [F, 0][ X, V 11 X ] + V 11 [0, J ΣR 1 X ][ X, V 11 X ] + ). 3.25) Applying 2.8) to 3.25) and simplifying, we obtain rcov{bluefα), BLUPGγ +Hε+H f ε f )}) = r [F, 0][ X, V 11 X ] + V 11 [0, J ΣR 1 X ][ X, V 11 X ] + ) ) X 0 0 X = r V 11 X R 1 ΣJ [ X, V 11 X ] V r[ X, V 11 X ] [F, 0] X 0 X R 1 ΣJ = r 0 X V 11 X 0 [ X, 0] V 11 0 [F, 0] 0 0 2r[ X, V 11 ]

12 40 Y. Tian = r 0 X X V 11 +r[ X V 11 X, X R 1 ΣJ ] 2r[ X, V 11 ] F 0 X X = r +r +r[ F X, V 11 X, R 1 ΣJ ] V 11 r X) 2r[ X, V 11 ] by 2.4) and 2.6)) = r[ X, V 11, R 1 ΣJ ] r[ X, V 11 ] = r[ X, V 11 ] r[ X, V 11 ] by Theorem 3.2a)) = 0, which implies that Cov{BLUEFα), BLUPGγ +Hε+H f ε f )} is a null matrix, establishing 3.21). Eq. 3.22) follows from 3.20) and 3.21). Eqs. 3.23), and 3.24) follow from 1.11), 3.19) and 3.20). Corollary 3.4. Let φ 1 = F 1 α+g 1 γ +H 1 ε+h f1 ε f, φ 2 = F 2 α+g 2 γ +H 2 ε+h f2 ε f, where F 1, F 2 R s k, G 1, G 2 R s p, H 1, H 2 R s n, and H f1, H f2 R s n f are known matrices, and assume that they are predictable by y in 1.1). Then, the following results hold. a) The sum φ 1 +φ 2 is predictable by y in 1.1), and the BLUP of φ 1 +φ 2 satisfies BLUPφ 1 +φ 2 ) = BLUPφ 1 )+BLUPφ 2 ). 3.26) b) BLUPφ 1 ) = BLUPφ 2 ) F 1 = F 2 and RR 1 ΣJ 1 R 1 ΣJ 2 ) R X), where J 1 = [G 1, H 1, H f1 ] and J 2 = [G 2, H 2, H f2 ]. Proof. Eq. 3.26) follows from 3.20). From Theorem 3.2, the two equations for the coefficient matrices of BLUPφ 1 ) = L 1 y and BLUPφ 2 ) = L 2 y are given by L 1 [ X, V 11 X ] = [F 1, J 1 ΣR 1 X ], L 2 [ X, V 11 X ] = [F 2, J 2 ΣR 1 X ]. The pair of matrix equations have a common solution if and only if X V11 X X V11 X r F 1 J 1 ΣR 1 X F 2 J 2 ΣR 1 X = r[ X, V 11 X, X, V 11 X ], 3.27) where by block elementary matrix operations X V11 X X V11 X r F 1 J 1 ΣR 1 X F 2 J 2 ΣR 1 X X V11 X 0 0 = r 0 0 F 2 F 1 J 2 ΣR 1 J 1 ΣR 1) X = r[ X, V 11 X ]+r[f 2 F 1, J 2 ΣR 1 J 1 ΣR 1) X ], r[ X, V 11 X, X, V 11 X ] = r[ X, V 11 X ].

13 Predictions under LRM 41 Hence, 3.27) is equivalent to [F 2 F 1, J 2 ΣR 1 J 1 ΣR 1) X ] = 0, which is further equivalent to b) by Lemma 2.1. As direct consequences of Theorem 3.2 and Corollary 3.3, we next give the BLUPs under 1.4). Corollary 3.5. Let R, R 1, and R 2 be as given in 1.7), and denote Z 1 = Cov{X f β, y} = X f Σ 11 X +X f Σ 12 and Z 2 = Cov{ε f, y} = Σ 31 X +Σ 32. Then, the following results hold. a) The future observation vector y f in 1.4) is predictable by y in 1.1) if and only if R X ) R[X f A) ]. In this case, BLUPy f ) = [X f A, V 21 X ][ X, V 11 X ] + +U[ X, V 11 ] ) y, 3.28) and Cov[BLUPy f )] = [X f A, V 21 X ][ X, V 11 X ] + V 11 [X f A, V 21 X ][ X, V 11 X ] + ), 3.29) Cov{BLUPy f ), y f } = [X f A, V 21 X ][ X, V 11 X ] + V 12, 3.30) Covy f ) Cov[BLUPy f )] = V 22 [X f A, V 21 X ][ X, V 11 X ] + V 11 [X f A, V 21 X ][ X, V 11 X ] + ), Cov[y f BLUPy f )] = [X f A, V 21 X ][ X, V 11 X ] + R 1 R 2 ) Σ 3.31) [X f A, V 21 X ][ X, V 11 X ] + R 1 R 2 ), 3.32) where U R nf n is arbitrary. b) X f β in 1.4) is predictable by y in 1.1) if and only if R X ) R[X f A) ]. In this case, BLUPX f β) = [X f A, Z 1 X ][ X, V 11 X ] + +U 1 [ X, V 11 X ] ) y, 3.33) and Cov[BLUPX f β)] = [X f A, Z 1 X ][ X, V 11 X ] + V 11 [X f A, Z 1 X ][ X, V 11 X ] + ), 3.34) Cov{BLUPX f β), X f β} = [X f A, Z 1 X ][ X, V 11 X ] + Z 1, 3.35) CovX f β) Cov[BLUPX f β)] = X f Σ 11 X f [X f A, Z 1 X ][ X, V 11 X ] + V 11 [X f A, Z 1 X ][ X, V 11 X ] + ), Cov[X f β BLUPX f β)] = [X f A, Z 1 X ][ X, V 11 X ] + R 1 [X f,0, 0]) Σ where U 1 R n f n is arbitrary. 3.36) [X f A, Z 1 X ][ X, V 11 X ] + R 1 [X f,0, 0]). 3.37)

14 42 Y. Tian c) ε f in 1.4) is always predictable by y in 1.1), and BLUPε f ) = Z 2 X V 11 X ) + +U 2 [ X, V 11 ] ) y, 3.38) Cov[BLUPε f )] = Cov{BLUPε f ), ε f } = Z 2 X V 11 X ) + Z 2, 3.39) Cov[ε f BLUPε f )] = Covε f ) Cov[BLUPε f )] where U 2 R n f n is arbitrary. = Σ 33 Z 2 X V 11 X ) + Z 2, 3.40) Finally, we give a group of fundamental decomposition equalities of the BLUPs of y, y f, and ỹ in 1.5). Corollary 3.6. The vector ỹ in 1.5) is predictable by y in 1.1) if and only if R[XA) ] R[X f A) ]. In this case, the following decomposition equalities always hold. y = BLUPy) = BLUEXAα) + BLUPXγ) + BLUPε), 3.41) BLUPy f ) = BLUEX f Aα)+BLUPX f γ)+blupε f ), 3.42) BLUPy) y BLUPỹ) = = 3.43) BLUPy f ) BLUPy f ) The additive decomposition equalities of the BLUPs in 3.41) and 3.42) are in fact built-in restrictions to BLUPs/BLUEs, which sufficiently demonstrate the key roles of BLUPs/BLUEs in statistical inferences of LRMs. Some previous discussions on built-in restrictions to BLUPs/BLUEs can be found; e.g., in [2, 14, 19, 20]. 4. Remarks. This paper established a general theory on BLUPs/BLUEs under LRM with original and future observations, and obtained many equations and formulas for calculating BLUPs/BLUEs of all unknown parameters in the LRM via solving a constrained quadratic matrix-valued function optimization problem in the Löwner partial ordering. Because the BLUPs/BLUEs in the previous sections are formulated by common operations of the given matrices and their generalized inverses in LRM, we can easily derive many new mathematical and statistical properties of the BLUPs/BLUEs under various assumptions. It is expected more interesting results on statistical inferences of LRMs can be derived from the equations and formulas in the previous sections. Here we mention a few: a) Derive closed-form formulas for calculating the ranks and inertias of the difference Cov[φ BLUPφ)] A, and use them to derive necessary and sufficient conditions for the following equality and inequalities Cov[φ BLUPφ)] = A A A, A, A) to hold under the assumptions in 1.1) 1.12), where A is symmetric matrix, say, A = Covφ) Cov[BLUPφ)].

15 Predictions under LRM 43 b) Establish necessary and sufficient conditions for the following decomposition equalities Cov[BLUPφ)] = Cov[BLUEFα)]+Cov[BLUPGγ)] +Cov[BLUPHε)]+Cov[BLUPH f ε f )], Cov[φ BLUPφ)] = Cov[BLUEFα)]+Cov[Gγ BLUPGγ)] + Cov[Hε BLUPHε)] + Cov[H f ε f BLUPH f ε f )] to hold respectively under the assumptions in 1.1) 1.12). c) Establish necessary and sufficient condition for BLUPφ) = φ like fixed points of matrix map) to hold. A special case is that BLUPTy) = Ty always holds for any matrix T, and is this case unique? It is expected that this type of work will bring deep understanding of statistical inferences of BLUPs/BLUEs from many new aspects. Finally, we give a general matrix formulation and derivation of BLUPs under random vector equations. Note that 1.1) is a special case of the following equation of random vectors A 1 y 1 +A 2 y 2 +A 3 y 3 = 0, 4.1) where A 1, A 2, and A 3 are three given matrices of appropriate sizes, and y 1, y 2 and y 3 are three random vectors of appropriate sizes satisfying E y 1 y 2 = b 1 b 2, Cov y 1 y 2 = Σ 11 Σ 12 Σ 13 Σ 21 Σ 22 Σ 23, 4.2) y 3 b 3 y 3 Σ 31 Σ 32 Σ 33 in which, b 1, b 2 and b 3 areconstant vectors,or fixed but unknown parametervectors. In this setting, taking expectation and covariance matrix of 4.1) yields A 1 b 1 +A 2 b 2 +A 3 b 3 = 0 and [A 1, A 2, A 3 ] Σ 11 Σ 12 Σ 13 Σ 21 Σ 22 Σ 23 A 1 A 2 = ) Σ 31 Σ 32 Σ 33 A 3 Assume now that one of y 1, y 2 and y 3 is observed, say, y 1, and the other two are unobservable, and we want to predict the vector L 2 y 2 + L 3 y 3 from 4.1) and 4.2). In this case, we let S = {L 1 y 1 L 2 y 2 L 3 y 3 EL 1 y 1 L 2 y 2 L 3 y 3 ) = 0}. 4.4) If there exists matrix L 1 such that [ CovL1 y 1 L 2 y 2 L 3 y 3 ) Cov L 1 y 1 L 2 y 2 L 3 y 3 ) s.t. L 1 y 1 L 2 y 2 L 3 y 3 S, 4.5)

16 44 Y. Tian the linear statistic L 1 y 1 is defined to the BLUP of L 2 y 2 + L 3 y 3 under 4.1) and is denoted by L 1 y 1 = BLUPL 2 y 2 +L 3 y 3 ). Note that CovL 1 y 1 L 2 y 2 L 3 y 3 ) = [L 1, L 2, L 3 ] Σ 11 Σ 12 Σ 13 Σ 21 Σ 22 Σ 23 L 1 L 2 := fl 1 ). Σ 31 Σ 32 Σ 33 L 3 Thus, 4.5) is equivalent to fl 1 ) f L 1 ) s.t. L 1 b 1 = L 2 b 2 +L 3 b ) By a similar approach as given in the previous sections, we can establish a group of equations as follows A 1 y 1 +BLUPA 2 y 2 )+BLUPA 3 y 3 ) = 0 if y 1 is obsevable, BLUPA 1 y 1 )+A 2 y 2 +BLUPA 3 y 3 ) = 0 if y 2 is obsevable, BLUPA 2 y 1 )+BLUPA 2 y 2 )+A 3 y 3 = 0 if y 3 is obsevable. Eqs. 4.1) 4.3) contain almost all linear structures occurred in regression analysis. It is expected that more valuable results can be obtained on 4.1) 4.6), which can serve as a mathematical foundation under various regression model assumptions. REFERENCES [1] R.B. Bapat. Linear Algebra and Linear Models. 3rd ed., Springer, [2] B.A. Brumback. On the built-in restrictions in linear mixed models with Application to smoothing spline analysis of variance. Commun. Statist. Theor. Meth., 39: , [3] A. Chaturvedi, S. Kesarwani and R. Chandra. Simultaneous prediction based on shrinkage estimator. Recent Advances in Linear Models and Related Areas, Essays in Honour of Helge Toutenburg, Springer, pp , [4] A. Chaturvedi, A.T.K. Wan and S.P. Singh. Improved multivariate prediction in a general linear model with an unknown error covariance matrix. J. Multivariate Analysis, 83: , [5] R. Christensen. Plane Answers to Complex Questions: The Theory of Linear Models. 3rd ed, Springer, New York, [6] D. Harville. Extension of the Gauss Markov theorem to include the estimation of random effects. Ann. Statist., 4: , [7] S.J. Haslett and S. Puntanen. Equality of BLUEs or BLUPs under two linear models using stochastic restrictions. Statist. Papers, 51: , [8] S.J. Haslett and S. Puntanen. Note on the equality of the BLUPs for new observations under two linear models. Acta Comm. Univ. Tartu. Math., 14:27 33, [9] S.J. Haslett and S. Puntanen. On the equality of the BLUPs under two linear mixed models. Metrika, 74: , [10] C.R. Henderson. Best linear unbiased estimation and prediction under a selection model. Biometrics, 31: , [11] J. Isotalo and S. Puntanen. Linear prediction sufficiency for new observations in the general GaussCMarkov model. Comm. Statist. Theor. Meth., 35: , [12] B. Jonsson. Prediction with a linear regression model and errors in a regressor. Internat. J. Forecast., 10: , [13] G. Marsaglia and G.P.H. Styan. Equalities and inequalities for ranks of matrices. Linear Multilinear Algebra, 2: , 1974.

17 Predictions under LRM 45 [14] R.A. McLean, W.L. Sanders and W.W. Stroup. A unified approach to mixed linear models. Amer. Statist., 45:54 64, [15] R. Penrose. A generalized inverse for matrices. Proc. Cambridge Phil. Soc., 51: , [16] D. Pfeffermann. On extensions of the Gauss Markov theorem to the case of stochastic regression coefficients. J. Roy. Statist. Soc. Ser. B, 46: , [17] C.R. Rao. A lemma on optimization of matrix function and a review of the unified theory of linear estimation. Statistical Data Analysis and Inference, Y. Dodge ed.), North Holland, pp , [18] G.K. Robinson. That BLUP is a good thing: the estimation of random effects. Statist. Science, 6:15 32, [19] S.R. Searle. The matrix handling of BLUE and BLUP in the mixed linear model. Linear Algebra Appl., 264: , [20] S.R. Searle. Built-in restrictions on best linear unbiased predictors BLUP) of random effects in mixed models. Amer. Statist., 51:19 21, [21] Shalabh. Performance of Stein-rule procedure for simultaneous prediction of actual and average values of study variable in linear regression models. Bull. Internat. Stat. Instit., 56: , [22] Y. Tian. More on maximal and minimal ranks of Schur complements with applications. Appl. Math. Comput., 152: , [23] Y. Tian. Solving optimization problems on ranks and inertias of some constrained nonlinear matrix functions via an algebraic linearization method. Nonlinear Analysis, 75: , [24] Y. Tian. Formulas for calculating the extremum ranks and inertias of a four-term quadratic matrix-valued function and their applications. Linear Algebra Appl., 437: , [25] Y. Tian. A new derivation of BLUPs under random-effects model. Metrika, DOI: /s [26] H. Toutenburg. Prior Information in Linear Models. Wiley, New York, 1982.