Linear Mixed-Effects Models. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

Transcription

1 Linear Mixed-Effects Models Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

2 The Linear Mixed-Effects Model y = Xβ + Zu + e X is an n p design matrix of known constants β R p is an unknown parameter vector Z is an n q matrix of known constants u is a q 1 random vector e is an n 1 vector of random errors Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

3 The Linear Mixed-Effects Model y = Xβ + Zu + e The elements of β are considered to be non-random and are called fixed effects. The elements of u are random variables and are called random effects. The elements of the error vector e are always considered to be random variables. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

4 Because the model includes both fixed and random effects (in addition to the random errors), it is called a mixed-effects model or, more simply, a mixed model. The model is called a linear mixed-effects model because (as we will soon see) E(y u) = Xβ + Zu, a linear function of fixed and random effects. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

5 We assume that E(e) = 0 Var(e) = R E(u) = 0 Var(u) = G Cov(e, u) = 0. It follows that opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

6 E(y) = E(Xβ + Zu + e) = Xβ + ZE(u) + E(e) = Xβ Var(y) = Var(Xβ + Zu + e) = Var(Zu + e) = Var(Zu) + Var(e) = ZVar(u)Z + R = ZGZ + R Σ. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

7 We usually consider the special case in which [ ] ([ ] [ u 0 G 0 N, e 0 0 R ]) = y N(Xβ, ZGZ + R). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

8 The conditional moments, given the random effects, are E(y u) = Xβ + Zu Var(y u) = R. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

9 Example 1 Suppose a study was conducted to compare two plant genotypes (genotype 1 and genotype 2). Suppose 10 seeds (5 of genotype 1 and 5 of genotype 2) were planted in a total of 4 pots. Suppose 3 genotype 1 seeds were planted in one pot, and the other 2 genotype 1 seeds were planted in another pot. Suppose the same planting strategy was used when planting genotype 2 seeds in the other two pots. The seeds germinated and emerged from the soil as seedlings. After a four-week growing period, each seedling was dried and weighed. Let y ijk denote the weight for genotype i, pot j, seedling k. Provide a linear mixed-effects model for the dry-weight data. Determine y, X, β, Z, u, G, R, and Var(y). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

10 Consider the model y ijk = µ + γ i + p ij + e ijk p 11, p 12, p 21, p 22 i.i.d. N(0, σ 2 p) independent of the e ijk terms, which are assumed to be iid N(0, σ 2 e). This model can be written in the form y = Xβ + Zu + e, where Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

11 y = y 111 y 112 y 113 y 121 y 122 y 211 y 212 y 213 y 221 y 222, X = µ, β = γ 1 γ 2, Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

12 Z = , u = p 11 p 12 p 21 p 22, e = e 111 e 112 e 113 e 121 e 122 e 211 e 212 e 213 e 221 e 222. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

13 G = Var(u) = Var([p 11, p 12, p 21, p 22 ] ) = σ 2 pi 4 4 R = Var(e) = σ 2 ei Var(y) = ZGZ + R = Zσ 2 piz + σ 2 ei = σ 2 pzz + σ 2 ei. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

14 ZZ = Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

15 Thus, Var(y) = σ 2 pzz + σ 2 ei is a block diagonal matrix. The first block is Var y 111 y 112 y 113 = σ 2 p + σ 2 e σ 2 p σ 2 p σ 2 p σ 2 p + σ 2 e σ 2 p σ 2 p σ 2 p σ 2 p + σ 2 e. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

16 Note that Var(y ijk ) = σp 2 + σe 2 i, j, k. Cov(y ijk, y ijk ) = σp 2 i, j, and k k. Cov(y ijk, y i j k ) = 0 if i i or j j. Any two observations from the same pot have covariance σp. 2 Any two observations from different pots are uncorrelated. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

17 Note that Var(y) may be written as σ 2 ev where V is a block diagonal matrix with blocks of the form 1 + σp/σ 2 e 2 σp/σ 2 e 2... σp/σ 2 e 2 σ p/σ 2 e σp/σ 2 e 2... σp/σ 2 e σ 2 p/σ 2 e σ 2 p/σ 2 e σ 2 p/σ 2 e Thus, if σ 2 p/σ 2 e were known, we would have the Aitken Model. y = Xβ + ε, where ε = Zu + e N(0, σ 2 V), σ 2 σ 2 e. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

18 Thus, if σ 2 p/σ 2 e were known, we would use GLS to estimate any estimable Cβ by Cˆβ GLS = C(X V 1 X) X V 1 y. However, we seldom know σ 2 p/σ 2 e or, more generally, Σ or V. For the general problem where Var(y) = Σ is an unknown positive definite matrix, we can rewrite Σ as σ 2 V, where σ 2 is an unknown positive variance and V is an unknown positive definite matrix. As in our simple example, each entry of V is usually assumed to be a known function of few unknown parameters. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

19 Thus, our strategy for estimating an estimable Cβ involves estimating the unknown parameters in V to obtain CˆβĜLS = C(X ˆV 1 X) X ˆV 1 y. In general, CˆβĜLS = C(X ˆV 1 X) X ˆV 1 y is an nonlinear estimator that is an approximation to Cˆβ GLS = C(X V 1 X) X V 1 y, which would be the BLUE of Cβ if V were known. opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

20 In special cases, CˆβĜLS may be a linear estimator. For example, if there exists a matrix Q such that VX = XQ, then we know that Cˆβ GLS = Cˆβ and CˆβĜLS = Cˆβ, which is a linear estimator of Cβ. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

21 However, even for our simple example involving seedling dry weight, CˆβĜLS is a nonlinear estimator of Cβ for C = [1, 1, 0] Cβ = µ + γ 1, C = [1, 0, 1] Cβ = µ + γ 2, and C = [0, 1, 1] Cβ = γ 1 γ 2. Confidence intervals and tests for these estimable functions are not exact. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

22 In our simple example involving seedling dry weight, there was only one random factor (pot). When there are m random factors, we can partition Z and u as Z = [Z 1,..., Z m ] and u = u 1. u m, where u j is the vector of random effects associated with factor j (j = 1,..., m). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

23 We can write Zu as [Z 1,..., Z m ] u 1. u m m = Z j u j. j=1 We often assume that all random effects (including random errors) are mutually independent and that the random effects associated with the jth random factor have variance σ 2 j (j = 1,..., m). Under these assumptions, Var(y) = ZGZ + R = m σj 2 Z j Z j + σei. 2 j=1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

24 Example 2 Consider an experiment involving 4 litters of 4 animals each. Suppose 4 treatments are randomly assigned to the 4 animals in each litter. Suppose we obtain two replicate muscle samples from each animal and measure the response of interest for each muscle sample. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

25 Let y ijk denote the kth measure of the response for the animal from litter j that received treatment i (i = 1, 2, 3, 4; j = 1, 2, 3, 4; k = 1, 2) Suppose where y ijk = µ + τ i + l j + a ij + e ijk, β = [µ, τ 1, τ 2, τ 3, τ 4 ] R 5 is an unknown vector of fixed parameters, u = [l 1, l 2, l 3, l 4, a 11, a 21, a 31, a 41, a 12,..., a 34, a 44 ] is a vector of random effects, and opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

26 e = [e 111, e 112, e 212,..., e 411, e 412,..., e 441, e 442 ] is a vector of random errors. With y = [y 111, y 112, y 212,..., y 411, y 412,..., y 441, y 442 ], we can write the model as a linear mixed-effects model y = Xβ + Zu + e, where opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

27 X = , Z = The matrix above repeated three more times Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

28 We can write less and be more precise using Kronecker product notation. X = [ 1 8 1, I ], Z = [ I , I ]. In this experiment, we have two random factors: litter and animal. We can partition our random effects vector u into a vector of litter effects and a vector of animal effects: Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

29 [ ] l u =, l = a l 1 l 2 l 3 l 4 a 11 a 44 a 21 a 31, a = a 41, a 12. We make the usual assumption that [ ] ([ ] [ ]) l 0 σ 2 u = N, l I 0, a 0 0 σai 2 where σl 2, σ2 a R + are unknown parameters. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

30 We can partition Z = [ I = [Z l, Z a ]. 2 1 ] We have Zu = [Z l, Z a ] [ ] l a = Z l l + Z a a and Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

31 Var(Zu) = ZGZ = [Z l, Z a ] [ ] [ ] σ 2 l I 0 Z l 0 σ 2 ai Z a = Z l (σ 2 l I)Z l + Z a(σ 2 ai)z a = σ 2 l Z lz l + σ2 az a Z a = σ 2 l I σ2 a I Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

32 We usually assume that all random effects and random errors are mutually independent and that the errors (like the effects within each factor) are identically distributed: l 0 σl 2 a N 0, I σai 2 0. e σei 2 The unknown variance parameters σl 2, σ2 a, σe 2 R + are called variance components. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

33 In this case, we have R = Var(e) = σ 2 ei. Thus, Var(y) = ZGZ + R = σl 2 Z lz l + σ2 az a Z a + σei. 2 This is a block diagonal matrix with a block as follows. (To get a block to fit on one slide, let l = σ 2 l, a = σ2 a, e = σ 2 e). opyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34

34 l + a + e l + a l l l l l l l + a l + a + e l l l l l l l l l + a + e l + a l l l l l l l + a l + a + e l l l l l l l l l + a + e l + a l l l l l l l + a l + a + e l l l l l l l l l + a + e l + a l l l l l l l + a l + a + e Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 34