Outline for today. Two-way analysis of variance with random effects

Transcription

1 Outline for today Two-way analysis of variance with random effects Rasmus Waagepetersen Department of Mathematics Aalborg University Denmark Two-way ANOVA using orthogonal projections March 4, / 28 2 / 28 Two-way ANOVA Consider two factors: T (treatment) and P (plot) with number of levels d T and d P. Moreover let P T be the cross-factor (has d P d T levels - one for each combination of levels of P and T ). Assume P T is balanced with n P T = m observations for each level. Then P and T balanced too with numbers of observations n P = md T and n T = md P for each level. Similar to one-way anova define: V 0 = L 0 V T = L T V 0 V P = L P V 0 Since P T balanced it follows that P T P P = P P P T = P 0 which implies Q T Q p = 0 where Q T = P T P 0 and Q P = P P P 0. Hence V T and V P are orthogonal and L P + L T = V 0 V P V T. Define further Model with random P and P T effects (e.g. to account for soil variation) y ptr = ξ + β t + U p + U pt + ɛ ptr p = 1,..., d P, t = 1,..., d T, r = 1,..., m V P T = L P T (L P + L T ) Orthogonal decomposition: V I = R n L P T In vector form: R n = V 0 V P V T V P T V I where µ L T = spanx T. y = µ + Z P U P + Z P T U P T + ɛ 3 / 28 Dimensions of V spaces: f 0 = 1, f P = d P 1, f T = d T 1, f P T = d P d T d P d T + 1 = (d P 1)(d T 1), f I = n d P d T. 4 / 28

2 Orthogonal projections on V spaces: Q 0 = P 0, Q P = P P Q 0, Q T = P T Q 0, Q P T = P P T Q P Q T Q 0 and Q I = I P P T. Covariance structure: CovY = σ 2 P n PP P + σ 2 P T n P T P P T + σ 2 I Q s are orthogonal projections on Ṽ P = V 0 + V P Ṽ P T = V P T + Q T Ṽ I = V I where and = λ P Q P + λ P T Q P T + σ 2 Q I λ I = σ 2 λ P T = σ 2 + n P T σp T 2 λ P = σ 2 + n P T σp T 2 + n PσP 2 This corresponds to orthogonal decomposition R n = ṼP ṼP T ṼI based on model without systematic effects ( V -spaces aggregated into Ṽ -spaces). Q P = Q 0 + Q P = P P Q P T = Q P T + Q T Q I = Q I 5 / 28 6 / 28 Orthogonal decomposition of data vector: Y = Q 0 Y + Q P Y + Q T Y + Q P T Y + Q I Y = Q P Y + Q P T Y + Q I Y Decomposition of mean vector µ L T = V 0 V T : µ = Q P µ + Q P T µ + Q I µ = Q 0 µ + Q T µ + 0 = µ 0 + µ T where µ 0 L 0 and µ T V T. Note: one-to-one correspondence between µ and (µ 0,µ T ) (reparametrization) 7 / 28 As for one-way anova Y is decomposed into independent normal vectors Q P Y N(µ 0, λ P QP ) QP T Y N(µ T, λ P T QP T ) Q I Y N(0, λ I QI ) Density for Y factorizes into product of densities for Q P Y, Q P T Y and Q I Y. Hence ˆµ 0 = P 0 Q P Y = P 0 Y = Ȳ 1 n ˆµ T = Q T Q P T Y = Q T Y ˆµ = ˆµ 0 + ˆµ T = P T Y ˆλ P = Q P Y P 0 Y 2 /d P = Q P Y 2 /d P ˆλ P T = Q P T Y Q T Y 2 /(d P d T d P ) = Q P T Y 2 /(d P d T d P ) ˆλ I = ˆσ 2 = Q I Y 2 /(n d P d T ) NB: since V P and V P T have dimensions f P = d P 1 and f P T = (d P 1)(d T 1) we often use these denominators instead of d P and d P d T d P for λ P and λ P T (REML). Note: MLE for µ is identical to MLE in the linear model with µ L T and no random effects. 8 / 28

3 Strata Crucial common point one- and two-way anova models considered: we are able to decompose covariance matrix as linear combination of scaled projections on orthogonal Ṽ -subspaces where coefficients are in one-to-one correspondence with the variance components (the variances of random effects). This enables factorization of likelihood based! Fixed factors assigned to strata corresponding to Ṽ spaces for the random factors Rule: fixed factor F belongs to B strata (B random factor) if B is the coarsest random factor which is finer than F (assuming that we work with designs where this rule gives unique allocation of each fixed factors to one random factor). NB: F is finer than G (or G coarser than F ) if levels of G can be obtained by merging levels of F. We then write G F. 9 / 28 Structure diagram I PxT P T Arrow from F to G if G is coarser than F and there is no intermediate factor which is coarser than F and finer than G. Note: three strata (black, green and red) corresponding to random factors I, P T and P. NB: here we can not have both P and T random unless O random too (if O systematic no unique allocation to random factor strata). Also want to respect hierarchical principle so P T can not be fixed if P random. O 10 / 28 Paper pulp example: two-way analysis of variance without treatment effect P: card board with d P = 5. T: position within cardboard (not a treatment) with d T = 4. Number of replications m = 4. Model: Variances σ 2 P, σ2 P T y ptr = ξ + U p + U pt + ɛ ptr and σ2 We can use exactly the same orthogonal decomposition as before. Only difference is that Q T µ = Q T 1 n ξ = 0 which means Q P T Y N(0, λ P T Q P T ) and REML and ML estimates for λ P T coincide and are equal to ˆλ P T = Q P T Y 2 /(d P d T d P ) NB: in ANOVA table (next slide), SSP= Q P Y 2, SST= Q T Y 2, SSPT= Q P T Y 2 and SSE= Q I Y / 28 ANOVA table Analysis of Variance Table Response: Reflektans factor(sted) < 2.2e-16 *** #SST factor(pap.nr.) < 2.2e-16 *** #SSP factor(sted):factor(pap.nr.) < 2.2e-16 *** #SSPT Residuals #SSE ˆσ 2 = ˆλ P T = ( )/15 = ˆλ P = /5 = (or ˆλ P = /4 = = ). 12 / 28

4 Using lmer and ML By relations λ P T = σ 2 + n P T σ 2 P T λ P = σ 2 + n P T σ 2 P T + n Pσ 2 P we obtain ˆσ P 2 = ( )/16 = ˆσ P T 2 = ( )/4 = (or = ( )/16 = ) ˆσ 2 P Recall: balanced design required - and difficult to remember rules for calculating variance components. > out2=lmer(reflektans~(1 StedPap.nr.)+(1 Pap.nr.),REML=F) > summary(out2) Linear mixed model fit by maximum likelihood Formula: Reflektans ~ (1 StedPap.nr.) + (1 Pap.nr.) AIC BIC loglik deviance REMLdev Random effects: Groups Name Variance Std.Dev. StedPap.nr. (Intercept) e Pap.nr. (Intercept) e Residual e Number of obs: 80, groups: StedPap.nr., 20; Pap.nr., 5 13 / / 28 Using lmer and REML > out2=lmer(reflektans~(1 StedPap.nr.)+(1 Pap.nr.)) > summary(out2) Linear mixed-effects model fit by REML Formula: Reflektans ~ (1 StedPap.nr.) + (1 Pap.nr.) AIC BIC loglik MLdeviance REMLdeviance Random effects: Groups Name Variance Std.Dev. StedPap.nr. (Intercept) e Pap.nr. (Intercept) e Residual e number of obs: 80, groups: StedPap.nr., 20; Pap.nr., 5 Explanation of Reflektans~(1 StedPap.nr.)+(1 Pap.nr.)): no fixed formula: intercept always included as default (1 StedPap.nr.) random intercepts for groups identified by variable StedPap.nr. (1 Pap.nr.) random intercepts for groups identified by variable Pap.nr. random effects specified by different terms independent. Note REML and ML estimates for σ 2 and σp T 2 rounding error) coincide (up to 15 / / 28

5 A more complicated example: gene-expression Gene (DNA string) composed of substrings (exons) which may be more or less expressed according to treatment. Note: P E T = I is balanced so every sub-factor (E, E T etc.) is balanced. This implies orthogonal decomposition (see a following slide): Expression measured as intensities on micro-array (chip). One chip pr. patient-treatment. R 160 = V 0 V P V E V T V P T V E T V P E V I Structure diagram with random factors P, P V, I : P PxT Factors: E (exon 8 levels), P (patient, 10 levels), T (treatment, 2 levels) Y : vector of intensities (how much is exon expressed). I ExT T E O Model: y ept = ξ + α e + β t + γ et + U p + U pt + ɛ ept U pt and U p random chip and patient effects. PxE Decomposition with respect to random factors: R 160 = Ṽ P Ṽ P T Ṽ I Main question: are exons differentially expressed - i.e. are γ et 0 or not (we will return to this question later) 17 / 28 where Ṽ P = V 0 V P, Ṽ P T = V T V P T and Ṽ I = V E V E T V P E V I. 18 / 28 Decomposition of CovY : CovY = n P σ 2 P P P+n P T σ 2 P T P P T +σ 2 I = λ P Q P +λ P T Q P T +λ I Q I Decomposition of µ: µ = Q P µ + Q P T µ + Q I µ = Q 0 µ + Q T µ + Q E µ + Q E T µ As before decomposition of Y into independent Gaussian vectors: Q P Y N(Q 0 µ, λ P QP ) QP T Y N(Q T µ, λ P T QP T ) Q I Y N ( (Q E + Q E T )µ, λ I Q I ) Orthogonality of V -spaces We use results of exercise 5 to deduce that P P E P E T = P E Q P P E T = P P P E T P 0 P E T = P 0 P 0 = 0 The second equality shows that V P and V E T are orthogonal. Further: Q P E = P P E Q P Q E Q 0 Q E T = P P E Q E Q T Q 0 Using the result in the upper equation and pairwise orthogonality of Q P, Q E, Q T, Q 0 we get Q P E Q E T = 0 Thus V P E and V E T are orthogonal too. Orthogonality of V P T, V P and V T was shown for two-way ANOVA. 19 / 28 Proceeding this way all V spaces orthogonal. 20 / 28

6 Anova table: > fit1=lm(intensity~treat*factor(exon)+factor(patient)+ factor(patient):treat,data=gene1) > anova(fit1) Analysis of Variance Table Response: intensity treat *** factor(exon) < 2.2e-16 *** factor(patient) e-09 *** treat:factor(exon) treat:factor(patient) *** Residuals ˆσ 2 = ˆλ I = λ P T = 8.19/9 = 0.91 λ P = /9 = ˆσ P T 2 = ( )/8 = ˆσ P 2 = ( )/16 = / 28 Using lmer: > fit1=lmer(intensity~treat*factor(exon)+ (1 patient)+(1 pattreat),data=gene1) > summary(fit1) Linear mixed model fit by REML Data: gene1 AIC BIC loglik deviance REMLdev Random effects: Groups Name Variance Std.Dev. pattreat (Intercept) patient (Intercept) Residual Number of obs: 160, groups: repltreat, 20; patient, / 28 With aov() > fit1=aov(intensity~treatment*factor(exon)+error(factor(patient)+factor(patient):t > summary(fit1) Error: factor(patient) Residuals Error: factor(patient):treatment treatment Residuals Signif. codes: 0 *** ** 0.01 * Error: Within factor(exon) <2e-16 *** treatment:factor(exon) Residuals Signif. codes: 0 *** ** 0.01 * Conveniently organizes factors into strata! K-way ANOVA Assume K factors F 1,..., F K so that F 1... F K balanced. Let D be the set of 0, I, F 1,..., F K and all cross-combinations of F 1,..., F K. Then, in analogy with three-way, all V spaces orthogonal. Let B D be the set of factors with random effects. We then have and we want Σ = B B σ 2 B n BP B and P B = P B = B B:B B F D:F B Q F Q B (1) where Q B orthogonal projections on some spaces Ṽ B where R n = B B Ṽ B. 22 / / 28

7 K-way ANOVA continued Given (1) we have required decomposition of Σ into sum of scaled orthogonal projections: Σ = B B σ 2 B n BP B = B B λ B QB where λ B = B B:B B n B σ 2 B. A sufficient (and necessary) condition for (1) is that: for all F D there exists a B B such that F B and B B for all other B with F B. (this must be checked for a given model. Note this implies I B and that B = B(F ) is unique). K-way ANOVA continued Thus we can again obtain decomposition Y of into independent Ỹ B = Q B Y, B B. See also more details in note Analysis of variance using orthogonal projections. Even more general set-up can be found in Jesper Møller: Centrale statistiske modeller og likelihood baserede metoder. Then we define Ṽ B = V F and Q B = F D:B(F )=B F D:B(F )=B Each F D belongs to precisely one Ṽ B so R n = B B Ṽ B. Q F 25 / / 28 Exercises 1. Check that P T P P = P Check that L T + L P = V 0 V P V T. 3. Write down factorization of likelihood for balanced two-way. Derive estimates of mean and variance parameters. 4. Install the R-package faraway which contains the data set penicillin. The response variable is yield of penicillin for four different production processes (the treatment ). The raw material for the production comes in batches ( blends ). The four production processes were applied to each of the 5 blends. Fit anova models with production process as a fixed factor and blend as random factor. Try to use both the anova table and lmer. 5. In a balanced three-way design with factors F 1, F 2, and F 3 show that P F1 F 2 P F2 F 3 = P F2 P F1 F 2 P F3 = P 0 Explain how this can be generalized to the following result: P i A F i P i B F i = P i A B F i for a K-way balanced design with factors F i, i = 1,..., K and A, B {1,..., K} (taking P i F i = P 0 ). 27 / / 28