Factorial and Unbalanced Analysis of Variance

Transcription

1 Factorial and Unbalanced Analysis of Variance Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 1

2 Copyright Copyright 2017 by Nathaniel E. Helwig Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 2

3 Outline of Notes 1) Balanced Two-Way ANOVA: Model Form & Assumptions Least-Squares Estimation Basic Inference Hypertension Example (pt 1) Multiple Comparisons Hypertension Example (pt 2) 2) Balanced Three-Way ANOVA: Model Form & Estimation Hypertension Example (pt 3) 3) Unbalanced ANOVA Models: Overview of problem Types of sums-of-squares Hypertension Example (pt 4) Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 3

4 Balanced Two-Way ANOVA Balanced Two-Way ANOVA Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 4

5 Balanced Two-Way ANOVA Model Form & Assumptions Two-Way ANOVA Model (cell means form) The Two-Way Analysis of Variance (ANOVA) model has the form y ijk = µ jk + e ijk for i {1,..., n jk }, j {1,..., a}, and k {1,..., b} where y ijk R is real-valued response for i-th subject in factor cell (j, k) µ jk R is real-valued population mean for factor cell (j, k) e ijk iid N(0, σ 2 ) is a Gaussian error term n jk is number of subjects in cell (j, k) and n = a j=1 b k=1 n jk (note: n jk = n j, k in balanced two-way ANOVA) a and b are number of levels for first and second factors Implies that y ijk ind N(µ jk, σ 2 ). Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 5

6 Balanced Two-Way ANOVA Model Form & Assumptions Two-Way ANOVA Model (effect coding) Using effect coding, the mean for factor cell (j, k) has the form µ jk = µ + α j + β k + (αβ) jk for j {1,..., a} and k {1,..., b} where µ is overall population mean α j is main effect of first factor such that a j=1 α j = 0 β k is main effect of second factor such that b k=1 β k = 0 (αβ) jk is interaction effect such that a j=1 (αβ) jk = 0 k and b k=1 (αβ) jk = 0 j Set (αβ) jk = 0 j, k to fit an additive model. Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 6

7 Balanced Two-Way ANOVA Model Form & Assumptions Two-Way ANOVA Model (matrix form) In matrix form, the two-way ANOVA model is y = Xb + e where 1 x 11 x 1(a 1) z 11 z 1(b 1) x 11 z 11 x 1(a 1) z 11 x 11 z 1(b 1) x 1(a 1) z 1(b 1) 1 x 21 x 2(a 1) z 21 z 2(b 1) x 21 z 21 x 2(a 1) z 21 x 21 z 2(b 1) x 2(a 1) z 2(b 1) 1 x 31 x 3(a 1) z 31 z 3(b 1) x 31 z 31 x 3(a 1) z 31 x 31 z 3(b 1) x 3(a 1) z 3(b 1) X = x n1 x n(a 1) z n1 z n(b 1) x n1 z n1 x n(a 1) z n1 x n1 z n(b 1) x n(a 1) z n(b 1) b = ( µ α 1 α a 1 β 1 β b 1 (αβ) 11 (αβ) (a 1)1 (αβ) 1(b 1) (αβ) (a 1)(b 1) ) where X has 1 + (a 1) + (b 1) + (a 1)(b 1) = ab columns 1 if i-th observation is in j-th level of first factor x ij = 1 if i-th observation is in a-th level of first factor 0 otherwise 1 if i-th observation is in k-th level of second factor z ik = 1 if i-th observation is in b-th level of second factor 0 otherwise i {1,..., n} and additional subscripts on y and e are dropped Implies that y N(Xb, σ 2 I n ). Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 7

8 Balanced Two-Way ANOVA Model Form & Assumptions Two-Way ANOVA Model (assumptions) The fundamental assumptions of the two-way ANOVA model are: 1 x ij, z ik and y i are observed random variables (known constants) 2 e i iid N(0, σ 2 ) is an unobserved random variable 3 µ jk are unknown constants 4 (y i x ij, z ik ) ind N(µ jk, σ 2 ) note: homogeneity of variance Interpretation of µ jk depends on model form Additive: µ jk = µ + α j + β k Interaction: µ jk = µ + α j + β k + (αβ) jk Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 8

9 Balanced Two-Way ANOVA Ordinary Least-Squares Least-Squares Estimation We want to find the effect estimates (i.e., ˆµ, ˆα j, ˆβ k, and ( αβ) ˆ jk terms) that minimize the ordinary least squares criterion n b a jk SSE = (y ijk µ α j β k (αβ) jk ) 2 k=1 j=1 i=1 If n jk = n j, k the least-squares estimates have the form ˆµ = 1 b a k=1 j=1 abn ( 1 ˆα j = b k=1 bn ( 1 ˆβ k = a j=1 an ( 1 n i=1 n y ijk ( αβ) ˆ jk = n i=1 y ijk n i=1 y ijk n i=1 y ijk = ȳ which implies that ŷ ijk = ȳ jk for all (i, j, k). ) ˆµ = ȳ j ȳ ) ˆµ = ȳ k ȳ ) ˆµ ˆα j ˆβ k = ȳ jk ȳ j ȳ k + ȳ Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 9 Proof

10 Balanced Two-Way ANOVA Least-Squares Estimation Fitted Values and Residuals Form of fitted values depends on fit model: Additive: ˆµ jk = ȳ j + ȳ k ȳ Interaction: ˆµ jk = ȳ jk Residuals have the form ê ijk = y ijk ˆµ jk where form of ˆµ jk depends on fit model (additive versus interaction). Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 10

11 Balanced Two-Way ANOVA ANOVA Sums-of-Squares Basic Inference In balanced two-way ANOVA model with interaction: SST = b a n k=1 j=1 i=1 (y ijk ȳ ) 2 df = abn 1 SSR = n b a k=1 j=1 (ȳ jk ȳ ) 2 df = ab 1 SSE = b a n k=1 j=1 i=1 (y ijk ȳ jk ) 2 df = abn ab In balanced two-way ANOVA model with no interaction: SST = b a n k=1 j=1 i=1 (y ijk ȳ ) 2 df = abn 1 SSR = n b a k=1 j=1 ([ȳ j + ȳ k ȳ ] ȳ ) 2 df = a + b 2 SSE = b a n k=1 j=1 i=1 (y ijk [ȳ j + ȳ k ȳ ]) 2 df = abn (a + b 1) Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 11

12 Balanced Two-Way ANOVA Basic Inference Partitioning the Variance From MLR notes we know that SST = SSR + SSE. If n jk = n j, k can partition SSR = SSA + SSB + SSAB where SSA = bn a j=1 ˆα2 j df = a 1 SSB = an b k=1 ˆβ 2 k df = b 1 SSAB = n b k=1 a j=1 ( ˆ αβ) 2 jk df = (a 1)(b 1) Implies that SSR = SSA + SSB for additive model (if n jk = n j, k). Proof Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 12

13 Balanced Two-Way ANOVA Basic Inference Extended ANOVA Table and F Tests We typically organize the SS information into an ANOVA table: Source SS df MS F p-value SSR n b k=1 a j=1 (ȳ jk ȳ ) 2 ab 1 MSR F p SSA bn aj=1 (ȳ j ȳ ) 2 a 1 MSA F a p a SSB an bk=1 (ȳ k ȳ ) 2 b 1 MSB F b p b SSAB n bk=1 aj=1 (ȳ jk ȳ j ȳ k + ȳ ) 2 (a 1)(b 1) MSAB Fab p ab SSE b a n k=1 j=1 i=1 (y ijk ȳ jk ) 2 ab(n 1) MSE SST b a n k=1 j=1 i=1 (y ijk ȳ ) 2 abn 1 MSR = SSR SSA SSB, MSA =, MSB = ab 1 a 1 b 1, MSAB = SSAB (a 1)(b 1), MSE = SSE ab(n, 1) F = MSR MSE F ab 1,ab(n 1) and p = P(F ab 1,ab(n 1) > F ), Fa = MSA MSE F a 1,ab(n 1) and pa = P(F a 1,ab(n 1) > Fa ), Fb = MSB MSE F b 1,ab(n 1) and pb = P(F b 1,ab(n 1) > Fb ), Fab = MSAB MSE F (a 1)(b 1),ab(n 1) and pab = P(F (a 1)(b 1),ab(n 1) > Fab ), Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 13

14 Balanced Two-Way ANOVA Basic Inference ANOVA Table F Tests F and p -value are testing H 0 : α j = β k = (αβ) jk = 0 j, k versus H 1 : ( j, k {1,..., a} {1,..., b})(α j = β k = (αβ) jk = 0 is false) Equivalent to H 0 : µ jk = µ j, k versus H 1 : not all µ jk are equal F a statistic and p a-value are testing H 0 : α j = 0 j versus H 1 : ( j {1,..., a})(α j 0) Testing main effect of first factor F b statistic and p b -value are testing H 0 : β k = 0 k versus H 1 : ( k {1,..., b})(β k 0) Testing main effect of second factor F ab statistic and p ab -value are testing H 0 : (αβ) jk = 0 j, k versus H 1 : ( j, k {1,..., a} {1,..., b})((αβ) jk 0) Testing interaction effect Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 14

15 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: Data Description Hypertension example from Maxwell & Delany (2003). Total of n = 72 subjects participate in hypertension experiment. Factor A: drug type (a = 3 levels: X, Y, Z) Factor B: diet type (b = 2 levels: yes, no) Randomly assign n jk = 12 subjects to each treatment cell: Note there are (ab) = (3)(2) = 6 treatment cells Observations are independent within and between cells Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 15

16 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: Descriptive Statistics Sum of blood pressure for each treatment cell ( 12 i=1 y ijk): Diet Drug No (k = 1) Yes (k = 2) Total X (j = 1) Y (j = 2) Z (j = 3) Total Sum-of-squares of blood pressure for each treatment cell ( 12 i=1 y ijk 2 Diet Drug No (k = 1) Yes (k = 2) Total X (j = 1) Y (j = 2) Z (j = 3) Total Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 16

17 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: OLS Estimation (by hand) Least-squares estimates are cell means: ˆµ jk = ȳ jk and n ˆµ = 1 b a k=1 j=1 i=1 abn y ijk = ȳ = = ( ) 1 ˆα 1 = b n k=1 i=1 bn y i1k ˆµ = ȳ 1 ȳ = = ( ) 1 ˆα 2 = b n k=1 i=1 bn y i2k ˆµ = ȳ 2 ȳ = = ( ) 1 ˆα 3 = b n k=1 i=1 bn y i3k ˆµ = ȳ 3 ȳ = = ( ) 1 ˆβ 1 = a n j=1 i=1 an y ij1 ˆµ = ȳ 1 ȳ = = ( ) 1 ˆβ 2 = a n j=1 i=1 an y ij2 ˆµ = ȳ 2 ȳ = = Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 17

18 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: OLS Estimation (by hand) Continuing from the previous slide... ( αβ) ˆ 11 = ȳ 11 ȳ 1 ȳ 1 + ȳ = = 5 36 ( αβ) ˆ 12 = ȳ 12 ȳ 1 ȳ 2 + ȳ = = 5 ( αβ) ˆ 21 = ȳ 21 ȳ 2 ȳ 1 + ȳ = = ( αβ) ˆ 22 = ȳ 22 ȳ 2 ȳ 2 + ȳ = = ( αβ) ˆ 31 = ȳ 31 ȳ 3 ȳ 1 + ȳ = = ( αβ) ˆ 32 = ȳ 32 ȳ 3 ȳ 2 + ȳ = = Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 18

19 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: Enter Data (in R) > bp = scan("/users/nate/desktop/hypertension.dat") Read 72 items > diet = factor(rep(rep(c("no","yes"),each=6),6)) > drug = factor(rep(rep(c("x","y","z"),each=12),2)) > biof = factor(rep(c("present","absent"),each=36)) > hyper = data.frame(bp=bp, diet=diet, drug=drug, biof=biof) > hyper[1:20,] bp diet drug biof no X present no X present no X present no X present no X present no X present yes X present yes X present yes X present yes X present yes X present yes X present no Y present no Y present no Y present no Y present no Y present no Y present yes Y present yes Y present Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 19

20 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: OLS Estimation (in R) Effect coding for drug and diet: > contrasts(hyper$drug) <- contr.sum(3) > contrasts(hyper$drug) [,1] [,2] X 1 0 Y 0 1 Z -1-1 > contrasts(hyper$diet) <- contr.sum(2) > contrasts(hyper$diet) [,1] no 1 yes -1 > mymod = lm(bp ~ drug * diet, data=hyper) > summary(mymod) # I deleted some output Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** drug e-05 *** drug ** diet e-06 *** drug1:diet * drug2:diet Signif. codes: 0 *** ** 0.01 * Residual standard error: on 66 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 5 and 66 DF, p-value: 3.385e-07 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 20

21 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: Sums-of-Squares (by hand 1) Defining n = b k=1 a j=1 n jk, the relevant sums-of-squares are SST = b a n jk k=1 j=1 i=1 (y ijk ȳ ) 2 = b ( a n jk k=1 j=1 i=1 y ijk 2 1 b ) a n jk 2 n k=1 j=1 i=1 y ijk = (13284)2 = SSE = b a n jk k=1 j=1 = i=1 (y ijk ȳ jk ) 2 = b k=1 a j=1 n jk i=1 y 2 ijk b k=1 a j=1 ( ) njk 2 i=1 y ijk [ ] /12 = n jk SSR = SST SSE = = 9780 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 21

22 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: Sums-of-Squares (by hand 2) The sums-of-squares for the main and interaction effects are given by SSA = bn a j=1 (ȳ j ȳ ) 2 = bn a j=1 ˆα2 j [ = (2)(12) ( 10) ] = 3675 SSB = an b k=1 (ȳ k ȳ ) 2 = an b ˆβ k=1 k 2 [ = (3)(12) ( 8.5) ] = 5202 SSAB = n b a k=1 j=1 (ȳ jk ȳ j ȳ k + ȳ ) 2 = n b a k=1 j=1 ( αβ) ˆ 2 jk [ = 12 ( 5) ( 2.75) ( 2.25) 2] = 903 and since n jk = n = 12 j, k, we have SSR = SSA + SSB + SSAB 9780 = Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 22

23 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: ANOVA Table (by hand) Putting things together in ANOVA table: Source SS df MS F p-value SSR <.0001 SSA SSB <.0001 SSAB SSE SST Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 23

24 Balanced Two-Way ANOVA Hypertension Example (Part 1) Hypertension Example: ANOVA Table (in R) > anova(mymod) Analysis of Variance Table Response: bp Df Sum Sq Mean Sq F value Pr(>F) drug *** diet e-06 *** drug:diet Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 24

25 Balanced Two-Way ANOVA Multiple Comparisons Multiple Comparisons Overview Still have multiple comparison problem: Overall test is not very informative Can examine effect estimates for group differences Need follow-up tests to examine linear combinations of means Still can use the same tools as before: Bonferroni Tukey (Tukey-Kramer) Scheffé Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 25

26 Balanced Two-Way ANOVA Multiple Comparisons Two-Way ANOVA Linear Combinations Assuming interaction model, we now have ˆL = b a k=1 j=1 c jkȳjk and ˆV (ˆL) = ˆσ 2 b a k=1 j=1 c2 jk /n jk where c jk are the coefficients and ˆσ 2 is the MSE. Assuming the additive model, we have ˆL a = a j=1 c jȳj and ˆV (ˆLa ) = ˆσ 2 a j=1 c2 j /n j ˆL b = b k=1 c kȳ k and ˆV (ˆLb ) = ˆσ 2 b k=1 c2 k /n k where c j and c k are main effect coefficients, ˆσ 2 is the MSE, and n j = b k=1 n jk and n k = a j=1 n jk are the marginal sample sizes. Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 26

27 Balanced Two-Way ANOVA Multiple Comparisons Two-Way Multiple Comparisons in Practice For interaction model, you follow-up on ˆµ jk = ȳ jk Bonferroni for any f tests (independent or not) Tukey (Tukey-Kramer) for all pairwise comparisons Scheffé for all possible contrasts For additive model, you follow-up on ˆµ j = ȳ j and ˆµ k = ȳ k Bonferroni for any f tests (independent or not) Tukey (Tukey-Kramer) for all pairwise comparisons Scheffé for all possible contrasts For additive model, Tukey and Scheffé control FWER for each main effect family separately. Use Bonferroni in combination with Tukey/Scheffé to control FWER for both families simultaneously Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 27

28 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Interaction (by hand) All ab(ab 1)/2 = 15 possible pairwise comparisons of ˆµ jk : ˆL = ȳ jk ȳ j k and ˆV (ˆL) = 194.2(2/12) = and we know that 2(ˆL) ˆV (ˆL) q ab,abn ab, so 100(1 α)% CI is given by ˆL ± 1 q (α) 2 ab,abn ab ˆV (ˆL) where q (α) ab,abn ab is critical value from studentized range. For example, 95% CI for µ 21 µ 11 is given by: (ˆµ 21 ˆµ 11 ) ± 1 q (.05) 2 6,66 ˆV (ˆL) ( ) ± 1 ( ) = [ ; ] 12 2 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 28

29 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Interaction (in R) All ab(ab 1)/2 = 15 possible pairwise comparisons of ˆµ jk : > mymod = aov(bp ~ drug * diet, data=hyper) > TukeyHSD(mymod, "drug:diet") Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = bp ~ drug * diet, data = hyper) $ drug:diet diff lwr upr p adj Y:no-X:no Z:no-X:no X:yes-X:no Y:yes-X:no Z:yes-X:no Z:no-Y:no X:yes-Y:no Y:yes-Y:no Z:yes-Y:no X:yes-Z:no Y:yes-Z:no Z:yes-Z:no Y:yes-X:yes Z:yes-X:yes Z:yes-Y:yes Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 29

30 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Additive (by hand A part 1) All a(a 1)/2 = 3 possible pairwise comparisons of ˆµ j : and the variance is given by where ˆσ 2 = Y X : ˆLa1 = = Z X : ˆLa2 = = Z Y : ˆLa3 = = 2.5 ˆV (ˆL aj ) = ˆσ 2 a j=1 c2 j /n j = ( )(2/24) = SSE+SSAB abn (a+b 1) = = Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 30

31 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Additive (by hand A part 2) Note 2(ˆLaj ) ˆV (ˆLaj ) q a,abn (a+b 1), so 100(1 α)% CI is given by ˆL aj ± 1 2 q (α) a,abn (a+b 1) ˆV (ˆLaj ) where q (α) a,abn (a+b 1) is critical value from studentized range. The 95% CI for all three pairwise comparisons is given by ˆL a1 ± 1 q (.05) 3,68 ˆV (ˆL a1 ) = ± 1 ( ) = [ ; ] 2 2 ˆL a2 ± 1 q (.05) 3,68 ˆV (ˆL a2 ) = ± 1 ( ) = [ ; ] 2 2 ˆL a3 ± 1 q (.05) 3,68 ˆV (ˆL a3 ) = 2.5 ± 1 ( ) = [ ; ] 2 2 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 31

32 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Additive (by hand B part 1) All b(b 1)/2 = 1 possible pairwise comparison of ˆµ k : and the variance is given by where ˆσ 2 = yes no : ˆLb = = 17 ˆV (ˆL b ) = ˆσ 2 b k=1 c2 k /n k = ( )(2/36) = SSE+SSAB abn = (a+b 1) 68 = Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 32

33 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Additive (by hand B part 2) Note 2(ˆLb ) ˆV (ˆLb ) q b,abn (a+b 1), so 100(1 α)% CI is given by ˆL b ± 1 q (α) 2 b,abn (a+b 1) ˆV (ˆL b ) where q (α) b,abn (a+b 1) is critical value from studentized range. The 95% CI for pairwise comparison is given by ˆL b ± 1 2 q (.05) 2,68 ˆV (ˆL b ) = 17 ± 1 2 ( ) = [ ; ] Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 33

34 Balanced Two-Way ANOVA Hypertension Example (Part 2) Hypertension Example: Additive (in R) All a(a 1)/2 = 3 possible pairwise comparisons of ˆµ j : > mymod = aov(bp ~ drug + diet, data=hyper) > TukeyHSD(mymod, "drug") Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = bp ~ drug + diet, data = hyper) $drug diff lwr upr p adj Y-X Z-X Z-Y All b(b 1)/2 = 1 possible pairwise comparison of ˆµ k : > mymod = aov(bp ~ drug + diet, data=hyper) > TukeyHSD(mymod,"diet") Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = bp ~ drug + diet, data = hyper) $diet diff lwr upr p adj yes-no e-06 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 34

35 Balanced Three-Way ANOVA Balanced Three-Way ANOVA Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 35

36 Balanced Three-Way ANOVA Model Form and Estimation Three-Way ANOVA Model (cell means form) The Three-Way Analysis of Variance (ANOVA) model has the form y ijkl = µ jkl + e ijkl for i {1,..., n jkl }, j {1,..., a}, k {1,..., b}, l {1,..., c}, where y ijkl R is response for i-th subject in factor cell (j, k, l) µ jkl R is population mean for factor cell (j, k, l) e ijkl iid N(0, σ 2 ) is a Gaussian error term n jkl is number of subjects in cell (j, k, l) (note: n jkl = n j, k, l in balanced three-way ANOVA) (a, b, c) is number of factor levels for Factors (A, B, C) Implies that y ijkl ind N(µ jkl, σ 2 ). Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 36

37 Balanced Three-Way ANOVA OLS Estimation (cell means form) Model Form and Estimation Similar to balanced two-way ANOVA, we want to minimize c b l=1 k=1 j=1 i=1 n a jkl (y ijkl µ jkl ) 2 which is equivalent to minimizing n jkl i=1 (y ijkl µ jkl ) 2 for all j, k, l Taking the derivative of SSE jkl = n jkl i=1 (y ijkl µ jkl ) 2, we see that n dsse jkl jkl = 2 y ijkl + 2n jkl µ jkl dµ jkl and setting to zero and solving gives ˆµ jkl = 1 n jkl njkl i=1 y ijkl = ȳ jkl i=1 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 37

38 Balanced Three-Way ANOVA Model Form and Estimation Three-Way ANOVA Model (effect coding) The three-way ANOVA with all interactions assumes that µ jkl = µ + α j + β k + γ l + (αβ) jk + (αγ) jl + (βγ) kl + (αβγ) jkl for j {1,..., a}, k {1,..., b}, and l {1,..., c} where µ is overall population mean α j is main effect of first factor such that a j=1 α j = 0 β k is main effect of second factor such that b k=1 β k = 0 γ l is main effect of third factor such that c l=1 γ l = 0 (αβ) jk is A B interaction effect such that a j=1 (αβ) jk = 0 k and b k=1 (αβ) jk = 0 j (αγ) jl is A C interaction effect such that a j=1 (αγ) jl = 0 l and c l=1 (αγ) jl = 0 j (βγ) kl is B C interaction effect such that b k=1 (βγ) kl = 0 l and c l=1 (βγ) kl = 0 k (αβγ) jkl is A B C interaction effect such that a j=1 (αβγ) jkl = 0 k, l and b k=1 (αβγ) jkl = 0 j, l and c l=1 (αβγ) jkl = 0 j, k Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 38

39 Balanced Three-Way ANOVA OLS Estimation (effect coding) Model Form and Estimation The OLS estimates of the various effects are given by ˆµ = ȳ ˆα j = ȳ j ȳ ˆβ k = ȳ k ȳ ˆγ l = ȳ l ȳ ( αβ) ˆ jk = ȳ jk ȳ j ȳ k + ȳ ( αγ) ˆ jl = ȳ j l ȳ j ȳ l + ȳ ( βγ) ˆ kl = ȳ kl ȳ k ȳ l + ȳ ( αβγ) ˆ jkl = ȳ jkl [ˆµ + ˆα j + ˆβ k + ˆγ l + ( αβ) ˆ jk + ( αγ) ˆ jl + ( βγ) ˆ kl ] Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 39

40 Balanced Three-Way ANOVA Fitted Values and Residuals Model Form and Estimation Form of fitted values depends on fit model: Additive: All 2-way Int: 3-way Int: ˆµ jkl = ˆµ + ˆα j + ˆβ k + ˆγ l ˆµ jkl = ˆµ + ˆα j + ˆβ k + ˆγ l + ( αβ) ˆ jk + ( αγ) ˆ jl + ( βγ) ˆ kl ˆµ jkl = ȳ jkl Residuals have the form ê ijkl = y ijkl ˆµ jkl where form of ˆµ jkl depends on fit model (additive versus interaction). Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 40

41 Balanced Three-Way ANOVA ANOVA Sums-of-Squares Basic Inference Defining N = abcn, the three-way ANOVA sums-of-squares are: SST = c b a n l=1 k=1 j=1 i=1 (y ijkl ȳ ) 2 df = N 1 SSR = n c b a l=1 k=1 j=1 (ȳ jkl ȳ ) 2 df = abc 1 SSE = c l=1 b k=1 a j=1 n i=1 (y ijkl ȳ jkl ) 2 df = N abc SSR = SS A + SS B + SS C + SS AB + SS AC + SS BC + SS ABC SS A = bcn a j=1 ˆα2 j df = a 1 SS B = acn b k=1 ˆβ 2 k df = b 1 SS C = abn c l=1 ˆγ2 l df = c 1 SS AB = cn b k=1 a j=1 ( ˆ αβ) 2 jk df = (a 1)(b 1) SS AC = bn c a l=1 j=1 ( αγ)2 ˆ jl df = (a 1)(c 1) SS BC = an c b l=1 k=1 ( βγ) ˆ 2 kl df = (b 1)(c 1) SS ABC = n c b a l=1 k=1 j=1 ( αβγ) ˆ 2 jkl df = (a 1)(b 1)(c 1) Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 41

42 Balanced Three-Way ANOVA Basic Inference Memory Example: Data Description (revisited) Hypertension example from Maxwell & Delany (2003). Total of N = 72 subjects participate in hypertension experiment. Factor A: drug type (a = 3 levels: X, Y, Z) Factor B: diet type (b = 2 levels: yes, no) Factor C: biof type (c = 2 levels: present, absent) Randomly assign n jkl = 6 subjects to each treatment cell: Note there are (abc) = (3)(2)(2) = 12 treatment cells Observations are independent within and between cells Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 42

43 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Look at Data > bp = scan("/users/nate/desktop/hypertension.dat") Read 72 items > diet = factor(rep(rep(c("no","yes"),each=6),6)) > drug = factor(rep(rep(c("x","y","z"),each=12),2)) > biof = factor(rep(c("present","absent"),each=36)) > hyper = data.frame(bp=bp, diet=diet, drug=drug, biof=biof) > contrasts(hyper$drug) <- contr.sum(3) > contrasts(hyper$diet) <- contr.sum(2) > contrasts(hyper$biof) <- contr.sum(2) > hyper[1:15,] bp diet drug biof no X present no X present no X present no X present no X present no X present yes X present yes X present yes X present yes X present yes X present yes X present no Y present no Y present no Y present Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 43

44 Balanced Three-Way ANOVA Basic Inference Hypertension Example: All Interactions > mymod = lm(bp ~ drug * diet * biof, data=hyper) > anova(mymod) Analysis of Variance Table Response: bp Df Sum Sq Mean Sq F value Pr(>F) drug e-05 *** diet e-07 *** biof *** drug:diet drug:biof diet:biof drug:diet:biof * Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 44

45 Balanced Three-Way ANOVA Basic Inference Hypertension Example: All 2-Way Interactions > mymod = lm(bp ~ drug * diet + drug * biof + diet * biof, data=hyper) > anova(mymod) Analysis of Variance Table Response: bp Df Sum Sq Mean Sq F value Pr(>F) drug e-05 *** diet e-07 *** biof *** drug:diet drug:biof diet:biof Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 45

46 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Additive Model > mymod = lm(bp ~ drug + diet + biof, data=hyper) > anova(mymod) Analysis of Variance Table Response: bp Df Sum Sq Mean Sq F value Pr(>F) drug *** diet e-07 *** biof ** Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 46

47 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Interaction Plot If you choose the three-way interaction model, you could visualize the interaction using an interaction plot. Biofeedback Absent Biofeedback Present Mean BP Diet No Diet Yes Mean BP Diet No Diet Yes X Y Z X Y Z Drug Drug Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 47

48 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Interaction Plot (R code) yhat=tapply(hyper$bp,list(hyper$drug,hyper$diet,hyper$biof),mean) par(mfrow=c(1,2)) mytitles=c("biofeedback Absent","Biofeedback Present") for(k in 1:2){ plot(1:3,yhat[,1,k],ylim=c(165,215),xlab="drug", ylab="mean BP",main=mytitles[k],axes=FALSE,type="l") lines(1:3,yhat[,2,k],lty=2) legend("topleft",c("diet No","Diet Yes"),lty=1:2,bty="n") axis(1,at=1:3,labels=c("x","y","z")) axis(2) } Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 48

49 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Multiple Comparisons Assuming we chose the additive model, we would perform follow-up tests on the marginal means. Factor A: ˆµ aj = ˆµ + ˆα j = ȳ j Factor B: ˆµ bk = ˆµ + ˆβ k = ȳ k Factor C: ˆµ cl = ˆµ + ˆγ l = ȳ l If we chose three-way interaction model, we would perform follow-up tests on the individual cell means. ˆµ jkl = ˆµ + ˆα j + ˆβ k + ˆγ l + ( αβ) ˆ jk + ( αγ) ˆ jl + ( βγ) ˆ kl + ( αβγ) ˆ jkl = ȳ jkl Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 49

50 Balanced Three-Way ANOVA Basic Inference Hypertension Example: Multiple Comparisons > mymod = aov(bp ~ drug + diet + biof, data=hyper) > TukeyHSD(mymod, "drug") # I deleted some output Tukey multiple comparisons of means 95% family-wise confidence level $drug diff lwr upr p adj Y-X Z-X Z-Y > TukeyHSD(mymod, "diet") # I deleted some output Tukey multiple comparisons of means 95% family-wise confidence level $diet diff lwr upr p adj yes-no e-07 > TukeyHSD(mymod, "biof") # I deleted some output Tukey multiple comparisons of means 95% family-wise confidence level $biof diff lwr upr p adj present-absent Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 50

51 Unbalanced ANOVA Models Unbalanced ANOVA Models Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 51

52 Unbalanced ANOVA Models Overview of Problem Unbalanced ANOVA: Model Form Unbalanced ANOVA has same model form as balanced, but unequal sample sizes in each cell. 1-way: n j n j for some j, j 2-way: n jk n j k for some (jk), (j k ) 3-way: n jkl n j k l for some (jkl), (j k l ) In the previous slides, we assumed n jk = n j, k (two-way ANOVA) or n jkl = n j, k, l (three-way ANOVA), which made life easy. Effects were orthogonal in balanced design Parameter estimates had simple relation to cell/marginal means Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 52

53 Unbalanced ANOVA Models Overview of Problem Unbalanced ANOVA: Implications Main consequence for two-way (and higher-way) unbalanced design: Non-orthogonal SS (e.g., SSR SSA + SSB + SSAB) Design is less efficient (larger variances of parameter estimates) Unbalanced design also affects our estimation and follow-up tests Parameter estimates require matrix inversion: ˆb = (X X) 1 X y Need to do follow-up tests on least-squares means Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 53

54 Unbalanced ANOVA Models Overview of Problem Unbalanced ANOVA: Testing Effects Because of non-orthogonality, cannot test effects using F = MS? MSE. Instead we use the General Linear Model (GLM) F test statistic: F = SSE R SSE F df R df F SSE F df F F (dfr df F,df F ) where SSE R is sum-of-squares error for reduced model SSE F is sum-of-squares error for full model df R is error degrees of freedom for reduced model df F is error degrees of freedom for full model Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 54

55 Unbalanced ANOVA Models Overview of Problem Unbalanced ANOVA: Testing Example Consider two-way ANOVA and all 7 possible models y ijk = µ + α j + β k + (αβ) jk + e ijk (1) y ijk = µ + α j + β k + e ijk (2) y ijk = µ + α j + (αβ) jk + e ijk (3) y ijk = µ + β k + (αβ) jk + e ijk (4) y ijk = µ + α j + e ijk (5) y ijk = µ + β k + e ijk (6) y ijk = µ + e ijk (7) Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 55

56 Unbalanced ANOVA Models Overview of Problem Unbalanced ANOVA: Testing Example (continued) To test effect, use F test comparing full and reduced models. To test each effect there are multiple choices we could use for full and reduced models: A: F=1 and R=4 or F=2 and R=6 or F=5 and R=7 B: F=1 and R=3 or F=2 and R=5 or F=6 and R=7 AB: F=1 and R=2 or F=3 and R=5 or F=4 and R=6 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 56

57 Unbalanced ANOVA Models Types of Sum-of-Squares Types of Sums-of-Squares Type I SS Amount of additional variation explained by the model when a term is added to the model (aka sequential sum-of-squares). In two-way ANOVA, type I SS would compare: (a) Main Effect A: F=5 and R=7 (b) Main Effect B: F=2 and R=5 (c) Interaction Effect: F=1 and R=2 Type II SS Amount of variation a term adds to the model when all other terms are included except terms that contain the effect being tested (e.g., (αβ) jk contains α j and β k ). In two-way ANOVA, type II SS would compare: (a) Main Effect A: F=2 and R=6 (b) Main Effect B: F=2 and R=5 (c) Interaction Effect: F=1 and R=2 Type III SS Amount of variation a term adds to the model when all other terms are included, which is sometimes called partial sum-of-squares. In two-way ANOVA, type III SS would compare: (a) Main Effect A: F=1 and R=4 (b) Main Effect B: F=1 and R=3 (c) Interaction Effect: F=1 and R=2 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 57

58 Unbalanced ANOVA Models Types of Sums-of-Squares Types of Sum-of-Squares (in R) When fitting multi-way ANOVAs, anova function gives Type I SS. Order matters in unbalanced design! bp = drug + diet produces different Type I SS tests than bp = diet + drug if design is unbalanced Use Anova function in car package for Type II and Type III SS. Function performs Type II SS tests by default Use type=3 option for Type III SS tests Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 58

59 Unbalanced ANOVA Models Hypertension Example (Part 4) Hypertension Example: Type I > mymod = lm(bp ~ drug * diet * biof, data=hyper[1:71,]) > anova(mymod) Analysis of Variance Table Response: bp Df Sum Sq Mean Sq F value Pr(>F) drug e-05 *** diet e-07 *** biof *** drug:diet drug:biof diet:biof drug:diet:biof * Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 59

60 Unbalanced ANOVA Models Hypertension Example (Part 4) Hypertension Example: Type II > library(car) > Anova(mymod, type=2) Anova Table (Type II tests) Response: bp Sum Sq Df F value Pr(>F) drug e-05 *** diet e-07 *** biof *** drug:diet drug:biof diet:biof drug:diet:biof * Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 60

61 Unbalanced ANOVA Models Hypertension Example (Part 4) Hypertension Example: Type III > library(car) > Anova(mymod, type=3) Anova Table (Type III tests) Response: bp Sum Sq Df F value Pr(>F) (Intercept) < 2.2e-16 *** drug e-05 *** diet e-07 *** biof *** drug:diet drug:biof diet:biof drug:diet:biof * Residuals Signif. codes: 0 *** ** 0.01 * Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 61

62 Appendix Appendix Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 62

63 Appendix Ordinary Least-Squares (proof for µ) Expanding the first summation produces [ b a n SSE = yijk 2 2(µ + α j + β k + (αβ) jk ) k=1 j=1 i=1 n i=1 y ijk + n (µ + α j + β k + (αβ) jk ) 2 ] Taking the derivative with respect to µ we have dsse dµ = b a [ k=1 j=1 2 n = 2 ( b k=1 a j=1 n i=1 y ijk and setting to zero and solving for µ gives ˆµ = 1 b a n abn k=1 j=1 i=1 y ijk = ȳ i=1 y ijk + 2n µ + 2n (α j + β k + (αβ) jk ) ] ) + 2abn µ Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 63

64 Appendix Ordinary Least-Squares (proof for α j ) Taking the derivative with respect to α j we have dsse dα j = b [ k=1 2 n = 2 ( b k=1 n i=1 y ijk i=1 y ijk + 2n α j + 2n (µ + β k + (αβ) jk ) ] ) + 2bn α j + 2bn µ and setting to zero, using ˆµ for µ, and solving for α j gives ˆα j = 1 bn ( b n k=1 i=1 y ijk) ˆµ = ȳ j ȳ Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 64

65 Appendix Ordinary Least-Squares (proof for β k ) Taking the derivative with respect to β k we have dsse dβ k = a [ j=1 2 n = 2 ( a j=1 n i=1 y ijk i=1 y ijk + 2n β k + 2n (µ + α j + (αβ) jk ) ] ) + 2an β k + 2an µ and setting to zero, using ˆµ for µ, and solving for β k gives ˆβ k = 1 an ( a n j=1 i=1 y ijk) ˆµ = ȳ k ȳ Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 65

66 Appendix Ordinary Least-Squares (proof for (αβ) jk ) Taking the derivative with respect to (αβ) jk we have dsse n = 2 y ijk + 2n (αβ) jk + 2n (µ + α j + β k ) d(αβ) jk i=1 and setting to zero, using (ˆµ, ˆα j, ˆβ k ) for (µ, α j, β k ), and solving for (αβ) jk gives ( αβ) ˆ jk = 1 n ( n i=1 y ijk) ˆµ ˆα j ˆβ k = ȳ jk ȳ j ȳ k + ȳ Return Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 66

67 Appendix Partitioning the Variance (proof part 1) To prove SSR = SSA + SSB + SSAB when n jk = n j, k, note that y ijk ȳ = (y ijk ȳ jk ) + (ȳ jk [ȳ j + ȳ k ȳ ]) + (ȳ j ȳ ) + (ȳ k ȳ ) Now if we square both sides we have (y ijk ȳ ) 2 = (y ijk ȳ jk ) 2 + (ȳ jk [ȳ j + ȳ k ȳ ]) 2 + (ȳ j ȳ ) 2 + (ȳ k ȳ ) 2 + 2(y ijk ȳ jk ) {(ȳ jk [ȳ j + ȳ k ȳ ]) + (ȳ j ȳ ) + (ȳ k ȳ )} + 2(ȳ jk [ȳ j + ȳ k ȳ ]) [(ȳ j ȳ ) + (ȳ k ȳ )] + 2(ȳ j ȳ )(ȳ k ȳ ) Now if we apply the triple summation we have SST n b a jk SST = (y ijk ȳ ) 2 k=1 j=1 i=1 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 67

68 Appendix Partitioning the Variance (proof part 2) First, note that we have SSE = b SSAB = b SSA = b a njk k=1 j=1 a njk k=1 j=1 a njk k=1 j=1 SSB = b k=1 a j=1 i=1 (y ijk ȳ jk ) 2 i=1 (ȳ jk [ȳ j + ȳ k ȳ ]) 2 i=1 (ȳ j ȳ ) 2 njk i=1 (ȳ k ȳ ) 2 so we need to prove that the crossproduct terms are orthogonal. To prove that the first crossproduct term sums to zero, define (αβ) jk = (ȳ jk [ȳ j + ȳ k ȳ ]) + (ȳ j ȳ ) + (ȳ k ȳ ) and note that b a njk k=1 j=1 i=1 2(y ijk ȳ jk )(αβ) jk = 2 b a k=1 j=1 (αβ) njk jk i=1 (y ijk ȳ jk ) = 2 b a k=1 j=1 (αβ) jk(0) = 0 because we are summing mean-centered variable. Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 68

69 Appendix Partitioning the Variance (proof part 3) To prove that the second crossproduct term sums to zero, note that ( ˆ αβ) jk = (ȳ jk [ȳ j + ȳ k ȳ ]), ˆα j = (ȳ j ȳ ), and ˆβ k = (ȳ k ȳ ), so b a njk k=1 j=1 i=1 2( αβ) ˆ jk (ˆα j + ˆβ k ) = 2 b a k=1 j=1 n jk( αβ) ˆ jk (ˆα j + ˆβ k ) Now assuming that n jk = n j, k b a k=1 j=1 n jk( αβ) ˆ ( jk ˆα j = n a j=1 ˆα b ) j k=1 ( αβ) ˆ jk = n a j=1 ˆα j(0) = 0 b a k=1 j=1 n jk( αβ) ˆ jk ˆβk = n b ˆβ ( a k=1 k j=1 ( αβ) ˆ ) jk = n b ˆβ k=1 k (0) = 0 Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 69

70 Appendix Partitioning the Variance (proof part 4) To prove that the third crossproduct term sums to zero, note that b a njk k=1 j=1 i=1 2(ȳ j ȳ )(ȳ k ȳ ) = 2 b a k=1 j=1 n jk ˆα j ˆβ k and if n jk = n j, k we have that 2 b k=1 a j=1 n jk ˆα j ˆβ k = 2n b k=1 a j=1 ˆα j ˆβ k which completes the proof. = 2n b k=1 ˆβ k ( a j=1 ˆα j = 2n b k=1 ˆβ k (0) = 0 ) Return Nathaniel E. Helwig (U of Minnesota) Factorial & Unbalanced Analysis of Variance Updated 04-Jan-2017 : Slide 70