Multi-Way Analysis of Variance (ANOVA)

Transcription

1 Multi-Way Analysis of Variance (ANOVA) An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem John Tukey (American Mathmetician)

2 Multi-way ANOVA Just like one-way ANOVA but with more than one treatment Each treatment may still have many levels (e.g. VARIETY A,B,C and FARM farm1, farm2) We do not only look at the effect of each treatment but we must also look at the interaction between treatments Like the one-way ANOVA we use the F-statistic

3 Multi-way ANOVA Source of Variation df Sum of Squares Mean Squares F-value A (e.g. VARIETY) t A -1 SS A =MS A *df A MS A =signal A MS A /MS ERROR B (e.g. FARM) t B -1 SS B =MS B *df B MS B =signal B MS B /MS ERROR A X B (interaction) (t A -1)*(t B -1) SS AXB =MS AXB *df AXB MS AXB =signal AXB MS AXB =MS ERROR Error n-(t A *t B ) SS ERROR =MS ERROR *df ERROR MS ERROR =noise

4 Lentil Example Treatment A VARIETY A xa xa xa VARIETY B x B 389 x B x B VARIETY C x C x C x C

5 Lentil Example Treatment B FARM 1 C B A xabc x c x B xa FARM 2 A C B xabc xa x c x B

6 Lentil Example VARIETY\FARM FARM 1 FARM2 Marginal Mean A 720, 690, 740, 760 (727.5) 163, 176, 163, 168 (167.5) B 515, 480, 545, 492 (508) 375, 389, 405, 387 (389) C 540, 502, 510, 505 (514.25) 375, 385, 381, 400 (385.25) Marginal Mean

7 Lentil Example Main Effects MS A = r t B MS B = r t A t A i x t Ai xall t A 1 t B i x t Bi xall t B VARIETY\FARM FARM1 FARM2 A B C Marginal Mean 720, 690, 740, 760 (727.5) 515, 480, 545, 492 (508) 540, 502, 510, 505 (514.25) 163, 176, 163, 168 (167.5) 375, 389, 405, 387 (389) 375, 385, 381, 400 (385.25) Marginal Mean Treatment A - VARIETY MS VARIETY = MS VARIETY = = Treatment B - FARM MS FARM = MS FARM = =

8 Lentil Example Interaction MS AXB = r t A i t B j x ij x i xj + xall t A 1 t B 1 Interaction VARIETY x FARM 2 VARIETY\FARM FARM1 FARM2 A B C Marginal Mean 720, 690, 740, 760 (727.5) 515, 480, 545, 492 (508) 540, 502, 510, 505 (514.25) 163, 176, 163, 168 (167.5) 375, 389, 405, 387 (389) 375, 385, 381, 400 (385.25) Marginal Mean MS VARIETY x FARM = MS VARIETY x FARM = =

9 Lentil Example Error MS ERROR = Error t A i t B j r k x ijk x ij n t A t B 2 VARIETY\FARM FARM1 FARM2 A B C Marginal Mean 720, 690, 740, 760 (727.5) 515, 480, 545, 492 (508) 540, 502, 510, 505 (514.25) 163, 176, 163, 168 (167.5) 375, 389, 405, 387 (389) 375, 385, 381, 400 (385.25) Marginal Mean MS ERROR = MS ERROR = =

10 How to report results from a Multi-way ANOVA Source of Variation df Sum of Squares Mean Squares F-value P-value Variety (A) Farm (B) <0.05 Variety x Farm (AxB) <0.05 Error Multi-way ANOVA in R: anova(lm(yield~variety*farm)) anova(lm(yield~variety+farm)+variety:farm)

11 How to report results from a Multi-way ANOVA pt(f A, df A, df ERROR ) pt(f B, df B, df ERROR ) pt(f AxB, df AxB, df ERROR ) Source of Variation df Sum of Squares Mean Squares F-value P-value Variety (A) Farm (B) <0.05 Variety x Farm (AxB) <0.05 Error MS ERROR *df ERROR MS A *df A MS B *df B MS AxB *df AxB MS A /MS ERROR MS B /MS ERROR MS AxB /MS ERROR

12 How to report results from a Multi-way ANOVA Source of Variation df Sum of Squares Mean Squares F-value P-value Variety (A) Farm (B) <0.05 Variety x Farm (AxB) <0.05 Error If the interaction is significant you should ignore the main effects because the story is not that simple!

13 response Interaction plots Different story under different conditions Interaction plot in R: interaction.plot(maineffect1,maineffect2,response) interaction.plot(farm,variety,yield) main effect 1

14 Yield Yield Yield Yield Interaction plots Different story under different conditions 1. A B VARIETY is significant (*) FARM is significant (*) FARM2 has better yield than FARM1 No Interaction 2. Avg A Avg B VARIETY is not significant FARM is significant (*) VARIETY A is better on FARM2 and VARIETY B is better on FARM1 Significant Interaction A B A B VARIETY is significant (*) FARM is significant (*) small difference Main effects are significant, BUT hard to interpret with overall means Significant Interaction VARIETY is not significant FARM is not significant Cannot distinguish a difference between VARIETY or FARM No Interaction

15 Interaction plots Different story under different conditions An interaction detects non-parallel lines Difficult to interpret interaction plots for more than a 2-WAY ANOVA If the interaction effect is NOT significant then you can just interpret the main effects BUT if you find a significant interaction you don t want to interpret main effects because the combination of treatment levels results in different outcomes

16 Pairwise comparisons What to do when you have an interaction a.k.a Pairwise t-tests Lentil Example: 3 VARITIES (A, B, and C) C = t(t 1) 2 = 3(2) 2 = 3 A B A C B C Number of comparisons: t t 1 C = 2 t = number of treatment levels Probability of making a Type I Error in at least one comparison = 1 probability of making no Type I Error at all Experiment-wise Type I Error for = 0.05: probability of Type I Error = C Lentil Example: probability of Type I Error = probability of Type I Error = probability of Type I Error = Significantly increased probability of making an error! Therefore pairwise comparisons leads to compromised experiment-wise -level

17 Pairwise comparisons What to do when you have an interaction a.k.a Pairwise t-tests Another Example in R: There should NOT be a significant difference between these 2 groups Did anyone get a significant difference?

18 Pairwise comparisons What to do when you have an interaction a.k.a Pairwise t-tests Another Example in R: If we have = 0.05, this indicates that you will get a difference as big or bigger 1 out of every 20 times Now out of the 24 people in this room (C=24), what is the probability of getting at least one significant p-value (false positive)? HARD to Answer What is the probability of NOT getting at least one significant p-value (false positive)? EASIER to Answer Class Example: probability of Type I Error = probability of Type I Error = probability of Type I Error = That s a 71% chance of making an error!!!! We need to adjust our and p-values to correct for this bias!

19 Benferroni Adjustment Adjust -level for multiple comparisons Benferroni Adjustment: adj = α C New accounts for multiple comparisons Our new cutoff for significance Class Example: α adj = = But now we evaluate significance at this value Now at this new significance level Did anyone get a significant difference?

20 Pairwise comparisons Tukey Honest Differences (Test) a.k.a Pairwise t-tests with adjusted p-values Pairwise comparisons in R: lentil.model=aov (lm(yield~variety*farm)) TukeyHSD(lentil.model) If we have NO significant interaction effect we can just look at the main effects If we have a significant interaction effect use these values

21 How to report a significant difference in a graph A A B A,B Create a matrix of significance and use it to code your graph W X Y Z W - NS * NS X - * NS Y - NS Z - Same letter = non significant Different letter = significant W X Y Z