Garvan Ins)tute Biosta)s)cal Workshop 16/7/2015. Tuan V. Nguyen. Garvan Ins)tute of Medical Research Sydney, Australia

Size: px

Start display at page:

Download "Garvan Ins)tute Biosta)s)cal Workshop 16/7/2015. Tuan V. Nguyen. Garvan Ins)tute of Medical Research Sydney, Australia"

Augusta Warner
6 years ago
Views:

1 Garvan Ins)tute Biosta)s)cal Workshop 16/7/2015 Tuan V. Nguyen Tuan V. Nguyen Garvan Ins)tute of Medical Research Sydney, Australia

2 Analysis of variance Between- group and within- group varia)on explained Model of ANOVA Post- hoc comparisons R implementa)on

3 Between-group variation and Within-group variation

4 Sir Ronald A. Fisher, inventor of ANOVA Ronald A. Fisher, gene)cist, sta)s)cian, philosopher "a genius who almost single- handedly created the founda5ons for modern sta5s5cal science" "Fisher laid the founda5ons for most of experimental design, analysis of variance and much of sta5s5cal inference," Ronald Fisher ( )

5 The idea of ANOVA Analysis variable (Y) is con)nuous Comparison of mul)ple groups (k >2) Null hypothesis: all means are equal H o : µ 1 = µ 2 = = µ k Alterna)ve hypothesis H a : at least one pair is different

6 The concept of "variation" Given a series of n observed values X i (X 1, X 2, X 3, ) a deviate is defined as: Squared D: Sum of squares (eg varia)on): D = X i M D 2 = (X i - M) 2 SS = (X 1 - M) 2 + (X 2 - M) 2 + (X 3 - M) (X n - M) 2 n = ( X M ) i i= 1 2

7 Between- and within-group variation Key to understanding ANOVA: Between- group varia)on Within- group varia)on

8 "Between-group" variation Group 1 Group 2 Group 3 Group k X 11 X 21 X 31 X k1 X 12 X 22 X 32 X k2 X 13 X 23 X 33 X k3 X 14 X 24 X 34 X k4 X 15 X 25 X 35 X k5 X 16 X 26 X 36 X k6 M 1 M 2 M 3 M k

9 "Within-group" variation Group 1 Group 2 Group 3 Group k X 11 X 21 X 31 X k1 X 12 X 22 X 32 X k2 X 13 X 23 X 33 X k3 X 14 X 24 X 34 X k4 X 15 X 25 X 35 X k5 X 16 X 26 X 36 X k6 M 1 M 2 M 3 M k

10 Logic ofanova Group 1 Group 2 Group 3 Group k X 11 X 21 X 31 X k1 X 12 X 22 X 32 X k2 X 13 X 23 X 33 X k3 X 14 X 24 X 34 X k4 X 15 X 25 X 35 X k5 X 16 X 26 X 36 X k6 M 1 M 2 M 3 M k Compare between varia+on (B) with within group varia+on (W) If B > W, then that is a signal of difference between groups

11 An example A hormone measured in 4 groups of pa)ents A B C D

12 Between-group variation Mean A B C D Overall mean = 14.2

13 Between-group variation Mean N A B C D Overall mean = 14.2 Sum of squares of between- group differences: SSB = 7*( ) 2 + 8*( ) 2 + 6*( ) 2 + 9*( ) 2 = 643.9

14 Within-group variation Mean A B C D Sum of within- group sum of squares: SSW A = (8 7.4) 2 + (9 7.4) (5 7.4) 2 = 33.7

15 Within-group variation Mean A B C D SSW A = (8 7.4) 2 + (9 7.4) (5 7.4) 2 = 33.7 SSW B = SSW C = SSW D = 214.6

16 Within-group variation SSW A = (8 7.4) 2 + (9 7.4) (5 7.4) 2 = 33.7 SSW B = SSW C = SSW D = SSW = = 681.6

17 ANOVA table Source Degrees of freedom Sum of squares (SS) Giữa 4 nhóm Trong các nhóm Mean square (MS)

18 Bảng phân tích phương sai Source Degrees of freedom Sum of squares (SS) Mean square (MS) Giữa 4 nhóm Trong các nhóm Tổng số F-test = / 26.2 = 8.2

19 R codes A = c(8, 9, 11, 4, 7, 8, 5) B = c(7, 17, 10, 14, 12, 24, 11, 22) C = c(28, 21, 26, 11, 24, 19) D = c(26, 16, 13, 12, 9, 10, 11, 17, 15) x = c(a, B, C, D) group = c(rep("a", 7), rep("b", 8), rep("c", 6), rep("d", 9)) data = data.frame(x, group) data av = aov(x ~ group) summary(av)

20 Result > av=aov(x ~ group) > summary(av) Df Sum Sq Mean Sq F value Pr(>F) group *** Residuals Signif. codes: 0 *** ** 0.01 * There is a difference between groups

21 Analysis of variance: Summary ANOVA is used for tes)ng the hypothesis that involves comparison of mul)ple groups R func)on for ANOVA: aov analysis = aov(y ~ group)

22 Post hoc analyses

24 Post-hoc comparisons Planned comparisons when there are small number of hypotheses to be tested Post- hoc comparisons when all possible comparisons are tested Omnibus F test Significance (P < 0.05) signifies a difference existed Which specific groups are different?

25 Methods for post-hoc comparisons Classic methods Least significance difference (LSD): protected t test Bonferroni's adjustment Standard methods Tukey's Honestly Significant Difference Neuman- Keuls, Ryan, Scheffe, etc Newer / modern methods FDR

26 Least significance difference (LSD) Mul)ple t- test with no correc)on t = X 1 X 2 2MS error n

27 Carlo Emilio Bonferroni ( ) Italian sta)s)cian, University of Florence Best known for "Bonferroni's inequality", and Bonferroni's correc)on in post- hoc analysis

28 Bonferroni's adjustment When there are c tests of hypothesis, the probability of finding a chance significance is 1 - (1 α) c Correc)ng for mul)ple tests of hypothesis Use α* = α / c, where c is the number of tests If the observed P value < α*, then declare "significant"

29 Tukey s HSD HSD = Honestly Significant Difference Q = X j X k MSW / n n is the average number of subjects per group If observed Q > theore)cal Q (theore)cal Tukey's Studen)zed cri)cal value) then the difference is "sta)s)cally significant"

30 Tukey's studentized Studen)zed range sta)s)c Q kn, k, α = max X i min WMS X i N Difference between X 1 and X 2 is significant if: Q ij Xi X j N = > Q WMS k, n k, α When n is not the same, then N = 2n i n j /(n i +n j )

31 Newer method: False discovery rate (FDR) Advanced by Benjamini Hochberg in a seminal paper (J Roy Sta5st Soc B 1995), almost 27,000 cita)ons Ques)on: How many false discoveries we have made? FDR = # of falsely rejected null hypotheses total # of rejected null hypotheses

32 Benjamini & Hochberg's method Create a vector A of sorted p values Create a vector B by compu)ng j(α/n) for j=1,2,...,n Substract vector A from vector B; call this vector C Find the largest index d, (from 1 to 10) for which the corresponding number in C is nega)ve Reject all null hypotheses whose p values P d The null hypothesis for all other tests are not rejected

33 FDR explained The FDR of a set of hypothesis tests is the expected percent of falsely rejected hypotheses. If FDR = 0.3, then we should expect 70% of them to be correct

34 Which method is appropriate? The method that yields the narrowest confidence interval

35 Adjustment for multiple comparisons #Bonferroni pairwise.t.test(x, group, p.adjust="bonferroni", pool.sd=t) # Benjamin-Hochberg pairwise.t.test(x, group, p.adjust="bh", pool.sd=t)

36 > pairwise.t.test(x, group, p.adjust="bonferroni", pool.sd=t) Pairwise comparisons using t tests with pooled SD data: x and group A B C B C D > pairwise.t.test(x, group, p.adjust="bh", pool.sd=t) Pairwise comparisons using t tests with pooled SD data: x and group A B C B C D P value adjustment method: BH

37 m = aov(x ~ group) TukeyHSD(m) R code - Tukey s Method > TukeyHSD(m) Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = x ~ group) $group diff lwr upr p adj B-A C-A D-A C-B D-B D-C

38 plot(tukeyhsd(m), ordered=t) 95% family-wise confidence level D-C D-B C-B D-A C-A B-A Differences in mean levels of group

39 ANOVA: summary ANOVA An extension of 2- sample t- test to mul)ple groups Comparing between- group varia)on to within- group varia)on Many Post- hoc comparisons Help control type I error Beware of the alpha level and power issues

Multiple Pairwise Comparison Procedures in One-Way ANOVA with Fixed Effects Model

Biostatistics 250 ANOVA Multiple Comparisons 1 ORIGIN 1 Multiple Pairwise Comparison Procedures in One-Way ANOVA with Fixed Effects Model When the omnibus F-Test for ANOVA rejects the null hypothesis that