Compare several group means: ANOVA

Size: px

Start display at page:

Download "Compare several group means: ANOVA"

Jonas Charles
5 years ago
Views:

1 1 - Introduction Compare several group means: ANOVA Pierre Legendre, Université de Montréal August 009 Objective: compare the means of several () groups for a response variable of interest. The groups contain independent observations. The name analsis of variance (ANOVA) describes the fact that the total, within-group, and among group variances will be computed to test H 0 : 1, which concerns the means of the groups. Instead of using ANOVA, one ma be tempted to carr out a series of t- tests for all possible pairs of groups. Actuall, ANOVA cannot be replaced b a series of t-tests because multiple testing significantl increases the probabilit of tpe I error if H 0 is true. Example Consider 7 groups of observations drawn independentl and at random from the same statistical population. H 0 is obviousl true. 7 (7 1)/ 1 t-tests must be computed to compare all pairs of groups. If each t-statistic is tested at the 0.05 significance level, we have, in each case, 5 chances over 100 to reject H 0 when H 0 is true (tpe I error). Using the binomial distribution, the probabilit of rejecting H 0 at least once in these 1 tests is 0.66 instead of For a test to be valid, the tpe I error rate must be.

2 Compare several group means: anova Analsis of variance has been developed b the British agronomist Ronald A. Fisher at the Rothamsted Experimental Station, UK. The nominal variable (factor) describing the group to which each observation belongs is called the classification criterion. It ma represent either a fixed or a random factor. Fixed factor: For a particular factor, the levels (or treatments) of the factor represented in the experiment compose all possible levels of the factor in the population, or at least all those that are of interest in the particular experiment. Fixed factors are usuall of interest: one is interested in determining their effect on the response variable. Random factor: We assume that the levels of a factor in our experiment have been randoml assigned from all possible levels of the factor that exist in the real world. Random factors ma have been incorporated simpl because the are nown to affect the response variable. The statistical hpotheses are the following for groups for which the response variable has been observed or measured: H 0 : 1. The groups were drawn from the same population or from populations with identical means. H 1 : at least one of the means differs from the others. After carring out a global test, a posteriori multiple comparison tests ma be conducted to determine which group(s) differ from the others. Note: we will not compare the variances of groups along. The null hpothesis is not H 0 : 1. That hpothesis could be tested using a Bartlett, Levene, or log-anova test of homogeneit of the group variances.

3 Compare several group means: anova 3 We will, however, use the ratio to compare the means, just lie the t-test was comparing two means while taing the corresponding within-group variances into account. Applications: ANOVA is a widel used method to analze the results of experiments conducted in the lab or the field. Hurlbert * has shown that ecological surves can often be seen as mensurative experiments if the are structured b identifiable factors, and studied b ANOVA. The underling ecological processes can then be separatel studied through manipulative (controlled) experiments. Objective examples: - Analsis of the effect of a random factor: one often hopes to show that the data support H 0. If the groups do not differ in their means, the can be pooled in subsequent analses. Among-group variance Within-group variance - Analsis of the effect of a controlled factor: in most cases, one is hoping to reject H 0 in order to support the hpothesis (H 1 ) that a portion of the response variable s variation can be explained b the factor. Several classification criteria can be considered in the same analsis: next lecture. * Hurlbert, S. H Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54:

4 Compare several group means: anova 4 - Notation in single-classification (one-wa) anova Vector represents the response variable, vector x the factor (or classification criterion). The values in the response and explanator vectors are identified b two indices: the first index represents the group {1...}, the second index the replicate within the group. Thus, 3 is the value of the response observed for the second replicate of group #3. Example: 11 x x x 1 1 x 1 13 x 1 1 x x x 31 3 x x x 34 In the representation in the centre, x contains a group number for each observation. Before ANOVA in R, the classification criterion coded in that wa must be declared as a factor b the command: x as.factor(x). Vector x could equall well be coded b a series of binar (or dumm) variables, as shown in the right-hand column. With that coding, the variation of explained b x can be analsed b regression.

5 Compare several group means: anova Sources of variation Total dispersion (or sum-of-squares) TSS Within-group (or residual) dispersion (or sum-of-squares) RSS Among-group dispersion (or sum-of-squares) ASS Estimate the total dispersion, TSS TSS sum of squares of the differences to the overall mean, without consideration for the groups (j 1 ) to which the data belong. variance, Var Tot TSS ij j 1 i 1 i 1 where n. Since Tot n 1, we can use TSS to compute the total s TSS TSS can be transformed as follows: n 1 n i TSS ij j + j j 1i 1 TSS ij j + j Form: [a + b] TSS ij j + ij j j + j TSS ij j + j ij j + j i j

6 Compare several group means: anova 6 The sum of the differences to the mean of a group is zero, b definition of the group mean. Hence the middle element of the above equation is zero. It follows that TSS ij j + j j 1 i 1 j 1 i 1 TSS ij j + j j 1 i 1 j 1 Estimate the within-group (or residual) dispersion, RSS The total variation [sum of (differences to the mean) ] within the groups is not of primar interest. It is seen as experimental variation, called error b statisticians. In ecolog, the within-group variation corresponds to local innovation at the individual stud sites. For each group j, we compute RSS j ij j i 1 Summing these terms over all groups j, we obtain: RSS ij j j 1 i 1 j i ij j T j where T j is the sum of the observed values within group j. Degrees of freedom: R (n 1 1) + (n 1) + + (n 1) n RSS Within-group (residual) mean square (or variance): MS R Var n R

7 Compare several group means: anova 7 Estimate the among-group dispersion, ASS For each group j, we compute the squared difference between the mean of that group and the overall mean, j, then we sum these values over all groups. Before summing, each sum-of-squares j will be weighted b the number of elements in group j. Thus, if there are observations in group j, the dispersion due to that group is j. For groups, T j ASS j j 1 j 1 T n The number of degrees of freedom associated with a statistic is the number of independent components, i.e. the number of basic components in the calculation minus the number of parameters lining them. - The number of basic components are the deviations j. - The means are lined b a single parameter: the general mean. Hence A 1 and the among-group mean square (or variance) is: MS A Interesting relationships: TSS RSS + ASS and Tot n 1 (n ) + ( 1) R + A ASS Var A

8 Compare several group means: anova 8 Example Hpothetical experiment: 15 subjects were treated for headache using 3 pain relievers. The effect was noted on a 0 to 5 scale. Aspirin Paracetamol Placebo The calculation results are presented in an analsis of variance table: d.f. Sums of squares Mean squares Brand ASS MS A Residual 1 RSS MS R Total 14 TSS Vat Tot.0667 Mean squares, abbreviated MS, are variances. Beware: MS A + MS R Vat Tot (i.e., ), even though ASS + RSS TSS ( ). 4 - Two estimates of under H 0 The reasoning presented in this section will allow the construction of a test of significance for the differences among the group means. Assume that the populations, from which the groups of observations were drawn, are normall distributed and that the all have the same variance (i.e., ). 1 If H 0 is true (H 0 : 1 ), then the common variance can be estimated in two different was.

9 Compare several group means: anova 9 First method to estimate A fundamental hpothesis of ANOVA is that each of the group variances j estimates the same common variance. This allows us to loo for a robust estimate of the common variance b computing the weighted mean of the within-group variances. This step introduces the hpothesis of homogeneit of the group variances in the ANOVA test. That hpothesis has to be checed using an appropriate test prior to ANOVA (see section 6). Variance of a group j, weighted b the number of degrees of freedom of that group: 1 ij j i ij j i 1 Mean of the weighted variances of the groups: i i n n 1 ij j j 1 i 1 n RSS n MS R So, MS R is an estimate of the common variance, for H 0 true or not.

10 Compare several group means: anova 10 Second method to estimate If H 0 is true, the means j of the groups are all estimates of the same common mean. The variance of these estimates of can be written: s j j 1 The square root of that variance estimates the standard error of the mean. One can also estimate the standard error of the mean from the standard deviation of the data from a single group: s s n j j which can be squared: s s j If H 0 is true, one can estimate the population variance s j s j n j j b: can be incorporated within the sum to obtain the following estimate of the common variance: s s j j 1 1 ASS MS 1 A So, MS A is an estimate of the common variance if H 0 is true.

11 Compare several group means: anova 11 In summar: if H 0 is true and the groups of observations are drawn from the same statistical population, or from populations having the same mean and same variance, then MS R and MS A are both estimates of. These estimates should be nearl equal. 5 - Comparison test If H 0 is true (H 0 : 1 ), MS R and MS A represent two estimates of. Their ratio is expected to be near 1. In all cases, MS R is an estimate of. One had to chec the equalit of the variances of the populations from which the groups have been drawn (condition of homogeneit of the variances or homoscedasticit: 1 ) before the ANOVA. If H 1 is true, the among-group variance MS A is not an estimate of. Indeed, in that case, the distribution of the means 1,,, does not represent the sampling distribution of a common mean. In that case, the distribution of the means 1,,, is wider and more flat than the sampling distribution of the common mean. MS A is then necessaril larger than MS R. MS R and MS A are two independent components of the total variance since TSS RSS + ASS. If H 0 is true, their ratio (which is near 1) is a test-statistic distributed lie the Fisher-Snedecor F distribution:

12 Compare several group means: anova 1 F A MS A MS R (13-30) The degrees of freedom of the numerator and denominator are respectivel: 1 1 and n. Since MS A is the largest of the two mean squares if H 1 is true, that value is placed in the numerator in order to obtain a value of F A > 1 in that case. The test is one-tailed in all cases. The reason is the following: If H 0 is true, MS A MS R and hence F A 1; if H 1 is true, MS A > MS R and hence F A > 1. Decision rules Do not reject H 0 if F A < F where F is the critical value of F at significance level (e.g. 5%), or if the associated p-value >. Reject H 0 if F A F, or if the associated p-value. Example revisited: Here is the complete analsis of variance table. d.f. Sums of squares Mean squares F p-value Brand ASS MS A Residual 1 RSS MS R Total 14 TSS Vat Tot.0667 R language: functions aov and summar. The classification criterion must be declared as a factor b the command: x as.factor(x).

13 Compare several group means: anova Conditions of application of the F-test in ANOVA - Response variable quantitative (to be able to compute and s ). - Independence of the observations (not autocorrelated). - The population from which each group is drawn must be normal. - Equalit of the variances (homoscedasticit) of the groups. Reason: as in the two-group case, the F-test is simultaneousl testing two different null hpotheses: the equalit of means and the equalit of variances (Behrens-Fisher problem). Tests for equalit of the variances in the R language: Bartlett test (bartlett.test: more power to detect small differences), Levene test (levene.test: good compromise between power and robustness when the data are not quite normal). Violation of the conditions of application: - Homoscedasticit: the ANOVA F-test is robust in the presence of a certain amount of heteroscedasticit. The results thus remain valid if the within-group variances are not quite homogeneous. - Normalit: the ANOVA F-test is also robust in the presence of a certain amount of sewness (asmmetr) and urtosis of the distributions. For sewness ( 3 ), the following criterion can be used: the F-test can be used is 5 3 j. - Strong violation of the normalit condition: 1. Transform the data before analsis.. Test F A b permutations. R functions are available on the Web page 3. Use the nonparametric Krusal-Wallis test (R language: function rusal.test).

14 Compare several group means: anova 14 - If the observations are spatiall or temporall correlated, the test is either too liberal (rejecting H 0 too often) or too conservative (the opposite), depending on the tpe of spatial dependence among the observations. See Legendre et al. (004), Ecolog 85: One must control for the spatial structure in a modified form of the test. 7 - Alternative computation methods If ANOVA is used to test the difference between the means of two groups, the result of the F-test is the same as that of a two-tailed t-test without Welch correction. An ANOVA F-test is equivalent to the F-test of the coefficient of determination in a multiple regression between the response variable and a series of dumm variables representing the groups, shown in section (right-hand representation of x). See Practicals.

15 One-wa ANOVA. Example 1: H 0 is true Classification criterion Observations { Group 1 Group Group Within-group dispersion Group 1 Group Group Total dispersion Amonggroup dispersion n n 1 n ( i1 1 ) 0 n 1 5 T Σ( i1 1 ) 8.50 n 5 T Σ( i ) 9.36 n 3 5 T Σ( i3 3 ) n 15 T Σ( ij ) 9.6 Σ ( j ) 0.10 ASS TSS RSS 9.16 TSS ASS + RSS Sources of variation Dispersions (SS) Degrees of freedom Mean squares (variances) Total Among-group Within-group TSS 9.6 ASS 0.10 RSS / MS A 0.10 / 0.05 MS R 9.16 / 1.43 F MS A /MS R P F (0.05,, 1) 3.89 Daniel Borcard, 004

16 One-wa ANOVA. Example : H 0 is false Critère de classification Observations { Group 1 Group Group Within-group dispersion Group 1 Group Group 3 Total dispersion Amonggroup dispersion n n 1 5 T Σ( i1 1 ) n 5 T Σ( i ) 9.36 ( i3 3 ) n 3 5 T Σ( i3 3 ) n 1 n 1 n 15 Σ ( j ) T ASS Σ( ij ) TSS RSS 9.16 TSS ASS + RSS Sources of variation Dispersions (SS) Degrees of freedom Mean squares (variances) Total Among-group Within-group TSS ASS RSS / MS A / 8.7 MS R 9.16 / 1.43 F MS A /MS R 11.8 P F (0.05,, 1) 3.89 F (0.01,, 1) 6.93 Daniel Borcard, 004

One-way ANOVA. Experimental Design. One-way ANOVA

One-way ANOVA. Experimental Design. One-way ANOVA Method to compare more than two samples simultaneously without inflating Type I Error rate (α) Simplicity Few assumptions Adequate for highly complex hypothesis testing 09/30/12 1 Outline of this class