Comparing Several Means: ANOVA Understand the basic principles of ANOVA Why it is done? What it tells us? Theory of one way independent ANOVA Following up an ANOVA: Planned contrasts/comparisons Choosing contrasts Coding contrasts Post hoc tests Slide 1
When and Why When we want to compare means we can use a t test. This test has limitations: You can compare only 2 means: often we would like to compare means from 3 or more groups. It can be used only with one predictor/independent variable. ANOVA Compares several means. Can be used when you have manipulated more than one independent variable. It is an extension of regression (the general linear model). Slide 2
ANOVA Fisher: British statistician and geneticists. Introduced this analysis Let us assume that we test four different feeds and want to see if the body weights in pigs changes using the different feeds. We are going to test the effect of one factor feed type. The analysis is termed a one factor test or oneway ANOVA. A population of pigs is assigned, at random to each of the four treatments. To be specific there are four treatment levels. Parametric test
Why Not Use Lots of t Tests? If we want to compare several means why don t we compare pairs of means with t tests? Can t look at several independent variables Inflates the Type I error rate Type one error rate = 1 0.95 n
What Does ANOVA Tell Us? Null hypothesis: Like a t test, ANOVA tests the null hypothesis that the means are the same. Experimental hypothesis: The means differ. ANOVA is an omnibus test It test for an overall difference between groups. It tells us that the group means are different. It doesn t tell us exactly which means differ. Slide 5
What Does ANOVA Tell Us? If H 0 is rejected, there is al least one difference among the four means.
ANOVA as Regression
Placebo Group
High Dose Group
Low Dose Group
Output from Regression
Experiments vs. Correlation ANOVA in regression: Used to assess whether the regression model is good at predicting an outcome. ANOVA in experiments: Used to see whether experimental manipulations lead to differences in performance on an outcome. By manipulating a predictor variable can we cause (and therefore predict) a change in behavior? Same question is of interest in regression and experimental maniupulations: In experiments we systematically manipulate the predictor, in regression we don t. Slide 12
Theory of ANOVA We calculate how much variability there is between scores Total sum of squares (SS T ). We then calculate how much of this variability can be explained by the model we fit to the data How much variability is due to the experimental manipulation, model sum of squares (SS M )... and how much cannot be explained How much variability is due to individual differences in performance, residual sum of squares (SS R ). Slide 13
Theory of ANOVA We compare the amount of variability explained by the model (experiment), to the error in the model (individual differences) This ratio is called the F ratio. If the model explains a lot more variability than it can t explain, then the experimental manipulation has had a significant effect on the outcome. Slide 14
Theory of ANOVA Slide 15 If the experiment is successful, then the model will explain more variance than it can t SS M will be greater than SS R
ANOVA by Hand Testing the effects of Viagra on libido using three groups: Placebo (sugar pill) Low dose viagra High dose viagra The outcome/dependent variable (DV) was an objective measure of libido. Slide 16
Slide 17 The Data
Step 1: Calculate SS T ** where the mean is the grand mean
Total Sum of Squares (SS T ): 8 7 6 5 4 3 2 1 0 Grand Mean 0 1 2 3 4 Slide 20
Response Value 8 7 6 5 4 3 2 1 0 1 6 11 Participant Number
SS T = sum((observed Grand Mean) 2 ) SS T = S 2 (N 1)
Degrees of Freedom Degrees of freedom (df) are the number of values that are free to vary. In general, the df are one less than the number of values used to calculate the SS. DF Total = N 1 Slide 23
Model Sum of Squares (SS M ): Difference between the model estimate and the mean (or Grand Mean )
Model Sum of Squares (SS M ): 8 7 6 5 4 3 2 1 0 Grand Mean 0 1 2 3 4 Slide 25
Slide 26 Step 2: Calculate SS M
Model Degrees of Freedom How many values did we use to calculate SS M? We used the 3 means. Slide 27
Residual Sum of Squares (SS R ): 8 7 6 5 4 3 2 1 0 Grand Mean 0 1 2 3 4 Df = 4 Df = 4 Df = 4 Slide 28
Step 3: Calculate SS R SS R = sum([x i x i ] 2 )
Step 3: Calculate SS R 2.5 Slide 30
Residual Degrees of Freedom How many values did we use to calculate SS R? We used the 5 scores for each of the SS for each group. Slide 31
Double Check SS T = SS R + SS M 43.74 = 20.14 + 23.60 DF T = DF R + DF M 14 = 2 + 12 Slide 32
Step 4: Calculate the Mean Squared Error Slide 33
Slide 34 Step 5: Calculate the F Ratio
Step 6: Construct a Summary Table Source SS df MS F Model 20.14 2 10.067 5.12* Residual 23.60 12 1.967 Total 43.74 14 Slide 35
Multiple comparison tests The anova that you examined is used to test the hypothesis that there is no difference in the sample means among k treatment levels However we cannot conclude, after doing the test, which of the mean values are different from one another.
Tukey Test Tukey test balanced, orthogonal designs Step one: is to arrange and number all five sample means in order of increasing magnitude Calculate the pairwise difference in sample means. We use a t test analog to calculate a q statistic
Tukey Test S 2 is the error mean sqare by anova computation n is the of data in each of groups B and A Remember this is a completely balanced design.
Tukey Test Start with the largest mean, vs. the smallest mean. Then when the first largest mean has been compared with increasingly large second means, use the second largest mean. If the null hypothesis is accepted between two means then all other means within that range cannot be different.