Study Guide #3: OneWay ANALYSIS OF VARIANCE (ANOVA)

Transcription

1 Study Guide #3: OneWay ANALYSIS OF VARIANCE (ANOVA) About the ANOVA Test In educational research, we are most often involved finding out whether there are differences between groups. For example, is there a difference between males and female, between rural and urban students and so forth. The t-test is often used to compare differences of means between two groups, such as comparing outcomes between control and treatment groups in an experimental study. The t-test is a useful tool for comparing the means of two groups; however, the t-test is not good in situations calling for the comparison of three or more groups. With three or more groups, the t- test is not an effective statistical tool. On a practical level, using the t-test to compare many means is a cumbersome process in terms of the calculations involved. SO WHAT DO WE DO? There are many kinds of questions in which we might want to compare the means of several different groups at once. For example, we may be comparing the opinions of low, middle and high income parents; the perceptions of Malays, Chinese, Indians, Kadazandusun and Iban youths on drug addiction. 1

2 In such situations, the preferred statistical tool is the ANOVA, (Analysis Of Variance). Like the t-test, ANOVA can be used to examine differences among the means of several different groups at once. Mean 1 Mean 2 Mean 3 Mean Example: A researcher is interested in finding out whether there are differences in creative thinking among students of varying IQ levels. Creative thinking is measured using The Torrance Test of Creative Thinking while IQ is measured using the Raven s Progressive Matrices instrument and divided into 3 Groups (High, Average and Low). The null hypothesis generated is that all three groups will have the same mean score on the creative test. In formula terms, if we use the symbol μ [pronounced as mew ] to represent the average score, the null hypothesis is expressed through the following notation:

3 3 Null Hypothesis: Ho : μ1 = μ2 = μ3 Graphically, the null hypothesis can be represented in the following manner (see Figure 1): Creative Thinking Mean (Dependent Variable) High IQ Average IQ Low IQ Mean Figure 1: Null hypothesis Notice in the graph that all three groups have the same average score (all three points are on the dashed line) and all three groups have the same SD (noted by the fact that the line around the mean point for each group is the same size). So the null hypothesis is that all three groups will have the same average score on the Torrance Test of Creativity.

4 The alternate hypothesis is that all means are not the same. It's important to point out that the opposite is not that all means are different (i.e., μ1 μ2 μ3 ). It is possible that some of the means could be the same, yet if they are not all identical, we would reject the null hypothesis. 4 Creative Thinking (Dependent Variable) Mean = 4.00 High IQ Average IQ Low IQ Mean = SD = n = Figure 2: Alternative hypothesis Figure 2 highlights TWO important features in understanding ANOVA: o differences in scores within groups (i.e. student within group 1 have different scores)

5 o differences in scores between groups (i.e. students in group 1, in group 2 and in group 3 have different scores) 5 Step 1: Calculation of the Variation or Variances Between Groups The first step is to calculate the variation between groups by comparing the mean of each IQ group with the Mean of the Overall Sample (the mean score on the test for all students in this sample which is 4.00). This measure of between-group variance is referred to as "Between Sum of Squares" (or BSS). This is calculated by adding up, for all groups, the difference between the group's mean and the overall population mean (4.00), multiplied by the number of cases in the group. Between Sum of Squares = No. of students (Mean Group 1 Overall Mean)² + No. of students (Mean Group 2 Overall Mean)² + No. of students (Mean Group 3 Overall Mean)² +

6 Between Sum of Square = 313 ( )² ( )² ( )² = = This sum of squares has a number of degrees of freedom equal to the number of groups minus 1. In this case, df B = (3-1) = 2 Step 2: Calculation of the Between Mean Squares We divide the BSS figure by the number of degrees of freedom to get our estimate of the variation between groups, referred to as "Between Mean Squares" as: BSS Between Mean Square = = = df 2

7 7 Step 3: Calculation of the Variation Within Groups To measure the variation within groups, we find the sum of the squared deviation between scores on the Torrance Creative Test and the group average, calculating separate measures for each group, then summing the group values. This is a sum referred to as the "Within Sum of Squares" (or WSS). Within Sum of Squares = (313 1) 1.28² + (297 1) 1.30² + (340 1) 1.31² = =

8 As in Step 1, we need to adjust the WSS to transform it into an estimate of population variance, an adjustment that involves a value for the number of degrees of freedom within. To calculate this, we take a value equal to the number of cases in the total sample (N = 950), minus the number of groups (k = 3). 8 Then we can calculate the a value for "Within Mean Squares" as WSS Within Mean Square = = = 1.13 df 947

9 Step 3: Calculation of the F test statistic This calculation is relatively straightforward. Simply divide the Between Mean Squares, the value obtained in step 1, by the Within Mean Squares, the value calculated in step 2. 9 Between Mean Squares F = = = Within Mean Squares 1.13 Then compare this value to a standard table with values for the F distribution to calculate the significance level for the F value. In this case, the significance level is less than.01. This is extremely strong evidence against the null hypothesis, indicating that students' performance varies significantly across the three groups.

10 10 The SPSS Output: When calculating the OneWay ANOVA, using SPSS, the figures given will presented in the following manner: Source Between Sum of Squares BSS Degrees of Freedom dfb Mean Squares Between Mean Squares BSS / dfb Within WSS dfw Within Mean Squares WSS / dfw Total TSS = BSS + WSS

11 11 To fill in this table with the data from the example above, we have: Source Sum of Squares Degrees of Freedom Mean Squares F Between Within * Total * p <.05 If there is a significant difference between the 3 Groups, which of the three Groups are different?

12 To establish which of the 3 Groups are different, you have to follow up with Tukey s HSD tests. 12 Tukey HSD Creativity Scores N Subset for alpha =.05 IQ Group 1 2 low average high The Tukey HSD table shows that: o There is a significant difference between the Average IQ and Low IQ groups at p < 0.05 o There is no significant difference between the Low IQ group and the High IQ group. o There is no significant difference between the Average IQ group and the High IQ group.