The One-Way Independent-Samples ANOVA. (For Between-Subjects Designs)

The One-Way Independent-Samples ANOVA (For Between-Subjects Designs)

Computations for the ANOVA In computing the terms required for the F-statistic, we won t explicitly compute any sample variances or standard deviations Instead, in all intermediate steps, we ll deal exclusively with sums of squared deviations (SS) and means of squared deviations (MS) Computing the F-statistic using sample standard deviations or variances (as we did in the last lecture) gets you the same answer, but requires more calculations

Computations for the ANOVA: Preliminaries x x Start by computing and for each group, then compute: Grand total: The overall total, computed over all scores in all groups (samples) x T Total sum of squared scores: The sum of squared scores computed over all scores in all groups x k i n j x ij k n T xij i j

Computations for the ANOVA: SS terms Intro to ANOVA SS total : The sum of squared deviations of all observations from the grand mean or total SS x M x T T x T N SS SS SS total within Not strictly needed for computing the F ratio, but it makes computing the needed SS terms much easier

Computations for the ANOVA: SS terms Intro to ANOVA Total Variance SS df SS SS total within df df total within Between Treatments Variance Within Treatments Variance SS df SS SS total within df df total within SS df SS SS within total df df within total

Computations for the ANOVA: SS terms Intro to ANOVA SS : The sum of squared deviations of the sample means from the grand mean multiplied by the number of observations k SS n M M i i T i k i x xt n i i N or SS SS total SS within SS within (SS error ): The sum of squared deviations within each sample k SSwithin SS j SS1 SS... SS k j or SS SS SS within total

Computations for the ANOVA: df terms df total = N-1 : degrees of freedom associated with SS total N is the total number of scores Intro to ANOVA df = k-1 : degrees of freedom associated with SS k is the number of groups (samples) df within (or df error )= df total -df = N-k : degrees of freedom associated with SS within Can also be computed as: 1 k 1 df df... df n 1 n 1... n 1 k

Computing the F-statistic MS MS within SS df SS df within within F df, dfwithin MSwithin MS

The One-Way ANOVA: Steps 1. State Hypotheses. Compute F-ratio statistic: F df, dfwithin MSwithin MS For data in which I give you raw scores, you will have to compute: Sample means SS total, SS, & SS within df total, df, & df within 3. Use F-ratio distribution table to find critical F-value representing rejection region 4. Make a decision: does the F-statistic for your sample fall into the rejection region?

The One-Way ANOVA: Textbook Example Intro to ANOVA A psychologist wants to determine whether having a job interferes with student academic performance. She measures academic performance using students GPAs. She selects a sample of 30 students. Of these students,10 did not work, 10 worked part-time, and 10 worked full-time during the previous semester Conduct an ANOVA at a.05 level of significance testing the hypothesis that having a job interferes with student performance

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 x x3 xt x x 1 x xt n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between Within (error) Total 1. Compute degrees of freedom df df df total within N 1 9 k 1 N k 7

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 x x3 xt x x 1 x xt n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between Within (error) 7 Total 9. Compute SS total SS total x T 97.10 319.47 30 319.47 314.8 5.9 1 x N T

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 x x3 xt x x 1 x xt n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between Within (error) 7 Total 9 5.19 3. Compute SS (or SS within ) directly SS n M M T 10 3.43 3.4 10 3.3 3.4 10.98 3.4 0.361 0.036 0.676 1. 07

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 x x3 xt x x 1 x xt n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between 1.07 Within (error) 7 Total 9 5.19 4. Compute the missing SS value (SS or SS within ) via subtraction: SS SS SS within total 5.19 1.07 4. 1

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 34.30 x 33.00 x3 9.80 xt 97.10 x 119.07 x 109.90 x 90.50 x 319.47 1 T n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between 1.07 Within (error) 7 4.1 Total 9 5.19 5. Compute the MS values needed to compute the F ratio: MS SS 1.07 0.535 df MS within SSwithin 4.1 0.153 df 7 within

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 34.30 x 33.00 x3 9.80 xt 97.10 x 119.07 x 109.90 x 90.50 x 319.47 1 T n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between 1.07 0.535 Within (error) 7 4.1 0.153 Total 9 5.19 6. Compute the F ratio: F df F, df error,7 3.50 0.535 0.153 MS MS error

F table for α=0.05 reject H 0 df error df numerator 1 3 4 5 6 7 8 9 10 1 161.45 199.50 15.71 4.58 30.16 33.99 36.77 38.88 40.54 41.88 18.51 19.00 19.16 19.5 19.30 19.33 19.35 19.37 19.38 19.40 3 10.13 9.55 9.8 9.1 9.01 8.94 8.89 8.85 8.81 8.79 4 7.71 6.94 6.59 6.39 6.6 6.16 6.09 6.04 6.00 5.96 5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.8 4.77 4.74 6 5.99 5.14 4.76 4.53 4.39 4.8 4.1 4.15 4.10 4.06 7 5.59 4.74 4.35 4.1 3.97 3.87 3.79 3.73 3.68 3.64 8 5.3 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 9 5.1 4.6 3.86 3.63 3.48 3.37 3.9 3.3 3.18 3.14 10 4.96 4.10 3.71 3.48 3.33 3. 3.14 3.07 3.0.98 11 4.84 3.98 3.59 3.36 3.0 3.09 3.01.95.90.85 1 4.75 3.89 3.49 3.6 3.11 3.00.91.85.80.75 13 4.67 3.81 3.41 3.18 3.03.9.83.77.71.67 14 4.60 3.74 3.34 3.11.96.85.76.70.65.60 15 4.54 3.68 3.9 3.06.90.79.71.64.59.54 16 4.49 3.63 3.4 3.01.85.74.66.59.54.49 17 4.45 3.59 3.0.96.81.70.61.55.49.45 18 4.41 3.55 3.16.93.77.66.58.51.46.41 19 4.38 3.5 3.13.90.74.63.54.48.4.38 0 4.35 3.49 3.10.87.71.60.51.45.39.35 4.30 3.44 3.05.8.66.55.46.40.34.30 4 4.6 3.40 3.01.78.6.51.4.36.30.5 6 4.3 3.37.98.74.59.47.39.3.7. 8 4.0 3.34.95.71.56.45.36.9.4.19 30 4.17 3.3.9.69.53.4.33.7.1.16 40 4.08 3.3.84.61.45.34.5.18.1.08 50 4.03 3.18.79.56.40.9.0.13.07.03 60 4.00 3.15.76.53.37.5.17.10.04 1.99 10 3.9 3.07.68.45.9.18.09.0 1.96 1.91 00 3.89 3.04.65.4.6.14.06 1.98 1.93 1.88 500 3.86 3.01.6.39.3.1.03 1.96 1.90 1.85 1000 3.85 3.00.61.38..11.0 1.95 1.89 1.84

Work Status No Work Part-Time Full-Time 3.40 3.50.90 3.0 3.60 3.00 3.00.70.60 3.00 3.50 3.30 3.50 3.80 3.70 3.80.90.70 3.60 3.40.40 4.00 3.0.50 3.90 3.30 3.30.90 3.10 3.40 x1 x x3 xt x x 1 x xt n 1 =10 n =10 n 3 =10 N =30 M 1 =3.43 M =3.3 M 3 =.98 M T =3.4 Set up a summary ANOVA table: Source df SS MS F Between 1.07 0.535 3.50 Within (error) 7 4.1 0.153 Total 9 5.19 7. Compare computed F statistic with F crit and make a decision F crit 3.37 3.5 3.37; reject H 0 Conclusion: Having a job does significantly interfere with academic performance

The One-way ANOVA: Example Return to our running example: Do test scores vary as a function of the instructor? x 1 : sample scores from Dr. M s class x : sample scores from Dr. K s class x 3 : sample scores from Dr. A s class Null Hypothesis H 0 : µ 1 = µ = µ 3 Research Hypothesis H 1 : one of the population means is different Do we accept or reject the null hypothesis? Assume α = 0.05

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between Within (error) Total 465.04 1. Compute degrees of freedom df df df total within N 1 4 k 1 N k

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between Within (error) Total 4 465.04 3. Compute SS within (or SS ) directly (This time, we ll compute SS within ) SSwithin SS 89.71 91.60 68.00 49.31

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between Within (error) 49.31 Total 4 465.04 4. Compute the missing SS value (SS or SS within ) via subtraction: SS SS SS total within 465.04 49. 31 15.73

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between 15.73 Within (error) 49.31 Total 4 465.04 5. Compute the MS values needed to compute the F ratio: MS SS 15.73 107.87 df MS within SSwithin 49.31 11.33 df within

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between 15.73 107.87 Within (error) 49.31 11.33 Total 4 465.04 6. Compute the F ratio: F df F, df error, 9.5 107.87 11.33 MS MS error

Instructor Dr. M Dr. K Dr. A 73 6 7 71 66 68 76 66 70 68 66 6 65 58 69 7 61 66 75 67 65 67 68 67 6 n 1 =7 n =10 n 3 =8 N =5 M 1 =71.43 M =64.0 M 3 =67.50 M T =67.8 SS 1 =89.71 SS =91.60 SS 3 =68.00 SS T =465.04 Set up a summary ANOVA table: Source df SS MS F Between 15.73 107.87 9.5 Within (error) 49.31 11.33 Total 4 465.04 7. Compare computed F statistic with F crit and make a decision F crit 3. 44 9.5 3.44; reject H 0 Conclusion: Student test scores do vary across instructors

Effect Size for the One-Way ANOVA For ANOVAs, effect sizes are usually indicated using the R -family measure eta-squared (η ) R -family measures indicate the effect size in terms of proportion of variance accounted for by the treatment effect(s) For our example: variability explained by treatment effect R total variability SS SS 15.73 465.04 total 0.46

Post-hoc Tests for Multiple Comparisons Intro to ANOVA Rejecting H 0 only tells us that the omnibus null hypothesis (that all sample means are equal) is false However, we are often interested in knowing which particular means differ from each other Evaluating differences (usually pairwise) beyond the omnibus null hypothesis requires post-hoc testing

Post-hoc Tests The challenge in constructing a post-hoc multiple comparison test is keeping the experimentwise α low while maximizing the power of the test Power refers to the ability of a statistical test to pick up true differences population means Researchers use many different post-hoc tests tailored to particular families of comparisons. Most of these tests are based on the t-test We will cover two such tests: Fisher s LSD (protected t-test) The Bonferroni procedure

Fisher s Least Significant Difference (LSD) Test Fisher s LSD (protected t) test was the first proposed method for post-hoc pairwise comparisons It is nearly identical to the independent measures t-test. The only differences are that the denominator uses MS error in place of pooled variance and uses df error as the degrees of freedom for the t-statistic 1 t df error MS n M error 1 M MS n The t is protected in that the omnibus null hypothesis must be rejected for this test to be valid The test is fairly liberal, producing higher than intended experimentwise α for post-hoc tests involving more than 3 pairwise comparisons error

The Bonferroni Procedure The Bonferroni procedure simply adjusts the pairwise alpha for a group of comparisons to ensure that, in the worst case scenario, the experimentwise alpha will never exceed 0.05 The worst case occurs when the rejection of H 0 under different pairwise comparisons is mutually exclusive In this case, via the additive rule, the probability of falsely rejecting H 0 in k comparisons is α 1 + +α k = kα

The Bonferroni Procedure Thus, the Bonferroni procedure requires that you divide the pairwise alpha by the number of comparisons. For example, if you wanted to make 10 pairwise comparisons at a desired experimentwise α of 0.05, you would choose the rejection region using a pairwise criterion of α/10 =0.005 The Bonferroni procedure is a very conservative test. It is guaranteed to keep the experimentwise Type I error rate below α but is more likely to lead to Type II errors (acceptance of H 0 when it is false). The formula for the Bonferroni procedure is exactly like that for Fisher s LSD test.

Post hoc tests: Example (Fisher s LSD) Intro to ANOVA ANOVA Summary Table Source df SS MS F Between 1.07 0.535 3.50 Within (error) 7 4.1 0.153 Total 9 5.19 M M M n 1 3 3.43 3.3.98 n n 1 3 10 Let s do all possible comparisons: {1,},{1,3},{,3} t-statistic for Fisher s LSD test when comparing {A,B}: First, note that the denominator is the same for all comparisons: t df error MS n M A error A M B MS n error B t 7 M M M M M M 0.153 0.153 0.0306 10 10 A B A B A B

Post hoc tests: Example (Fisher s LSD) Intro to ANOVA M M M 1 3 3.43 3.3.98 Let s do all possible comparisons: {1,},{1,3},{,3} Now we simply apply this formula to all comparisons: {1,} {1,3} {,3} t M M 3.43 3.3 0. 13 0.74 1 7 t M M 3.43.98 0. 45.571 0. 175 1 3 7 t M M 3.3.98 0.35.0 3 7

Post hoc tests: Example (Bonferroni) M M M 1 3 3.43 3.3.98 Let s do all possible comparisons: {1,},{1,3},{,3} We have three comparisons, so the Bonferroni correction to α would be 0.05.017 # comparisons 3 {1,} {1,3} {,3} t M M 3.43 3.3 0. 13 0.74 1 7 t M M 3.43.98 0. 45.571 0. 175 1 3 7 t M M 3.3.98 0.35.0 3 7

t-distribution Table α t One-tailed test α/ α/ -t t Two-tailed test Level of significance for one-tailed test 0.5 0. 0.15 0.1 0.05 0.05 0.01 0.005 0.0005 Level of significance for two-tailed test df 0.5 0.4 0.3 0. 0.1 0.05 0.0 0.01 0.001 1 1.000 1.376 1.963 3.078 6.314 1.706 31.81 63.657 636.619 0.816 1.061 1.386 1.886.90 4.303 6.965 9.95 31.599 3 0.765 0.978 1.50 1.638.353 3.18 4.541 5.841 1.94 4 0.741 0.941 1.190 1.533.13.776 3.747 4.604 8.610 5 0.77 0.90 1.156 1.476.015.571 3.365 4.03 6.869 6 0.718 0.906 1.134 1.440 1.943.447 3.143 3.707 5.959 7 0.711 0.896 1.119 1.415 1.895.365.998 3.499 5.408 8 0.706 0.889 1.108 1.397 1.860.306.896 3.355 5.041 9 0.703 0.883 1.100 1.383 1.833.6.81 3.50 4.781 10 0.700 0.879 1.093 1.37 1.81.8.764 3.169 4.587 11 0.697 0.876 1.088 1.363 1.796.01.718 3.106 4.437 1 0.695 0.873 1.083 1.356 1.78.179.681 3.055 4.318 13 0.694 0.870 1.079 1.350 1.771.160.650 3.01 4.1 14 0.69 0.868 1.076 1.345 1.761.145.64.977 4.140 15 0.691 0.866 1.074 1.341 1.753.131.60.947 4.073 16 0.690 0.865 1.071 1.337 1.746.10.583.91 4.015 17 0.689 0.863 1.069 1.333 1.740.110.567.898 3.965 18 0.688 0.86 1.067 1.330 1.734.101.55.878 3.9 19 0.688 0.861 1.066 1.38 1.79.093.539.861 3.883 0 0.687 0.860 1.064 1.35 1.75.086.58.845 3.850 1 0.686 0.859 1.063 1.33 1.71.080.518.831 3.819 0.686 0.858 1.061 1.31 1.717.074.508.819 3.79 3 0.685 0.858 1.060 1.319 1.714.069.500.807 3.768 4 0.685 0.857 1.059 1.318 1.711.064.49.797 3.745 5 0.684 0.856 1.058 1.316 1.708.060.485.787 3.75 6 0.684 0.856 1.058 1.315 1.706.056.479.779 3.707 7 0.684 0.855 1.057 1.314 1.703.05.473.771 3.690 8 0.683 0.855 1.056 1.313 1.701.048.467.763 3.674 9 0.683 0.854 1.055 1.311 1.699.045.46.756 3.659 30 0.683 0.854 1.055 1.310 1.697.04.457.750 3.646 40 0.681 0.851 1.050 1.303 1.684.01.43.704 3.551 50 0.679 0.849 1.047 1.99 1.676.009.403.678 3.496 100 0.677 0.845 1.04 1.90 1.660 1.984.364.66 3.390

Post hoc tests: Example M M M 1 3 3.43 3.3.98 Let s do all possible comparisons: {1,},{1,3},{,3} Now we simply apply this formula to all comparisons: {1,} {1,3} {,3} Fisher s LSD: t M M 3.43 3.3 0. 13 0.74 1 7 0.74.05, retain H 0 t M M 3.43.98 0. 45.571 0. 175 1 3 7 3 7.571.05, reject H0.0.05, retain H0 t M M 3.3.98 0.35.0 Bonferroni: 0.74.473, retain H0.571.473, reject H0.0.473, retain H0