One-Way ANOVA Source Table ANOVA MODEL: ij = µ* + α j + ε ij H 0 : µ = µ =... = µ j or H 0 : Σα j = 0 Source Sum of Squares df Mean Squares F J Between Groups nj( j * ) J - SS B /(J ) MS B /MS W = ( N J ) SS ( Explained Variance) ( J ) SS Within Groups ( Error Variance) (Error Variance) Total Variance j= J n j ( i j) N J SS W /df W j= i= N ( i * ) N - i= s = SS T /N- where, N = total number of cases, J = number of groups, * = the grand mean of across all groups. i = each individual score on, and j = the mean for group j. n j = the number of cases in group j. R = η = SS B /SS T is the Proportion of Variance Explained by Group Differences. This is also known a Proportional Reduction in Error (PRE). Pairwise Post-Hoc Comparisons of Means However, a statistically significant F ratio only indicates that assuming the Null Hypothesis, H 0 : µ = µ =... = µ j, The Results were not likely to have occurred by chance. Or, we interpret this as, at least one mean is different. We don t know which one(s)! A common approach is to use a post-hoc test for group comparisons. I personally like Tukey s Honestly Significant Difference (HSD) for pairwise comparisons of means. Tukey s HSD controls for inflation of the Type I error rate when J(J - )/ pairwise comparsions are made. Because of this adjustment of the significance level, it is possible to obtain a statistically significant F-ratio when no pairwise comparisons are significant. The q statistic for Tukey s HSD can be computed as follows: q = L - S MS W ( + ) n L n S, Then q is compared to a critical value obtained from the Studentized Range Table (q-distribution). Thus, when two means are compared in a pairwise fashion, if the calculated q statistic is larger than the q-critical value then this pairwise difference in means is statistically significant. Similarly, one can solve for the Minimum Mean Difference that would be Honestly Significant MSW HSD = q ( + ) nl ns This HSD concept can also be used to construct Confidence Intervals: MSW CI: ( L - S ) ± q ( + ) nl ns If the Confidence Interval contains the null value of ZERO then you cannot claim an Honestly Significant Difference. If the Confidence DOES NOT contain the null value of ZERO then you can claim that there is an Honestly Significant Difference. W B
ANOVA MODEL: ij = µ* + α j + ε ij Total Within Group Between Group * ( - * ) ( - * ) j ( - j ) ( - j ) j * ( j - * ) ( j - * ) 6-4 6 Occupational 3-3 6-3 9 3 6-3 9 Therapy 3 3 0 0 3 6-3 9 6-4 6 (j = ) (OCT) 3 - SS W = 4 3 6-3 9 4 6-4 n = 5 4 3 S = 3 6-3 9 4 6-4 = 3 4 3 S = 3 6-3 9 5 6 - Maternal 5 6-6 6 0 0 6 6 0 0 Child Health 6 6 0 0 6 6 0 0 7 6 (j = ) (MCH) 7 6 SS W = 0 6 6 0 0 4 6-4 n = 5 4 6-4 S =.5 6 6 0 0 8 6 4 = 6 8 6 4 S =.58 6 6 0 0 0 6 4 6 Nutrition 0 9 9 6 3 9 9 6 3 9 Science 9 9 0 0 9 6 3 9 9 6 3 9 (j = 3) (NTS) 9 9 0 0 SS W3 = 9 6 3 9 8 6 4 n 3 = 5 8 9 - S 3 =.5 9 6 3 9 9 6 3 9 3 = 9 9 9 0 0 S 3 =.707 9 6 3 9 5 6 - Epidemiology 5 6-6 6 0 0 7 6 7 6 6 6 0 0 6 6 0 0 (j = 4) (EPI) 6 6 0 0 SS W4 = 4 6 6 0 0 7 6 n 4 = 5 7 6 S 4 = 6 6 0 0 5 6-4 = 6 5 6 - S 4 = 6 6 0 0 * = 6 SS T = 0 SS W = 0 SS W = 0 SS B = 90 (Σα j ) estimated by SS Between = Σ n j ( j - * ) = 5(3-6) + 5(6-6) + 5(9-6) + 5(6-6) = 90. (Σε ij ) estimated by SS Within = Σ( S j )( n j-) = (x4) + (.5x4) + (.5x4) + (x4) = 0 ANOVA Source Table H 0 : µ = µ = µ 3 = µ 4 Source SS df MS F Between 90 J- =3 90/3 = 30 30/.5 = 4.00 (Explained) Within 0 N-J = 0-4 =6 0/6 =.5 (Error) Total 0 N - = 9 0/9 = 5.79 η = 90/0 =.8 F(3,6) = 4.00, p <.05, η =.8. Reject H 0 : µ = µ = µ 3 = µ 4. Pairwise Comparisons using Tukey s HSD. q jk = j - k ( MSw/)(/n j + / n k ) q = - 3-6 q = (.5/ )(/ n + / n ) (.5/ ) ( / 5 + / 5) = 3/(.5) = 6. q = 6 > 4.046 from Studentized Range Table; thus, p <.05. q 3 =, p <.05. q 4 = 6, p <.05. q 3 = 6, p <.05. q 4 = 0, ns, p >.05. q 34 = 6, p <.05. MSW Conclusion: H A : µ < (µ = µ 4 ) < µ 3 The HSD = q ( + ) = 4.046(0.5) =.03 nl ns The 95% CI for the difference between Groups an d is: 3 ±.03 (0.977 5.03). The 95% CI does NOT CONTAIN ZERO, so there is an Honestly Significant Difference between OCT and MCH.
Complex Contrasts of Means. H 0 : ψ = 0, where ψ = a + a +... + a J J and Relationship of One-Way and Two-Way ANOVA a + a +... + aj = 0. In this case, ψ = a + a + a 3 3 + a 4 4 and a + a + a3. + a4 = 0. For example, comparing Nutrition Science (j = 3) to a combination of Maternal Child Health (j = ) and Epidemiology (j = 4) yields; ψ = (0)(3) + (/)(6) + (-)(9) + (/)(6) or a simpler method ψ = (0)(3) + ()(6) + (-)(9) + ()(6) = -6. The error term for any contrast of this form is: SE ψˆ j= j J a = ( MSW ) n. j For this contrast SE ψ = (.5)(0/5 + /5 + 4/5 + /5) = (.5)(.) =.5 Note this assumes a common or pooled error term (MS W ) An F test with and df w degrees-of-freedom is used to test the statistical significance of this contrast. F(, df w ) = ψ /SE ψ = (-6) /.5 = 36/.5 = 4.00, which is statistically significant. Thus, F(,6) = 4.00, p <.05, Reject H 0 : ψ = 0 To compare Occupational Therapy (j = ) to a combination of the other three groups yields; ψ = (-3)(3) + ()(6) + ()(9) + ()(6) =. SE ψ = (.5)(9/5 + /5 + /5 + /5) = (.5)(.4) = 3; and F = () / 3 = 48.00. Thus, F(,6) = 48.00, p <.05, Reject H 0 : ψ = 0 Pairwise Effect Sizes. ES jk = j - k ( SS T / ( N -) For example, ES = (3-6)/.4 = -.5. F tests can also be converted to Effect Sizes by the following: ES = df n F d f n F + df d or r = df n F d f d 3
data progs; data progs; input group name $ rate school $ sch orient $ ori; datalines; OCT SHRP APP OCT 3 SHRP APP OCT SHRP APP OCT 4 SHRP APP OCT 4 SHRP APP MCH 5 SOPH - APP MCH 6 SOPH - APP MCH 7 SOPH - APP MCH 4 SOPH - APP MCH 8 SOPH - APP 3 NTS 0 SHRP RES - 3 NTS 9 SHRP RES - 3 NTS 9 SHRP RES - 3 NTS 8 SHRP RES - 3 NTS 9 SHRP RES - 4 EPI 5 SOPH - RES - 4 EPI 7 SOPH - RES - 4 EPI 6 SOPH - RES - 4 EPI 7 SOPH - RES - 4 EPI 5 SOPH - RES - ; proc glm data=progs order=data;class name; model rate = name;means name/tukey cldiff ; contrast 'NTS vs (MCH-EPI)' name 0 - ; contrast 'OCT VS OTHERS' name -3 ; contrast 'SHRP vs SOPH' name - -; contrast 'Appl vs Research' name - -; contrast 'Interaction' name - - ; run; proc glm data=progs ;class group; model rate = group;means group/tukey; contrast 'NTS vs (MCH-EPI)' group 0 - ; contrast 'OCT VS OTHERS' group -3 ; contrast 'SHRP vs SOPH' group - -; contrast 'Appl vs Research' group - -; contrast 'Interaction' group - - ; run; proc glm data=progs;class school orient; model rate = school orient school*orient;run; proc glm data=progs; model rate = sch ori sch*ori;run; Relationship of One-Way and Two-Way ANOVA 4
Dependent Variable: rate Relationship of One-Way and Two-Way ANOVA The GLM Procedure Class Level Information Class Levels Values name 4 OCT MCH NTS EPI Number of Observations Read 0 Number of Observations Used 0 Sum of Source DF Squares Mean Square F Value Pr > F Model 3 90.0000000 30.0000000 4.00 <.000 Error 6 0.0000000.500000 Corrected Total 9 0.0000000 R-Square Coeff Var Root MSE rate Mean 0.888 8.63390.8034 6.000000 Source DF Type III SS Mean Square F Value Pr > F name 3 90.00000000 30.00000000 4.00 <.000 Tukey's Studentized Range (HSD) Test for rate NOTE: This test controls the Type I experimentwise error rate, but it generally has a higher Type II error rate than REGWQ. Alpha 0.05 Error Degrees of Freedom 6 Error Mean Square.5 Critical Value of Studentized Range 4.04609 Minimum Significant Difference.03 Means with the same letter are not significantly different. Tukey Grouping Mean N name A 9.0000 5 NTS B 6.0000 5 MCH B 6.0000 5 EPI C 3.0000 5 OCT Difference name Between Simultaneous 95% Comparison Means Confidence Limits NTS - MCH 3.0000 0.9770 5.030 *** NTS - EPI 3.0000 0.9770 5.030 *** NTS - OCT 6.0000 3.9770 8.030 *** MCH - EPI 0.0000 -.030.030 MCH - OCT 3.0000 0.9770 5.030 *** EPI - OCT 3.0000 0.9770 5.030 *** Comparisons significant at the 0.05 level are indicated by ***. 5
Dependent Variable: rate Contrast DF Contrast SS Mean Square F Value Pr > F NTS vs (MCH-EPI) 30.00000000 30.00000000 4.00 0.000 OCT VS OTHERS 60.00000000 60.00000000 48.00 <.000 SHRP vs SOPH 0.00000000 0.00000000 0.00.0000 Appl vs Research 45.00000000 45.00000000 36.00 <.000 Interaction 45.00000000 45.00000000 36.00 <.000 Dependent Variable: rate The GLM Procedure Class Level Information Class Levels Values school SHRP SOPH orient APP RES Number of Observations Read 0 Number of Observations Used 0 Sum of Source DF Squares Mean Square F Value Pr > F Model 3 90.0000000 30.0000000 4.00 <.000 Error 6 0.0000000.500000 Corrected Total 9 0.0000000 R-Square Coeff Var Root MSE rate Mean 0.888 8.63390.8034 6.000000 Source DF Type III SS Mean Square F Value Pr > F school 0.00000000 0.00000000 0.00.0000 orient 45.00000000 45.00000000 36.00 <.000 school*orient 45.00000000 45.00000000 36.00 <.000 6
Oneway Descriptives Relationship of One-Way and Two-Way ANOVA 95% Confidence Interval for Mean N Mean Std. Deviation Std. Error Lower Bound Upper Bound Minimum Maximum OCT 5 3.0000.0000.447.7583 4.47.00 4.00 MCH 5 6.0000.58.707 4.0368 7.963 4.00 8.00 NUT 5 9.0000.707.36 8.0 9.8780 8.00 0.00 EPI 5 6.0000.0000.447 4.7583 7.47 5.00 7.00 Total 0 6.0000.406.5380 4.8739 7.6.00 0.00 ANOVA Sum of Squares df Mean Square F Sig. Between Groups 90.000 3 30.000 4.000.000 Within Groups 0.000 6.50 Total 0.000 9 Post Hoc Tests Multiple Comparisons Dependent Variable: Tukey HSD Mean 95% Confidence Interval (I) D (J) D Difference (I-J) Std. Error Sig. Lower Bound Upper Bound OCT MCH -3.0000.707.003-5.03 -.9769 NUT -6.0000.707.000-8.03-3.9769 EPI -3.0000.707.003-5.03 -.9769 MCH OCT 3.0000.707.003.9769 5.03 NUT -3.0000.707.003-5.03 -.9769 EPI.0000.707.000 -.03.03 NUT OCT 6.0000.707.000 3.9769 8.03 MCH 3.0000.707.003.9769 5.03 EPI 3.0000.707.003.9769 5.03 EPI OCT 3.0000.707.003.9769 5.03 MCH.0000.707.000 -.03.03 NUT -3.0000.707.003-5.03 -.9769 * The mean difference is significant at the.05 level. Homogeneous Subsets Tukey HSD Subset for alpha =.05 D N 3 OCT 5 3.0000 MCH 5 6.0000 EPI 5 6.0000 NUT 5 9.0000 Sig..000.000.000 Means for groups in homogeneous subsets are displayed. a Uses Harmonic Mean Sample Size = 5.000. 7
Contrast Coefficients D Contrast Maternal Chi Health Occupation Therapy Epidemi ology Nutrition Science - - - - 3 - - Contrast Value of Contrast Std. Error t df Sig. (-tailed) Assume equal -6.0000.0000-6.000 6.000 Variances.0000.0000.000 6.000 3-6.0000.0000-6.000 6.000 Does not -6.0000.0000-6.000.765.000 Assume Equal.0000.0000.000.765.000 Variances 3-6.0000.0000-6.000.765.000 Univariate Analysis of Variance Descriptive Statistics Dependent Variable: Std. SCHOOL Orient Mean Deviation N SRHP Applied 3.0000.00000 5 Exper 9.0000.707 5 Total 6.0000 3.6599 0 SOPH Applied 6.0000.584 5 Exper 6.0000.00000 5 Total 6.0000.47 0 Total Applied 4.5000.0384 0 Exper 7.5000.7795 0 Total 6.0000.4063 0 Tests of Between-Subjects Effects Dependent Variable: Source Type III Sum of Squares df Mean Square F Sig. Corrected Model 90.000(a) 3 30.000 4.000.000 Intercept 70.000 70.000 576.000.000 SCHOOL.000.000.000.000 Orient 45.000 45.000 36.000.000 SCHOOL*Orient 45.000 45.000 36.000.000 Error 0.000 6.50 Total 830.000 0 Corrected Total 0.000 9 a R Squared =.88 (Adjusted R Squared =.784) 8
0 9 Nutrition Science M = 9.00 SD =.00 Estimated Marginal Means 8 7 6 5 4 Matern Health M = 6.00 SD =.58 Epidemiology M = 6.00 SD =.00 SCHOOL Orientation 3 Education Applied Occup Ther M = 3.00 SD = 0.7 ORIENTATION Department Membership SHRP applied SOPH Experimental Experimental Psychology 0 Nutrition Science M = 9.00 SD =.00 9 Estimated Marginal Means 8 7 6 5 4 Occup Therapy M = 3.00 SD = 0.7 Epidemiology M = 6 SD =.00 Matern Health M = 6 SD =.58 ORIENTATION Department 3 Education SHRP Applied SHRP SCHOOL Orientation SOPH Psychology Experimental SOPH 9