13: Additional ANOVA Topics

Transcription

1 13: Additional ANOVA Topics Post hoc comparisons Least squared difference The multiple comparisons problem Bonferroni ANOVA assumptions Assessing equal variance When assumptions are severely violated Kruskal-Wallis Summary

2 Illustrative Example p Pigment.sav - skin pigment levels in 4 families of same race (n = 5 in each group) Mean pigment levels Group 1: 38.6 Group 2: 46.0 Group 3: 46.4 Group 4: 52.4 EDA (right) ANOVA H 0 : 1 = 2 = 3 = 4 p.000 Reject null Which means differ?

3 Post Hoc Comparisons p Which means differ? Are all four groups different from each other? Is there one odd main out? After the fact (post hoc) comparisons Apriori contrasts pre-planned Posteriori not planned Different philosophies for each

4 Illustrative Example (pigment.sav) p How may post hoc comparisons? In testing k groups, there are k C 2 pairwise comparisons For illustrative example, k = 4 There are 4 C 2 = 4! / (2!)(4-2!) = 6 pairwise comparisons Test 1: H 0 : 1 = 2 Test 2: H 0 : 1 = 3 Test 3: H 0 : 1 = 4 Test 4: H 0 : 2 = 3 Test 5: H 0 : 2 = 4 Test 6: H 0 : 3 = 4

5 Least Square Difference (LSD) Method (p. 13.2) Illustrative data, test 1: H 0 : 1 = 2 Data xbar 1 = 38.6 xbar 2 = 46.0 Pooled estimate of variance s 2 W = MS w (from ANOVA table) = df w (also from ANOVA table) = N k = 20 4 = 16 Convert to p df w = 20 4 = 16 p =.0042

6 LSD Output from SPSS p Post hoc button during ANOVA procedure > LSD check box

7 The Problem of Multiplicity p A man and a woman who sits and deals out a deck of cards... (John Tukey, p. 13.4) Consider =.05 probability of correct retention =.95 In testing three correct null hypotheses at.05, Pr(three correct retentions) = Pr(at least one false rejection) = 1.86 =.14 This is the family-wise error rate Family-wise error rate increases with each additional test For m tests, family-wise error rate at.05 is 1.95 m e.g., 20 tests, family-wise error rate = =.65

8 Dealing w/ The Problem of Multiplicity p Depends on purpose of test For planned comparisons No adjustment necessary Proceed with LSD method For unplanned comparisons Make adjustments so family-wise error rate kept in check Many adjustment methods (see post hoc button in SPSS) We cover Bonferroni

9 Bonferroni s Method p Let m = number of comparisons Recall: m = k C 2 = k! / 2!(k 2)! p = p-value from LSD test p Bonf = p m Illustrative example, test 1 H 0 : 1 = 2 LSD derived p =.0042 There are four groups and six comparisons p Bonf = =.025

10 SPSS Bonferroni Output p. 13.5

11 ANOVA Assumptions p Validity assumptions good selection (random representation of populations; related to independence assumption below) information accurate (no information bias) comparability of group in factors other than that which identifies groups (no confounding) Distributional assumptions independence (random samples from k populations) normality (of sampling distribution of means central limit theorem) equal variance (homoscedasticity)

12 Comments Not in Reader notes Validity assumptions Difficult to assess (counterfactual) Are of utmost importance Trump distributional assumptions Distributional assumptions Can be assessed via data Often talked about Pale in importance compared to validity assumptions Moral dilemmas Do we pretend validity assumptions do not exist? Do we use limited time to fretting over distributional? Does expediency trump validity? Do we bother to defend distributional assumptions? Do we make the best of the situation?

13 Assessing Equal Variance p Compare standard deviations (> 2-fold difference in s?) Side-by-side boxplots (2-fold difference in hingespread?) F-ratio (2 groups) or Levene s test (k groups) (A) Equal means Equal variances (B) Unequal means Equal variances (C) Equal means Unequal variances (D) Unequal means Unequal variances

14 Small samples! Illustrative Data

15 Levene s test p SPSS One-way ANOVA Option (check homogeneity of variance ) Illustrative example (pigment.sav) H 0 : ² 1 = ² 2 = ² 3 = ² 4 F = 1.49 with 3 and 16 degrees of freedom (p =.25) No significant evidence of unequal variance But then again, no evidence of equal variance either

16 Options When Assumptions are Violated Severely p Descriptive analysis only Use ANOVA anyway Use more robust test (e.g., unequal variance t tests) Transform data (covered in Chap 15) Nonparametric testing

17 Kruskal-Wallis Nonparametric analogue to ANOVA H 0 : population medians are equal vs. H 1 : H 0 false Use SPSS > Analyze > Non-Parametric Tests > k Independent Samples Output chi-square stat, df, p value Interpret as other tests

18 Kruskal-Wallis Test pp Illustrative example: airsamples.sav Boxplot (right) F ratio test p = Conclude: variances unequal Kruskal-Wallis Does not require equal variance H 0 : population medians are equal SPSS derives ² K-W = 0.40 df = 1 p =.53 Conclusion: no significant difference in medians

19 t stat Mean dif