What Does the F-Ratio Tell Us?

Transcription

1 Planned Comparisons

2 What Does the F-Ratio Tell Us? The F-ratio (called an omnibus or overall F) provides a test of whether or not there a treatment effects in an experiment A significant F-ratio suggests that there are differences between of means However, the F-ratio does not tell us which means differ

3 Assessing Mean Differences With only two treatment groups we know that the two groups differ if the F is statistically significant With three or more means we need to do further tests to see which pairs of means contribute to the significant F This can be done informally using confidence intervals, or more precisely using significance testing procedures

4 Statistical Comparison Procedures Comparison (or contrast) procedures are used to test more specific hypotheses about differences between means These comparison procedures fall into two categories: Planned comparisons Post hoc (unplanned) comparisons (e.g., Tukey s honestly significant difference, Scheffe )

5 Planned Comparisons (Contrasts) An experiment is often designed to provide information regarding several research questions If particular comparisons are of interest prior to the data analysis, they may be analyzed as planned comparisons Planned comparisons can be performed without reference to the omnibus F-ratio, i.e., You can just directly test the contrast and it is not necessary to test the overall F.

6 Reading Comprehension Study Remember the study investigating the effects of two types of drugs on reading comprehension Group 1: Placebo (control group) Group 2: Drug A Group 3: Drug B

7 Comparison Examples One research question may involve a comparison of the two drugs, Drug A versus Drug B (a pairwise comparison) Another question may involve the comparisons of the two drug groups to the control group (a complex comparison) There other questions like, Drug A versus control and Drug B versus control (pairwise comparisons)

8 Simple vs. Complex Comparisons Comparison 1 is a simple comparison Comparison 2 is a complex comparison H 0 : 2 = 3 H 1 : 2 3 H 0 : ( )/2 = 1 H 1 : ( )/2 1

9 Contrast Coefficients The contrasts are formed by applying a set of weights, called contrast coefficients, to the means = c c c c a a = c i i c 1, c 2, c 3,, c a are the weights, or contrast coefficients. is the comparison difference among means.

10 Developing Contrasts Contrasts are usually developed such that the weights sum to zero For previous examples, the coefficients are as follows Simple = (0) 1 + (1) 2 + (-1) 3 Complex = (2) 1 + (-1) 2 + (-1) 3

11 More on Contrast Coefficients A mean is excluded from a comparison whenever it is assigned a weight of zero There are many coefficient sets that could be used for a given comparison The only restriction is that the sum of the weights must be zero Integer coefficients are generally preferred

12 Any contrast/comparison includes: 1. Multiplication of means by a set of coefficients. 2. Algebraic summation of the weighted means.

13 Steps in Testing Contrasts 1. Calculate comparison difference among means, Ψ. 2. Calculate SS comparison for the comparison. 3. Calculate MS comparison for the comparison. 4. Calculate F=MS comparison /MS S/A for the comparison. 5. Compare F comparison to F critical to test the significance of the comparison.

14 Information needed to test Contrasts 1. Means in each group. 2. Sample Size in each group. 3. Coefficients to capture the comparison. 4. MS S/A from the overall analysis. This will be the error term for testing all contrasts in between subjects ANOVA designs.

15 Research Scenario A researcher is interested in comparing two drugs and a placebo condition on reading comprehension in a sample of hyperactive boys (K&W p. 22). Fourth grade boys are randomized to one of three groups; (1) Placebo Control, (2) Drug A, (3) Drug B. One hour after receiving either drug or placebo, each boy studies an essay for 10 minutes and then is given an objective reading comprehension test. IV is Drug Condition, and DV is reading comprehension score. 15

16 Reading Comprehension Example Data (K&W, p. 22) The group means are 15, 6, and 9 The grand (overall) mean is 10 Reading txgroup Control Group Drug A Drug B Total Report Mean N Std. Deviation

17 SPSS ANOVA Output The SPSS ANOVA output is as follows ANOVA reading Between Groups Within Groups Total Sum of Squares df Mean Square F Sig

18 Information needed to test Contrasts for Reading Example 1. Means in each group; 15, 6, Sample Size in each group; n=5. 3. Coefficients to capture the comparison; to be shown below. 4. MS S/A from the overall analysis. This will be the error term for testing all contrasts in between subjects ANOVA designs; MS S/A =15, from the overall ANOVA analysis,df S/A =12.

19 Comparison of Control and Drug A The group means were 15, 6, and 9 Calculations for the simple contrast comparing control and Drug A means are: ˆ ( c ) YA 1 ( c2) YA2 ( c3) YA 3 (1)(15) ( 1)(6) (0)(9) 1 9 SS comparison ( n)( ˆ) 2 c 2 (1) 2 5(9) ( 1) 2 2 (0)

20 Comparison of Control and Drug A MS comparison =SS comparison /df comparison =202.5/1 = F = MS comparison /MS S/A = 202.5/15 = 13.5 F critical = F(1,12) = 4.75 The F comparison > F critical so reject H O that there is no difference between the Control and Drug A conditions. The difference between control (Mean 15) and Drug A (Mean 6) was statistically significant.

21 Complex Comparison Example

22 Comparison of Drug A and Drug B to Control: Complex Comparison The group means were 15, 6, and 9 Calculations for the complex contrast are ˆ ( c ) YA 1 ( c2) YA2 ( c3) YA 3 ( 2)(15) (1)(6) (1)(9) 1 15 SS comparison ( n)( ˆ) c ( 15) 2 2 ( 2) (1) (1)

23 Comparison of Drug A and Drug B to control: Complex Comparison MS comparison =SS comparison /df comparison =187.5/1 = F = MS comparison /MS S/A = 187.5/15 = 12.5 F critical = F(1,12) = 4.75 The F comparison > F critical so reject H O that there is no difference between the Drug A and Drug B compared to the control condition.

24 Interpreting the Complex Comparison The complex comparison essentially averaged the means for the two Drug conditions The aggregated Drug groups mean was (9 + 6) / 2 = 7.5 The control group mean was 15 The significant F test tells us that Drug of any kind produced lower reading comprehension scores than the control

25 Advice on Planned Comparisons Planned contrasts should be used if they address the substantive hypotheses of interest As Keppel notes, the meaningfulness of a comparison is of critical importance in the analysis of an experiment and not its inclusion in a set of comparisons.

26 Planned Comparisons Using SPSS When doing planned comparisons using ONEWAY procedure, SPSS uses a t- statistic The GLM procedure uses an F However, for any two-group comparison, it is always the case that t 2 = F Note also that the two-tailed p-value on the SPSS printout is identical to the p- value associated with the F-ratio