Factorial Analysis of Variance

Transcription

1 Factorial Analysis of Variance

2 Conceptual Example A repeated-measures t-test is more likely to lead to rejection of the null hypothesis if a) *Subjects show considerable variability in their change scores. b) Many subjects show no change c) Some subjects change a lot more than others. d) The degree of change is consistent across subjects. [Student incorrectly chose (a)] Sample Make-up Explanation: 1. Explain your mistake: I made a mistake because I was thinking of an independent-measures t-test and misunderstood considerable variability in the first answer to mean that the difference between groups was large. 2. Identify and explain the correct answer: The correct answer should have been (d). In the repeated-measures t-test, the test statistic is based on the difference scores, which are a measure of the degree of change for each subject. The numerator for the t statistic is a mean of difference scores and the denominator is an estimate of the standard error of the mean difference, which depends on the variability of the difference scores. If the amount of change is consistent across subjects, then the standard error term will be small, making the t-ratio larger. Large t-ratios are more likely to lead to rejection of the null hypothesis than small t-ratios.

3 Computational Example A researcher would like to examine how the chemical tryptophan, contained in foods such as turkey, can affect mental alertness. A sample of n = 9 college students is obtained and each student s performance on a familiar video game is measured before and after eating a traditional Thanksgiving dinner including roast turkey. The average score dropped by 10 points after the meal with a standard deviation of 5 points in the difference scores. Should you conclude that tryptophan affects mental alertness? Use a two-tailed test with α = Student s (Incorrect) Solution: n = 9, σ = 5, diff = -10? (assume μ=100 and M = 90) z 1.96 crit z obt M M M n 9 3 z obt / 3 Yes, tryptophan affects mental alertness because it is within the critical values 1.96

4 Sample Make-up Explanation: 1. Explain your mistake: I made three mistakes: (1) I forgot about one-sample t-tests and I figured that since we only had one sample that this had to be a z-test problem; (2) I used the wrong formula to compute the standard error; (3) I misinterpreted the result, given my obtained statistic, because I got confused about what was the null vs. the alternative hypothesis. I thought the null hypothesis was that tryptophan affects mental alertness 2. Identify and explain the correct answer: This required a related-samples t-test: M 10, s 5, n 9, t (8) MD tn D 1 s M D M s D n D D D D D crit t(8) The obtained t value falls outside of the critical values 1.96, so tryptophan does in fact affect mental alertness.

5 Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) In the context of ANOVA, an independent variable (or a quasiindependent variable) is called a factor, and research studies with multiple factors, in which every level of one factor is paired with every level of the other factors, are called factorial designs. Example: the Eysenck (1974) memory study, in which type-ofprocessing was one factor and age was another factor. Younger Older Counting Rhyming Adjective Imagery Intentional

6 Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) A design with m factors (with m>1) is called an m-way factorial design The Eysenck study described in the previous slide has two factors and is therefore a two-way factorial design We can design factorial ANOVAs with an arbitrary number of factors. For example, we could add gender as another factor in the Eysenck memory study However, for simplicity, we will deal only with two-way factorial designs in this course. We will also assume that each of our factorial samples (cells) contains the same number of scores n

7 Overview of the Factorial ANOVA Why might we want to use factorial designs? Factorial ANOVA (Two-Way) With a one-way ANOVA, we can examine the effect of different levels of a single factor: E.g., How does age affect word recall? Or, How does type of processing affect word recall? We need two different experiments to determine the effects of these two factors on memory if we use a one-way design. Moreover, using a factorial design allows us to detect interactions between factors

8 Overview of the Factorial ANOVA Example: Factorial ANOVA (Two-Way) We have developed a new drug for treating migraines, but suspect that it affects women differently than men The scores represent the number of weekly migraines reported following administration of the drug Low High Total Women M LW M HW M W Men M LM M HM M M Total M L M H

9 Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) Notice that the study involves two dosage conditions and two gender conditions, creating a two-by-two matrix with a total of 4 different treatment conditions. Each treatment condition is represented by a cell in the matrix. For an independent-measures research study (which is the only kind of factorial design that we will consider), a separate sample would be used for each of the four conditions.

10 Overview of the Factorial ANOVA Factorial ANOVA (Two-Way) As with one-way ANOVAs the goal for the two-factor ANOVA is to determine whether the mean differences that are observed for the sample data are significant differences and not simply the result of sampling error. For the example we are considering, the goal is to determine whether different dosages of a drug and differences in gender produce significant differences in the number of migraines reported.

11 Factorial ANOVA Main Effects Factorial ANOVA (Two-Way) To evaluate the sample mean differences, a two-factor ANOVA conducts three separate and independent hypothesis tests. The three tests evaluate: 1. The Main Effect for Factor A: The mean differences between the levels of factor A are obtained by computing the overall mean for each row in the matrix. In this example, the main effect of factor A would compare the overall mean number of migraines reported by women versus the overall mean number of migraines reported by men. 2. The Main Effect for Factor B: The mean differences between the levels of factor B are obtained by computing the overall mean for each column in the matrix. In this example, the ANOVA would compare the overall mean number of migraines reported for the low and high dosage conditions.

12 Example: Main Effects Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Main effect A Total M L =16 M H =10.5 Main effect B Main effect A: main effect of gender. Women report more migraines than men Main effect B: main effect of drug. Administering a higher of the drug leads to fewer reported migraines

13 Interactions The A x B Interaction: Often two factors will "interact" so that specific combinations of the two factors produce results (mean differences) that are not explained by the overall effects of either factor. For example, a particular drug may have different efficacies for men vs. women. Different s of the drug might produce very small changes in men, but dramatic, or even opposite, effects in women. This dependence on the effect of one factor (drug dosage) on another (sex or gender) is called an interaction.

14 Example: Interaction Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Total M L =16 M H =10.5 Main effect B In this case, the drug seems to be much more effective for women than for men. We would say that there is an interaction between the effect of gender and the effect of drug dosage

15 Example: No Interaction Low High Total Women M LW =20 M HW =14.5 M W =17.25 Men M LM =12 M HM =6.5 M M =9.25 Total M L =16 M H =10.5 Main effect B In this case, the main effect of the drug dosage is the same as in the previous case, but there is no longer a difference between the effect of drug dosage on women versus men. There is no interaction.

16 Interactions This is the primary advantage of combining two factors (independent variables) in a single research study: it allows you to examine how the two factors interact with each other. That is, the results will not only show the overall main effects of each factor, but also how unique combinations of the two variables may produce unique results.

17 More Examples

18 Simple Effects To interpret significant interactions, researchers often conduct a fourth type of hypothesis test for simple effects Simple effects (or simple main effects) involve testing the effect of one factor at a particular value of the second factor In our example, testing the effect of the drug for women only or for men only are examples of simple effects, as are testing the effect of gender in low only or high- only conditions Testing for simple effects essentially consists of running a separate one way ANOVA across all levels of one factor at a fixed level of the second factor For example, in the Eysenck memory study, we could run a one-way ANOVA to determine the effect of processing condition on word recall for young subjects only

19 Example: Interaction Low High Total Women M LW =20 M HW =10 M W =15 Men M LM =12 M HM =11 M M =11.5 Total M L =16 M H =10.5 Main effect B

20 Structure of the One-Way ANOVA (Independent- Measures) Total Variance Between Groups Variance Within Groups Variance

21 Structure of the Repeated-Measures ANOVA Total Variance Between Groups Variance Within Groups Variance Between Subjects Variance Error (residual) Variance

22 Structure of the Two-Way (Factorial) ANOVA Total Variance Between Groups Variance Within Groups Variance Factor A Variance Factor B Variance Interaction Variance

23 The Two-Way ANOVA Each of the three hypothesis tests in a two-factor ANOVA will have its own F-ratio and each F-ratio has the same basic structure: F MS MS between within Each MS value equals SS/df, and the individual SS and df values are computed in a two-stage analysis. The first stage of the analysis is identical to the single-factor (one-way) ANOVA and separates the total variability (SS and df) into two basic components: between treatments and within treatments.

24 The Two-Way ANOVA The second stage of the analysis separates the betweentreatments variability into the three components that will form the numerators for the three F-ratios: 1. Variance due to factor A 2. Variance due to factor B 3. Variance due to the interaction. Each of the three variances (MS) measures the differences for a specific set of sample means. The main effect for factor A, for example, will measure the mean differences between rows of the data matrix.

25 The Two-Way ANOVA: Possible Outcomes 1. All 3 hypothesis tests are not significant 2. Only main effect of Factor A is significant 3. Only main effect of Factor B is significant 4. Both main effects (for A and B) are significant 5. Only the interaction (A B) is significant 6. Main effect of Factor A and A B interaction are significant 7. Main effect of Factor B and A B interaction are significant 8. All 3 hypothesis tests are significant

26 Factorial ANOVA (Two-Way)

27 The Two-Way ANOVA: Steps 1. State Hypotheses Factorial ANOVA (Two-Way) whatever 2. Compute F-ratio statistic: F dfwhatever, dferror for each main effect and their interaction MSerror For data in which I give you cell means and SS s, you will have to compute: marginal means SS total, SS between, SS within, SS factor A, SS factor B, & SS A B df total, df between, df within, df factor A, df factor B, & df A B MS 3. Use F-ratio distribution table to find critical F-value(s) representing rejection region(s) 4. For each F-test, make a decision: does the F-statistic for your test fall into the rejection region?

28 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender Within (error) Total 1. Compute degrees of freedom df df df df df total between N 1 19 # cells 1 3 within total between gender df df # levels 1 1 # gender levels df df df df 1 gender between gender

29 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 Gender 1 Dose 1 Dose x gender 1 Within (error) 16 Total Compute SS within (SS error ) SSwithin SS

30 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups 3 Gender 1 Dose 1 Dose x gender 1 Within (error) Total Compute SS between (SS cells ) SS n M M between cell T

31 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender 1 Dose 1 Dose x gender 1 Within (error) Total Compute SS gender (SS Factor A ) SS n M M gender row gender T

32 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose 1 Dose x gender 1 Within (error) Total Compute SS (SS Factor B ) SS n M M col T

33 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender 1 Within (error) Total Compute SS gender (SS A B ) SSd ose SS SS SS gender between gender

34 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender Within (error) Total Compute the MS values needed to compute the 3 required F ratios: MS gender SSgender df 1 gender MS gender SS gender df 1 gender MS error SSerror df 16 error

35 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender Within (error) Total Compute F ratios for each of the two main effects (gender and ) and their interaction: F whatever df MS whatever whatever, dfe rror MSerror F F F gender 1,16 1,16 MSgender MS error MS MS error MS gender gender 1, MS error

36 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender Dose Dose x gender Within (error) Total Finally, look up F crit for each of your obtained F values In this case, we happen to be lucky that they all have the same degrees of freedom (1,16), so we only have to look up one F crit

37 F table for α=0.05 reject H 0 df error df numerator

38 F table for α=0.05 reject H 0 df error df between

39 Low Women M=20 SS=20 Men M=12 SS=32 () M T High M=10 SS=18 M=11 SS=24 M L =16 M H = , N 20 n 5, n 10, n 10 row col (gender) M W =15 M M =11.5 Factorial ANOVA (Two-Way) Set up a summary ANOVA table: Source df SS MS F p Between groups Gender * <0.05 Dose * <0.05 Dose x gender * <0.05 Within (error) Total Both main effects are significant, as is their interaction. This suggests that: 1. The number of reported migraines differs between men and women Women report more migraines than men 2. The number of reported migraines is affected by the drug Fewer migraines are reported in the high condition 3. The effect of the drug differs across genders Women are more sensitive to the drug than are men

40 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study Age x study Within (error) Total Compute degrees of freedom df df df df df total between age study N 1 99 # cells 1 9 df df within total between 90 # age levels 1 1 # study levels 1 4 df df df df agestudy between age study 4

41 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups 9 Age 1 Study 4 Age x study 4 Within (error) 90 Total Compute SS between (SS cells ) SS n M M between cell T

42 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age 1 Study 4 Age x study 4 Within (error) 90 Total Compute SS within (SS error ) SS SS SS within total between

43 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age 1 Study 4 Age x study 4 Within (error) Total Compute SS age (SS Factor A ) SS n M M age row age T

44 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study 4 Age x study 4 Within (error) Total SS n M M study col study T 5. Compute SS study (SS Factor B )

45 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study Age x study 4 Within (error) Total Compute SS age study (SS A B ) SS SS SS SS age study between age study

46 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study Age x study Within (error) Total Compute the MS values needed to compute the 3 required F ratios: MS age SSage 240 df 1 age 240 MS agestudy SSagestudy df 4 agestudy MS study SSstudy df 4 study MS error SSerror df 90 error

47 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C =6.75 M R =7.25 M A =12.9 M IM =15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study Age x study Within (error) Total Compute F ratios for each of the two main effects (age and study condition) and their interaction: F whatever df MS whatever whatever, dfe rror MSerror F F F age study MS age 1, MS error MSagestudy , MS 8.03 age study 4,90 MSstudy MS 8.03 error error

48 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C = 6.75 M R = 7.25 M A = 12.9 M IM = 15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age Study Age x study Within (error) Total Finally, look up F crit for each of your obtained F values In this case, we have two distinct values to look up: F crit (1,90) and F crit (4,90)

49 F table for α=0.05 reject H 0 df error df numerator

50 F table for α=0.05 reject H 0 df error df numerator

51 M T Counting Rhyming Adjective Imagery Intentional (age) Old M=7.0 M=6.9 M=11.0 M=13.4 M=12.0 M O =10.06 Young M=6.5 M=7.6 M=14.8 M=17.6 M=19.3 M Y =13.16 (study) M C = 6.75 M R = 7.25 M A = 12.9 M IM = 15.5 M IN = , SS , N 100 total n 10, n 50, n 20 row col Summary ANOVA table: Source df SS MS F p Between groups Age * <0.05 Study * <0.05 Age x study * <0.05 Within (error) Total Both main effects are significant, as is their interaction. This suggests that: 1. The number of remembered words differs between old and young subjects Younger subjects remember more words 2. The number of remembered words differs across study conditions More words are remembered in the deeper processing conditions, though we would need post-hoc tests to see which of these differences are significant 3. The effect of the study condition differs as a function of age Younger subjects get more of an advantage from deep processing

52 Effect Size for the Two-Way ANOVA Factorial ANOVA (Two-Way) Effect sizes for two-way ANOVAs are usually indicated using the R 2 -family measure eta-squared (η 2 ) R 2 -family measures indicate the effect size in terms of proportion of variance accounted for by the treatment effect(s) 2 variability explained by treatment effect R total variability We compute this measure for each of our effects. E.g.: SS 2 age 240 age 0. 09, SS total SS 2 study study 0.57, SS total SSagestudy SS age study total