One- factor ANOVA. F Ra5o. If H 0 is true. F Distribu5on. If H 1 is true 5/25/12. One- way ANOVA: A supersized independent- samples t- test

Transcription

1 F Ra5o F = variability between groups variability within groups One- factor ANOVA If H 0 is true random error F = random error " µ F =1 If H 1 is true random error +(treatment effect)2 F = " µ F >1 random error F Distribu5on One- way ANOVA: A supersized independent- samples t- test T- test components: Measure of effect Measure of baseline variability = x 1 x 2 Std Err of the mean F- test components: Measure of effect Measure of baseline variability = Variance between groups Variance within group Variance components for F frac0on: variance! In this context, called mean square If H 1 is true random error +(treatment effect)2 F = " µ F >1 random error 1

2 One- way ANOVA: A supersized independent- samples t- test One- way ANOVA: A supersized independent- samples t- test T- test components: Measure of effect Measure of baseline variability = x 1 x 2 Std Err of the mean MS = SS df F = MS effect MS error F- test components: Measure of effect Measure of baseline variability = Variance between groups Variance within group Denominator: MS error - - Within- groups variance Variance components for F frac0on: variance! In this context, called mean square Pooled variance: SS 1 + SS 2 df 1 + df 2 MS error : SS 1 + SS 2 + SS 3. + SS n df 1 + df 2 + df 3. + df n Hours of sleep deprivation hours 24 hours 48 hours zero Means: SS: Df: MS error = SS 1 + SS 2 + SS 3 df 1 + df 2 + df 3 = One- way ANOVA: A supersized independent- samples t- test One- way ANOVA: A supersized independent- samples t- test T- test components: Measure of effect Measure of baseline variability = x 1 x 2 Std Err of the mean MS = SS df F = MS effect MS error F- test components: Measure of effect Measure of baseline variability = Variance between groups Variance within group Variance components for F frac0on: variance! In this context, called mean square Numerator: MS effect - - Between- groups variance (Are the group means more different than chance would predict?) Hours of sleep deprivation hours 24 hours 48 hours zero Means: Grand mean: 5 SS effect = nσ(x group - x grand ) 2 n: the number of samples in a group Basically, you re coun<ng each member of a group as though it is its group s mean SS effect = 3(2-5) 2 + 3(5-5) 2 + 3(8-5) 2 = 54 Df: 2 MS effect = SS effect df = 54/2 = 27 2

3 One- way ANOVA: A supersized independent- samples t- test One- way ANOVA: For pirates (Rrrrrr) MS = SS df F = MS effect MS error = = 7.36 > mydata=read.csv("week8sleepdepdata.csv",header=true) > mydata Subject Deprivation Aggression 1 s1 zero 0 2 s2 zero 4 3 s3 zero 2 4 s4 twentyfour 3 5 s5 twentyfour 6 6 s6 twentyfour 6 Hours of sleep deprivation hours 24 hours 48 hours zero Means: Grand mean: 5 SS effect = nσ(x group - x grand ) 2 n: the number of samples in a group Basically, you re coun<ng each member of a group as though it is its group s mean SS effect = 3(2-5) 2 + 3(5-5) 2 + 3(8-5) 2 = 54 7 s7 fortyeight 6 8 s8 fortyeight 8 9 s9 fortyeight 10 > myfirstanova=aov(aggression ~ Deprivation, data=mydata) > summary(myfirstanova) Df Sum Sq Mean Sq F value Pr(>F) Deprivation * Residuals Signif. codes: 0 *** ** 0.01 * Formula: Dependent ~ Independent F table Df: 2 MS effect = SS effect df = 54/2 = 27 Now what? Now what? summary(myfirstanova) Df Sum Sq Mean Sq F value Pr(>F) Deprivation * Residuals Hours of sleep deprivation hours 24 hours 48 hours zero summary(myfirstanova) Df Sum Sq Mean Sq F value Pr(>F) Deprivation * Residuals Hours of sleep deprivation hours 24 hours 48 hours zero Signif. codes: 0 *** ** 0.01 * Signif. codes: 0 *** ** 0.01 * What s the exact rela5onship between sleep depriva5on and aggression? Step 1: Make some graphs! Looks prehy linear. But are all differences significant? Temp5ng: Do 3 t- tests. 0 hours vs 24 hours 24 hours vs 48 hours 48 hours vs 0 hours Aggression hours 24 hours 48 hours It s (sort of) okay to do mul5ple t- tests, but it increases your chance of gekng a Type I error. Temp5ng: Do 3 t- tests. 0 hours vs 24 hours 24 hours vs 48 hours 48 hours vs 0 hours Aggression hours 24 hours 48 hours 3

4 One Factor Analysis of Variance (ANOVA) H 0 " µ 1 = µ 2 =... = µ k,# 1 = # 2 =...= # k where k is the number of levels and the populations are normally distributed. Mul5ple comparisons Problem: Too many extra tests run the risk that you ll have a chance of gekng at least one Type I error (a false posi5ve). Solu0on: Correct things so that you don t run such a high risk for Type I errors. But WHICH pair(s) of means are not equal?? Mul5ple comparisons Several types of correc5ons, All of which effec5vely lower alpha. Tradeoff: more risk of Type II errors so that you don t make too many Type I errors Things you can correct : Alpha itself The cri5cal value that leads you to reject H 0 Mul5ple comparisons Correc5ng alpha The Bonferroni correc5on: α = α familywise / n, where n is the # of comparisons Logic: if you do two tests and the null hypothesis is true, then you are doubling your chances of gekng a Type I error (false posi5ve) 4

5 Mul5ple comparisons Changing your cri5cal value Tukey HSD (Highly Significant Difference) Use it when comparing all possible pairs of condi5ons HSD = q*sqrt(ms within /n) Get q from a table with k (# of groups) and df within n = # per group HSD is difference between group means that exceeds chance Es5ma5ng Effect Size: Cohen s d for a One Factor ANOVA Other Mul5ple Comparison Tests d = X i " X j MS within Scheffe s Test Protected t-test (Least Significant Difference test) For individual comparisons between groups pg

6 Mul5ple comparisons No correc5on at all: protected t- test (Not everyone thinks this is kosher) What protects you? A priori predic5ons You had pre- planned to examine certain contrasts Usually these are strongly theore5cally- mo5vated Cog sci theories aren t always specific enough to give you such strong predic5ons. But if possible, make predic5ons and test them Ideally, your experiment is focused and precise, rather than being a haphazard fishing expedi5on* Effect sizes summarized For effects in an ANOVA: eta- squared (η 2 ) You can t really calculate Cohen s d η 2 varies between 0 and 1 (different from d!!) It s the amount of the total variance that s due to differences between groups: SS effect / SS total Can think of as % variance accounted for For a mul0ple comparison: d = (x 1 - x 2 )/ MS within Just like before, but use MS within instead of Std Dev. Progress Check 16.1 Progress Check Mean: Mean:

7 Progress Check 16.1 Progress Check Mean: Mean: Repor5ng Results Aggression scores for subjects deprived of sleep for zero hours (M=2, SD=2.0), those deprived for 24 hours (M=5, SD=1.7), and those deprived for 48 hours (M=8, SD=2.0) differ significantly (F(2,6)=7.36, MSE=3.67, p<.05, eta 2 =.71). According to Tukey s HSD test, however, only the difference of 6 between mean aggression scores for the zero and 48-hour groups is significant (HSD=4.77, p<.05, d=3.13). 7

8 One Factor ANOVA? Use Kruskal- Wallis H- test (Non- parametric analog of one factor ANOVA) Mean: Mean: Sec5on 20.5 One Factor ANOVA Summary Tests the null hypothesis that the means of mul5ple groups came from the same normal distribu5on (i.e., same mean and standard devia5on). Based on the F sta5s5c, which is the ra5o of between group variance to within group variance (large values of F lead to the rejec5on of the null hypothesis) Useful because it avoids increased rates of Type I error due to mul5ple comparisons Significant F s are followed up by mul5ple comparison tests (e.g., Tukey s HSD) Use nonparametric Kruskal- Wallis H test for non- normal data Coming up Repeated- measures ANOVA Analogous to a paired t- test Two- factor ANOVA Sleep depriva5on x caffeine intake You can have interac<ons: the effect of sleep depriva5on depends on how much caffeine you ve consumed (drawing ensues) Mixed designs: Some measures are repeated AX par5cipants, some are not 8

9 1- Factor Repeated- Measures ANOVA Table 17.1 A B C Mean: Research Design: Between- Subjects vs. Within- Subjects Manipula5ons EPO Placebo A B C Grand Mean = 5 Calcula5ons F = MS effect MS error MS error = SS error / df error df error = df within df subjects EPO Placebo Placebo EPO SS effect = nσ(x group - x grand ) 2 n: the number of samples in a group SS within = SS 1 + SS 2 + SS 3 SS subject = kσ(x subject - x grand ) 2 k: the number of groups SS error = SS within SS subjects 9

10 A B C Grand Mean = 5 Calcula5ons SS effect = 3(2-5) 2 + 3(5-5) 2 + 3(8-5) 2 = 54 F = MS effect MS error df error = df within df subjects df error = 6 2 = 4 MS error = SS error / df error MS error = 4 / 4 = 1 SS within = (0-2) 2 + (4-2) 2 + (2-2) 2 + (3-5) 2 + (6-5) 2 + (6-5) 2 + (6-8) 2 + (8-8) 2 + (10-8) 2 = 22 SS subject = 3(3-5) 2 + 3(6-5) 2 + 3(6-5) 2 = 18 SS error = = 4 MS effect = 54 / 2 = 27 F(2,4) = 27/1 = 27 A B C Grand Mean = 5 Calcula5ng df df groups: # groups 1 = 3 1 = 2 df within: (# subjects per group 1) * # groups (3-1) * 3 = 6 df subjects: # subjects 1 = 3 1 = 2 df error: df within df subjects = 6 2 = 4 Calcula5ng df a slightly- different example Calcula5ng df a really- different example A B C D df groups: # groups 1 = 3 1 = 2 df within: (# subjects per group 1) * # groups = (4-1)*3 = 9 df subjects: # subjects 1 = 4 1 = 3 df error: df within df subjects = 9 3 = 6 s s s s s s s s df groups: # groups 1 = 12 1 = 11 df within: (# subjects per group 1) * # groups = (8-1)*12 = 84 df subjects: # subjects 1 = 8 1 = 7 df error: df within df subjects = 84 7 = 77 10

11 Es5mated Effect Size (ANOVAs) (par5al) eta squared One Factor ANOVA SS between " 2 = = SS between SS between + SS within SS total Repeated Measures One Factor ANOVA SS between " 2 p = SS between + SS error pg. 521 Es5mated Effect Size (ANOVAs) (par5al) eta squared eta 2 p Effect Size 0.01 Small 0.09 Medium eta 2 = propor5on of explained variance Between Within Total 76 8 pg. 377 (same as for One Factor ANOVA) 0.25 Large " 2 = SS Between SS Total = =.71 0 " # 2 "1 11

12 One Factor Analysis of Variance (ANOVA) H 0 " µ j1 = µ j 2 =...= µ jk,# j1 = # j 2 =...= # jk where k is the number of levels, j is the jth subject, and the populations are normally distributed. But WHICH pair(s) of means are not equal?? Es5ma5ng Effect Size: Cohen s d for a One Factor ANOVA Repeated Measures One Factor ANOVA pg. 379 d = X i " X j MS within d = X i " X j MS error Repor5ng Results Mean aggression scores of 2, 5, and 8 were obtained when the same subjects were exposed to 0, 24, and 48 hours of sleep deprivation, respectively. There is evidence that, on average, aggression scores increase with hours of sleep deprivation (F(2,4)=27, MSE=1.0, p<.01, partial eta 2 =.93). According to Tukey s HSD test, all pairs of differences were significant (HSD=2.87, p<.05, 3<d<6). 12

13 Sphericity Assump5on r(x 0, x 24 ) = r(x 0, x 48 ) = r(x 24, x 48 ) " The degree of correla5on between all possible pairs of data groupings are equal. " For example, this assump5on might be violated if there was much more variance in the effect of the variable at one level (e.g., 48 hours of sleep depriva5on) than at another level (e.g., 0 hours of sleep depriva5on). How to deal with viola5ons of repeated measures ANOVA assump5ons: Correc5ons for viola5ng sphericity assump5on Greenhouse- Geisser epsilon Hunyh- Feldt epsilon Not Counterbalanced: Where you can use repeated- measures ANOVA Related Samples - Each observa5on in one condi5on is paired on a one- to- one basis basis, with a single observa5on in the other sample. EPO Placebo Counterbalanced: 50% EPO Placebo " Inves5gators want to see if strokes to the right hemisphere of the brain impair language produc5on. They find 10 stroke pa5ents and pair each pa5ent with a healthy volunteer of equal age, sex, and educa5onal background. Pa5ent #1 Volunteer #1 50% Placebo EPO Male, 72 years old, College Degree Male, 72 years old, College Degree 13

14 Repeated Measures One Factor ANOVA Summary Tests the null hypothesis that the means of mul5ple levels of a variable are the same when samples are paired across those mul5ple levels (e.g., they re measures from the same individuals). Like the regular one factor ANOVA, except the within subject Mean Squares is replaced with the error Mean Squares when compu5ng F, Tukey s HSD, and Cohen s d. Keep in mind the assump5on of sphericity and repeated measures issues like counterbalancing. Two- factor ANOVA! Do crowds and/or gender effect our willingness to respond to a poten5al threat? A social psychologist recruits 6 men and 6 women. Each volunteer is placed in a wai5ng room. Some5me ayer the volunteer enters the wai5ng room, smoke starts to emanate from a hea5ng vent. The psychologist measures the amount of 5me it takes the volunteer to do something about the smoke. 0, 2, or 4 other people (friends of the psychologist) are in the wai5ng room with the volunteer. Three Possible (non- exclusive) Differences 1. There s a difference between men and women (a main effect of gender). 2. The number of other people in the room makes a difference (a main effect of crowd size). 3. Crowd size affects men and women differently (there s an interac5on between gender and crowd size). 14

15 5/25/12 Two Factor ANOVA 15

16 Ask them what think the ANOVA results will be pg. 521 Es5mated Effect Size (ANOVAs) (par5al) eta squared Two Factor ANOVA Simple Effect of Crowd Size at Male SS column " 2 p (column) = SS column + SS within d = X i " X j MS within d = X i " X j MS error 16

17 No Simple Effect of Crowd Size at Female No Simple Effect of Gender at Crowd Size of Zero An alternative way to describe the interaction Simple Effect of Gender at Crowd Size of Two Simple Effect of Gender at Crowd Size of Four Note that you don t need to describe the interaction both ways 17

18 5/25/12 pg

19 Repor5ng Results pg. 408 An ANOVA with factors Crowd Size and Gender found that mean reaction times increase with crowd size (F(2,6)=6.75, MSE=5.33, p<.05, η 2 =.69) and are larger for males than females (F(1,6) =36.02, p<.01, η p2 =.86). However these findings must be qualified because of the significant interaction (F(2,6)=5.25, p<.05, η p2 =. 64). An analysis of simple effects for crowd size confirms that reaction times increase with crowd size for males (Fse(2,6)=11.63, p<.01) but not for females (Fse(2,6)=0.38, p>.05). Furthermore, when compared with the mean reaction time of 10 sec. for males with zero people, the mean reaction of 17 sec. for males with two people is significantly longer (HSD=7.07, p=.05, d=3.03), and the mean reaction time of 21 sec. for males with four people also is significantly longer (HSD=10.32, p<.01, d=4.76). To summarize, the mean reaction times of males, but not females, increases in the presence of crowds of two or four people. Two Factor ANOVA Summary Tests the null hypothesis that the means of mul5ple levels (e.g., male or female) of two variables (e.g., gender and crowd size) are independent samples from the same normal distribu5on. If sample size per each cell is equal, viola5on of equal variance assump5on is probably ok. Differs from previous hypothesis tests in that it takes into account a possible interac5on between the two variables. An interac5on occurs when the effect of one variable (e.g., crowd size) depends on the value of the other variable (e.g., gender). Two Factor ANOVA Summary Size of ANOVA effects can be quan5fied with par5al eta squared (e.g., the percentage of the variance that is accounted for by a factor or interac5on ayer removing the variance due to non- noise causes). Follow up significant effects and interac5ons with simple effect tests. To find out which pairs of cells have different means, use a mul5ple comparisons test (e.g., Tukey s HSD) and you can quan5fy the effect size with Cohen s d. 19