ANOVA INTERPRETING. It might be tempting to just look at the data and wing it

Transcription

1 Introdction to Statistics in Psychology PSY 2 Professor Greg Francis Lectre 33 ANalysis Of VAriance Something erss which thing? ANOVA Test statistic: F = MS B MS W Estimated ariability from noise and mean di erences F = Estimated ariability from noise if H is tre, and F is s ciently larger than, then a rare eent has happened. Since rare eents are rare, when F>>wespposethatH is not tre Rareness is established by the p ale, which is gotten from an F distribtion with K df in the nmerator and N Kdfin the denominator HYPOTHESES The nll is an omnibs hypothesis. It spposes no di erence between any poplation means H : µ i = µ j 8 i, j the alternatie is the complement H : µ i 6= µ j form some i, j Note, there is no one-tailed ersion of ANOVA 2 3 Ihappentohaedatafrom8di erent classes that all completed an experiment where sbjects responded as qickly as possible whether a set of letters formed a word or not It might be tempting to jst look at the data and wing it For example, looking at the means, it seems that class Psy2Spring5 has a mch larger mean than any other class More than one mean might di er from other means Een if the mean for Psy2Spring5 is di erent from the others, might other means also be di erent? The conclsion is that at least one poplation mean seems to be di erent from the other poplation means. Something is di erent The ANOVA does not tell yo which mean is di erent from the others; or if more than one mean is di erent from others. 4 Bt that class also has a small nmber of stdents (n =4),andalargestandarddeiation(s =36.9), so we wold expect qite a bit of ariability in the mean ale. Maybe this big mean is not so rare, gien the ariability de to random sampling 5 We wold really like to know which means seem to be di erent from which other means 6

2 TYPE I ERROR Mltiple testing problem To motiate ANOVA, we mentioned that it is problematic to jst test all pairwise comparisons of grop means. With 8 means, there wold be 28 tests. So the Type I error rate wold be arond ( ) 2 =( )=.76 Instead of jst testing all possible comparisons, sppose we first reqire that the ANOVA prodces a significant reslt. If H is tre, the ANOVA shold only conclde that some di erence exists with a probability of.5 (or whateer yo choose as ) TYPE I ERROR Ths, we can control the oerall Type I error rate by insisting that or data prodce a significant ANOVA before we start testing di erent means We want to check that something is di erent before we check which means are di erent! If we test Psy2Spring5 against each of the other seen means, the Type I error rate can be no bigger than what it was for the ANOVA In fact, it has to be a bit smaller than the sed for the ANOVA becase we hae to satisfy two criteria If H is tre, 95% of the time, we neer compare the means to each other t tests One approach is to jst rn t tests (Welch s test) to compare di erent means Francis2F t tests One approach is to jst rn t tests (Welch s test) to compare di erent means PSY28HKIED There is a better (and more general approach) ANOVA assmes/reqires homogeneity of ariance 2 i = 2 j 8 i, j For the t-test we pooled ariances/standard deiations to get a better estimate of With more poplations, we can pool all of the sample ariances and thereby get a still better estimate Ths, een when we compare Psy2Spring5 against Francis2F5, wecansethedata from the other samples to get a better estimate of POOLED ESTIMATE Fortnately, the pooled estimate of ariance is easy to find We compted it in the ANOVA, it is MS W Ths, the standard error that we se for the t test is s X X 2 tms W + C A n n 2 2

3 s X Francis2F5 t + n n 2 A = t ( ) Compare to the traditional t test, where s X = So with the pooled ariance, we get t = X s X = Compare to t =4.966 for the traditional t test The degrees of freedom is based on how many scores contribte to the ariance calclation, so we get df = N K =45 8=47 compare to df = n + n 2 2=4+8 2=93 For traditional t test (smaller with Welch s test) 4 + A = s X PSY28HKIED t + n n 2 A = Compare to the traditional t test, where s X =7.446 So with the pooled ariance, we get t = X s X t ( ) 4 + A = = The degrees of freedom is based on how many scores contribte to the ariance calclation, so we get df = N K =45 8=47 so p =.66 Compare to t =.853 for the traditional t test compare to df = n + n 2 2=4+5 2=7,and p =.8 BETTER IS BETTER With a contrast, we get a better estimate of s X X 2,whichsometimes means we can reject H.Notalways, thogh. It is possible for a standard t test to reject H,btthecorresponding contrast test does not reject H (becase the sample s 2 is smaller than MS W ) We do not hae any cases like that in or crrent data set Generally speaking, sing MS W is better than sing the pooled s 2 becase more data contribtes to the estimate OTHER Comparing two means is actally a special case of sing contrasts We can also compare arios combinations of means For example, we might wonder if the mean for classes taght by Dr. Francis di ers from the mean for classes not taght by Dr. Francis OTHER We set p contrast weights, c i,for each class mean Or nll hypothesis will be H : (c iµ i )= i= and we reqire that the contrast weights sm to : i= c i = Or alternatie hypothesis is H a : (c iµ i ) 6= i= (one-tailed tests are also possible) TEST STATISTIC We compte the weighted sm of means L = (c ix i ) i= which has a standard error of: c 2 s L = i tms W i= n i and or test statistic is t = L s L which follows a t distribtion with df = N K where N is the sm of sample sizes across all grops and K is the nmber of grops 6 7 8

4 To compare the mean of the for classes taght by Dr. Francis to the mean of other for classes, we se contrast weights of ± Other sets of contrast weights compare other combinations. For example, to contrast the mean of the non-us based class, PSY28HKIED, againstall the other classes, we cold se: It can be appropriate to set some weights eqal to. For example, if yo want to compare the mean from two classes in 25 against the mean from three classes in 26, yo can set weights as: 9 Yo do not hae to se integer ales for the c i terms, bt it helps to aoid ronding isses. 2 2 SPECIAL CASE Comparing two means is jst a special case where the contrast weights for those means are set to ± andthe other weights are set to : This gies the same reslt as we compted preiosly 22 MULTIPLE TESTING There are an enormos nmber of di erent contrasts that yo cold create If yo reqire a significant ANOVA before rnning any contrasts, then yo can control the Type I error rate to be no higher than Howeer, we hae a new kind of conditional Type I error Gien that the ANOVA indicates there is some di erence in means, what means (or combinations of means) di er? For some contrasts the H is tre, bt, jst de to random sampling, they indicate that there is a di erence 23 MULTIPLE TESTING Ths, we hae a new mltiple testing problem for identifying the di erences; een thogh we only get to that sitation with probability if the ANOVA omnibs H is tre Worse, it cold be that µ 7 6= µ 8,so yo reject the ANOVA H bt then yo rn contrasts for other means where µ i = µ j Generally, it is not a good idea to try all possible contrasts. Contrasts (and hypothesis testing in general) make the most sense when yo hae some specific plans to compare combinations of means 24

5 CONCLUSIONS interpreting an ANOVA identifying di erences contrast tests NEXT TIME power for ANOVA power for contrasts Keep it simple! 25 26