8/04/2011. last lecture: correlation and regression next lecture: standard MR & hierarchical MR (MR = multiple regression)

Transcription

1 psyc3010 lecture 7 analysis of covariance (ANCOVA) last lecture: correlation and regression next lecture: standard MR & hierarchical MR (MR = multiple regression) 1 announcements quiz 2 correlation and regression to be completed online May 15 and 16 assesses material taught in Lectures 6, 7, 8, and 9 Lecture 10 will be a review of regression topics plus fun material on mediation and indirect effects practice questions and quiz will be posted on Blackboard assignment 2 Multiple Regression due on May 23 (Week 12) will learn all skills and concepts by Week 9 all files on Blackboard next week (Week 8) 2 1

2 Pros Feedback Repetition, revision, elaboration helpful Clear, Made it easy Funny, Fun Interactive, Engaging,? breaks good Slides great, prepared Good pace Thorough, detailed Enthusiasm Detailed summaries helpful ; Flow charts great ; Great examples ; Lectopia great Lecture slides up early ; Caring Lecture ended early Improve Go slower More review, repetition needed Make week 1 lecture slides available early Trouble printing Explanations unclear More entertainment Jargon frustrating, boring More practice questions Changing terms/notation frustrating Assumed knowledge More real life examples Key concepts slide at start End early Spread slides out Use harder questions Simple simple comparisons / 3 effects exasperating last lectures this lecture 2 lectures ago: importance of maximising power in research (maximising likelihood of correctly detecting effects that exist in the population, and rejecting H 0 ) how blocking designs can increase power last lecture: correlation (association between two variables) and regression (association + prediction) this lecture: an additional strategy to maximise power: Analysis of Covariance (ANCOVA) 4 2

3 topics covered in this lecture review of blocking designs and introduction to analysis of covariance (ANCOVA) 1 st use of ANCOVA: reduce error variance 2 nd use of ANCOVA: adjust treatment means structural model & assumptions of ANCOVA why ANCOVA is controversial 5 review of blocking designs and introduction to ANCOVA 6 3

4 how blocking designs can help problem = there is a lot of unexplained variance ( error variance ) in your 1-way experiment, and so the effect of your focal IV is not statistically significant solution = add a 2 nd IV that you know will explain additional variance (based on previous research) adding this control or concomitant variable changes the design of your study (now a 2-way factorial) 2 nd IV explains additional systematic variance, and so now there is less unexplained / residual / error variance increased chance that your focal IV has a significant effect: MS F = MS treat error ideally the same as in your 1-way design ideally smaller than in your 1-way design 7 ancova analysis of covariance has the same goal as blocking but works differently: blocking works at the level of design the reduction in the size of the error term is a consequence of including a factor that explains a good proportion of variance in the DV With ancova the error term is adjusted statistically 8 4

5 remind me what is covariance? variance is the tendency for scores to vary around some mean value co-variance is the tendency for two scores to vary together If a participant s score on one variable deviates from the mean, the score on the other (covarying) variable also deviates positive covariance = both deviate in the same direction [ Review pp (Howell 6 th ed.) if nec] a covariate is like the control variable used for blocking, with a couple of differences: 2 ( X ij X ) / 1 ( X X )( Z Z ) 1 the covariate is a continuous variable and treated as such (i.e., participants are not matched at discrete levels) in ancova, the covariate is used to remove error from both the error term and treatment effect 9 Analysis of Covariance ANCOVA Originally a technique for analysing experiments and removing nuisance variation Attempt to reduce error term by measuring another variable and estimating its parameters if the variable affects the DV and it is not part of the statistical model for the ANCOVA, the variable is in the unmeasured error 10 5

6 H o H 1 power β μ o μ 1 α 11 Use ANCOVA to reduce error (just like with blocking) H o H 1 power β μ o μ 1 α 12 6

7 ANCOVA All forms of ANOVA can be performed with a covariate (or several) A covariate is another IV/predictor in the model but continuous (ordered scores, not discrete groups) Can reduce error term if it is related to the DV if unrelated you lose DF (lose power) without compensatory reduction in error (i.e., bad trade- off) 13 Uses of ANCOVA 1. To control unwanted variation that would otherwise inflate the error with which we test our models (classical usage) 2. To control for group differences, esp. in the analysis of clinical trials or other pre/post designs (controversial, see Howell 16.5) 14 7

8 Oneway ANCOVA structural model X (the DV) for participant I in Jth group Grand mean 1 st IV factor A group j X ij = μ + α j + βζ βζ ij ij + ε ij 2 nd IV score on variable Z multiplied by a fixed weight (beta) Error Covariate is just another source of variance Use the term βζ ij because of continuous nature; Score on DV goes up or down depending on score on Z Implicitly, we have specified no interaction between covariate and the IV the presence of such an interaction is a violation of ANCOVA assumptions stats software normally provides output to check as a default Howell includes interaction in the model 15 X ij the structural models ij = μ + τ j + e ij One-way ANOVA model β s are not the same!! X ijk ijk = μ + α j + β k + αβ jk + e ijk Factorial ANOVA model X ij = μ + α j + βz ij + e ijk One-way ANCOVA model 16 8

9 how ANCOVA reduces error variance covariate = another IV or predictor in the model but continuous (ordered scores, not discrete groups) if the covariate is associated with the DV: this relationship accounts for some systematic variance unexplained by the focal IV accounting for this systematic variance reduces the amount of unexplained variance in the design A smaller error term because we ve partioned out the variance due to the covariate means an increase in statistical power in testing the effect of the focal IV (just as when using blocking designs) 17 an example research study comparison of driving performance with three different car sizes: are smaller cars easier to handle? easily addressed using 1-way ANOVA: DV: rating after 10 laps on set course 3 performance cars are compared: BMW Z3 (small) Subaru WRX (medium) Ford GTP (large) different groups of drivers used for each condition 18 9

10 results show lots of overlapping variance this indicates a large error term this results in low power to reduce the variance, we could identify a a covariate which past research tells us is related to the DV in this case: driving experience μ o μ 1 μ

11 21 mean scores for each car 22 11

12 mean scores for each car juxtaposed with actual scores on and 23 SS 2 = (X X ) error ij j in a regular ANOVA, SS error = the sum of the squared deviations of scores around their group means 24 12

13 a lot of the variance is due to the relationship between experience and we can remove this from the error term first 25 slope of regression line describes avg. covariance between the two variables reduce error by computing pooled error term based on deviations around each group s regression slope 26 13

14 the DV scores are not clustered around the mean based on random chance alone they vary systematically (based on relationship with covariate) Unadjusted _ SS ( X X error = ij j ) 2 unadjusted error includes the chance variance + covariance 27 Estimate covariate s effects with a regression line Calculate error as deviation from the Yhat instead of the mean Y If the covariate is related to the DV, the regression line is a better anchor around which scores cluster (smaller error) adjusted error 28 14

15 As an adolescent I aspired to lasting fame, I craved factual certainty, and I thirsted for a meaningful vision of human life - so I became a scientist. This is like becoming an archbishop so you can meet girls. -- Matt Cartmill, anthropology professor and author (1943- ) 29 how does that do anything different from blocking? at this stage it does not the effects of the covariate are subtracted from the error term, making it smaller The covariate is a more powerful way to do this if the control variable is continuous, but it s conceptually the same the next thing ancova does is quite different treatment means are adjusted to account for differences on the covariate random assignment to IV conditions normally prevent differences in covariate means (confounds should be designed out) But in case covariate does differ across groups, ANCOVA effectively partials out the effects of the covariate from the focal IV as well as the error term 30 15

16 ANCOVA adjusts treatment means (DV) why we would do this if focal IV affects DV scores there is a significant difference among treatment means between the levels of the IV if covariate also differs between levels of focal IV which variable explains difference in DV treatment means? confound! we care about the effect of the focal IV, not the effect of the covariate ANCOVA teases apart the effects of the covariate and the IV by asking the question: would the focal IV have an effect on the DV if all participants were equivalent on the covariate? 31 How ANCOVA adjusts treatment means on the DV problem: participants in each level of focal IV also differ in their scores on the covariate variable solution: Calculate the overall covariate mean. We assume this is the population mean. In an unconfounded population, all groups of the focal IV are assumed have this covariate mean. For your sample, if a group s mean is different on the covariate than the overall covariate mean, that is a confound. Adjust the group s mean on the DV to be what it would be if the group s covariate mean were the overall covariate mean, by using the regression line 32 16

17 33 in this case, there are no differences between the groups on the covariate, as you would expect, given random assignment 34 17

18 mean scores meet on regression line in this case, the 3 conditions have different covariate means confound: what s causing the difference in DV scores? 35 shift DV scores to new point on regression line 1. calculate overall covariate mean 2. adjust DV scores according to regression line 3. test group main effect using adjusted means 36 18

19 here, observe a larger effect for car if adjust means so each group has average driving experience 37 logic of ANCOVA adjusted treatment means assume that covariate means are the same at each level of the focal IV thus, any differences in the adjusted treatment means can be attributed to the focal IV only would groups differ on the DV if they were equivalent on the covariate? refines error term by subtracting variation that is predictable from covariate larger adjustment when covariate-dv relationship is strong refines treatment effect to adjust for any systematic group differences on covariate that existed before experimental treatment 38 19

20 comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA DV Dependent = Variable: ATTRACT Source Car Error Total Tests of Between-Subjects Effects Type III Sum of Squares df Mean Square F Sig way ANOVA effect is not significant 39 comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA Tests of Between-Subjects Effects DV Dependent = Variable: (block ATTRACT on experience e.g., no training, some training, professional) Source Car Experience Car x Experience Error Total Type III Sum of Squares df Mean Square F Sig blocking, using factorial ANOVA reduction of error term from to effect is marginally significant 40 20

21 comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA Tests of Between-Subjects Effects DV Dependent = Variable: (experience ATTRACT as a continuous scale, included as a covariate) Source Car Regression Error Total Type III Sum of Squares df Mean Square F Sig increase in treatment effect from to way ANCOVA reduction of error term from to effect is now significant! 41 structural model and assumptions of ANCOVA 42 21

22 blocking ANCOVA vs blocking conceptually simpler requires fewer assumptions ANCOVA easier to administer can use continuous covariate removes effect from error term and DV useful in two situations: covariate related to IV and DV (confound) covariate related to DV only does require specific assumptions 43 assumptions of ANCOVA all the regular ANOVA assumptions: homogeneous variance normal distribution independence of errors plus: see Lecture 2 and 2 nd year stats notes relationship between covariate and DV is linear relationship between covariate and DV is linear within each group relationship between DV and covariate is equal across treatment groups - homogeneity of regression slopes 44 22

23 re: assumption 1 DV DV covariate Linear relationships covariate Non-linear relationships Non-linear relationships generally cannot be detected with ANCOVA degrades power. 45 re: assumption 3 DV DV covariate homogeneity of regression slopes covariate heterogeneity of regression slopes homogeneity of regression slopes is important because adjustments to treatment means are based upon an average within-cell regression coefficient 46 23

24 47 adjusting treatment effects: the fine print the process is still considered questionable some people object to the idea of comparing adjusted treatment means at all real observed means are not compared comparison means are estimated using regression slope, which may not be reliable if treatment group does affect the covariate as well as the DV, what does the adjusted DV mean really tell you? some people don t mind adjusted means when the adjustment makes the treatment effect larger but it doesn t always make the treatment effect larger, so it doesn t always work in your favor! 48 24

25 adjustment has no effect on mean differences example A 49 example B 50 25

26 example B adjustment increases mean differences 51 example C 52 26

27 adjustment decreases mean differences example C 53 A final comparison The strength of ANCOVA is the ability to handle continuous data most psychological variables are continuously distributed, splitting people into groups is inefficient (lose info) and error prone (mis-categorisation at group boundaries magnifies error) if your data is continuous, it is best to analyse it using a method which can deal with such data (ANCOVA is more powerful than Blocking) If adjusted and observed means are very different, concerns re interpretation arise 54 27

28 readings analysis of covariance (this lecture) Field (3 rd ed): Chapter 11 Field (2 nd ed): Chapter 9 Howell (all eds): Chapter 16 standard & hierarchical multiple regression (next lecture) Field (3 rd ed): Chapter 7 Field (2 nd ed): Chapter 5 Howell (all eds): Chapter