Analysis of Covariance

Transcription

1 Analysis of Covariance (ANCOVA) Bruce A Craig Department of Statistics Purdue University STAT 514 Topic 10 1

2 When to Use ANCOVA In experiment, there is a nuisance factor x that is 1 Correlated with y 2 Unaffected by treatment Can measure x but can t control it (otherwise block) Factor x then called a covariate or concomitant variable ANCOVA adjusts y for effect of covariate x Combination of regression and analysis of variance Without adjustment, effects of x on y Will inflate σ 2 May alter trt mean comparisons (in extreme cases) STAT 514 Topic 10 2

3 Examples Pretest/Posttest score analysis: The change in score y may be associated with current GPA. Also the posttest score y may be associated with the pretest score x. Analysis of covariance provides a way to handicap students. Weight gain experiments in animals: When comparing different feeds, the weight gain y may be associated with the dominance x of the animal. While it may be hard to control for dominance, it is not too difficult to measure. Comparing competing drug products: The effect of the drug y after two hours may be associated with the initial mental and physical shape of the subject. Variables describing mental and physical shape x at baseline may be used as covariates. STAT 514 Topic 10 3

4 Model Description Consider single covariate in CRD Statistical model is y ij = µ+τ i +β(x ij x.. )+ǫ ij { i = 1,2,...,a j = 1,2,...n i Additional assumptions x ij not affected by treatment x and y are linearly related Constant slope across groups (can be relaxed) Note: Subtracting off x.. is not needed (conceptual) STAT 514 Topic 10 4

5 Estimation Conceptual Approach: Fit one-way model (y = trt) Fit one-way model (x = trt) Regress residuals (residuals1 = residuals2) Provides estimate of slope after adjusting for trt Model estimates are ˆµ = y.. ˆβ = (y ij y i. )(x ij x i. )/ (x ij x i. ) 2 ˆτ i = y i. y.. ˆβ(x i. x.. ) STAT 514 Topic 10 5

6 F Tests Test H 0 : τ 1 = τ 2 =... = τ a = 0 Compare treatment means after adjusting for differences among treatments due to differences in covariate levels Trt and covariate not orthogonal (order of fit matters) F 0 = SS(trt x)/a 1 SS E /(N a 1) Test: β = 0 Sum of Squares regression (SS x ): ˆβ 2 (x ij x i. ) 2 F 0 = SS x /1 SS E /(N a 1) STAT 514 Topic 10 6

7 Mean Estimates Adjusted treatment means Estimate ˆµ i = ˆµ+ ˆτ i = y i. ˆβ(x i. x.. ) Using the expected value of y when x is equal to the average covariate value Can really use any value of x, just make sure it is reasonable for all factor levels Variance: ˆσ 2( 1/n+(x i. x.. ) 2 / (x ij x i. ) 2) Pairwise differences Estimate: ˆτ i ˆτ i = y i. y i. ˆβ(x i. x i.) Variance: ˆσ 2( 2/n+(x i. x i.) 2 / (x ij x i. ) 2) STAT 514 Topic 10 7

8 Analysis of Covariance Table Looking at the breaking strength (in pounds) of a monofilament fiber produced by 3 different machines Known that strength depends on the fiber thickness Machines designed to keep thickness within specification limits but thickness will vary fiber to fiber Will consider diameter of the fiber as a covariate STAT 514 Topic 10 8

9 SAS Code data ancova; input machine str dia datalines; ; symbol1 i=rl v=circle; proc gplot; plot str*dia=machine; proc glm; class machine; model str = machine; lsmeans machine / adjust=tukey; proc glm; class machine; model str = machine dia; lsmeans machine / adjust=tukey; STAT 514 Topic 10 9

10 Boxplot STAT 514 Topic 10 10

11 SAS Output - No Covariate The GLM Procedure Dependent Variable: str Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE str Mean Source DF Type I SS Mean Square F Value Pr > F machine Source DF Type III SS Mean Square F Value Pr > F machine STAT 514 Topic 10 11

12 SAS Output - No Covariate The GLM Procedure Least Squares Means Adjustment for Multiple Comparisons: Tukey-Kramer LSMEAN machine str LSMEAN Number str LSMEAN LSMEAN machine Number A A B A B B STAT 514 Topic 10 12

13 Difference Plot STAT 514 Topic 10 13

14 Table of Means The MEANS Procedure machine= Variable N Mean Std Dev Minimum Maximum str dia machine= Variable N Mean Std Dev Minimum Maximum str dia machine= Variable N Mean Std Dev Minimum Maximum str dia STAT 514 Topic 10 14

15 Scatterplot STAT 514 Topic 10 15

16 SAS Output The GLM Procedure Dependent Variable: str Sum of Source DF Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square Coeff Var Root MSE str Mean Source DF Type I SS Mean Square F Value Pr > F machine <.0001 dia <.0001 Source DF Type III SS Mean Square F Value Pr > F machine dia <.0001 STAT 514 Topic 10 16

17 SAS Output The GLM Procedure Least Squares Means Adjustment for Multiple Comparisons: Tukey-Kramer LSMEAN machine str LSMEAN Number str LSMEAN LSMEAN machine Number A A A A A ****Must use LSMEANS to get adjusted means **** STAT 514 Topic 10 17

18 Difference Plot STAT 514 Topic 10 18

19 Summary Positive linear association between diameter and strength. Are slopes constant? Will investigate shortly. Model including covariate better explains the data. Percent of explained variation jumps from 40.5% to 91.9%. MSE drops from to Because Machine 3 had narrower fibers, its adjusted mean strength is shifted upwards. Likewise Machine 2 had wider fibers so mean shifted downward No significant difference among the machines relies on assumption that diameter not different across machines STAT 514 Topic 10 19

20 Nonconstant Slope in ANCOVA Statistical model for constant slope is { i = 1,2,...,a y ij = µ+τ i +β(x ij x..)+ǫ ij j = 1,2,...n i Can allow for different slope by including interaction { i = 1,2,...,a y ij = µ+τ i +(β +(βτ) i )(x ij x..)+ǫ ij j = 1,2,...n i In SAS, simply add interaction term into model Provides test for nonconstant slope STAT 514 Topic 10 20

21 SAS Code data ancova; input machine str dia datalines; ; proc glm; class machine; model str = machine dia; lsmeans machine / adjust=tukey lines; proc glm; class machine; model str = machine dia machine*dia; lsmeans machine / adjust=tukey lines; run; STAT 514 Topic 10 21

22 SAS Output The GLM Procedure Sum of Source DF Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square Coeff Var Root MSE str Mean Source DF Type I SS Mean Square F Value Pr > F machine dia <.0001 dia*machine Source DF Type III SS Mean Square F Value Pr > F machine dia <.0001 dia*machine STAT 514 Topic 10 22

23 Regression Approach to ANCOVA Consider ANCOVA model with a = 3 y j = β 0 +β 1 X 1j +β 2 X 2j +β 3 X 3j +ǫ j j = 1,2,...N X 1j = 1 if Trt 1 and X 1j = 1 if Trt 3 X 2j = 1 if Trt 2 and X 2j = 1 if Trt 3 X 3j = (x j x.. ) Trt 1: y j = β 0 +β 1 +β 3 (x j x.. )+ǫ j Trt 2: y j = β 0 +β 2 +β 3 (x j x.. )+ǫ j Trt 3: y j = β 0 β 1 β 2 +β 3 (x j x.. )+ǫ j Results in estimates ˆµ = ˆβ 0 ˆτ 1 = ˆβ 1 ˆτ 2 = ˆβ 2 ˆβ = ˆβ 3 STAT 514 Topic 10 23

24 Analysis of Covariance Can incorporate covariate into any model For two factor model y ijk = µ+τ i +β j +(τβ) ij +β(x ijk x... )+ǫ ijk Assume constant slope for each ij combination Can include interaction terms to vary slope Plot y vs x for each combination STAT 514 Topic 10 24

25 Background Reading ANCOVA Model: Montgomery General Regression Significance Test: Montgomery STAT 514 Topic 10 25