Example: Poisondata. 22s:152 Applied Linear Regression. Chapter 8: ANOVA

Transcription

1 s:5 Applied Linear Regression Chapter 8: ANOVA Two-way ANOVA Used to compare populations means when the populations are classified by two factors (or categorical variables) For example sex and occupation for predicting salary advantages compared to doing two separate one-way ANOVAs more efficient to study two factors simultaneously rather than separately can reduce residual variation in a model by including a second factor believed to affect the response (more predictors) can study interaction between factors Example: Poisondata The data give the survival times (in 0 hour units) in axfactorialexperiment,thefactorsbeing (a) three poisons and (b) four treatments. Each combination of the two factors is used for four animals, the allocation to animals being completely randomized. The variables in the dataset are: Time : Survival time (in 0 hour units) Poison : Factor with levels,, : Factor with levels,,, Cells in a two-way ANOVA: Notation first categorical variable: factor A with a levels second categorical variable: factor B with b levels n ij is the sample size for level i of A and level j of B N is total observations Cell means model: Y ijk = µ ij + ijk with ijk iid N(0, σ ) µ ij is population mean for level i of A and level j of B (there are ab distinct cell means) Allows for freely fit mean in each cell It will take ab parameters to describe the mean structure. Y ijk is kth observation from population with level i of A and level j of B

2 Effects model: Y ijk = µ + α i + β j + γ ij + ijk α i is the main effect parameter for level i of A β j is the main effect parameter for level j of B γ ij is the interaction parameter for level i of A and level j of B There are (+a+b+ab)parameterstodescribethe mean structure, this is an over-parameterization. We ll need to use constraints to make the parameters estimable. But, the effects model can be useful when there are only main effects present (no interaction) because it s a much simpler model to interpret (more on this later). Y ijk is kth observation from population with level i of A and level j of B 5 Example: Poisondata Time : Survival time (in 0 hour units) Poison : Factor with levels,, : Factor with levels,,, > poison.data=read.delim("poison.data) > head(poison.data) Poison Time > levels() NULL ## R sees it as numeric, redefine as categorical: > =as.factor() > is.factor() [] TRUE > levels() [] "" "" "" "" 6 > levels(poison) NULL ##Seen as numeric, redefine as categorical: > Poison=as.factor(Poison) > is.factor(poison) [] TRUE > levels(poison) [] "" "" "" Get the cell means: > means=tapply(time,list(poison,),mean) > means Get the number of observations in each cell: > table(poison,) Poison 7 This is a balanced -way ANOVA. To get the appropriate tests in R in a - way ANOVA, we will use the sum-to-zero constraints for the dummy regressors. These are: i α i =0, j β j =0and i,j γ ij =0 > contrasts(poison) ## Reassign using sum-to-zero constraint: > contrasts(poison)=contr.sum(levels(poison)) > contrasts(poison) [,] [,] ## Do the same for the other factor: > contrasts()

3 > contrasts()=contr.sum(levels()) > contrasts() [,] [,] [,] You can get a quick plot of the means using the interaction.plot function. > interaction.plot(poison,,time) mean of Time More sophisticated plot: > poison.num=as.numeric(poison) > plot(c(0.5,.5),range(time),type="n",xlab="poisson", ylab="survival Time",axes=F,main="Survival Time vs. Poison by group (jittered)") > axis() > axis(,at=:,c("","","")) > box() > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > lines(c(,,),means[,],col=) > lines(c(,,),means[,],col=) > lines(c(,,),means[,],col=) > lines(c(,,),means[,],col=) > legend(,.,c("trt ","TRT ","TRT ","TRT "),col=c(,,,),lty=) Poison 9 0 Survival Time Survival Time vs. Poison by group (jittered) Poisson TRT TRT TRT TRT Fit the model that includes interaction and get the overall F -test: > lm.out=lm(time ~ Poison + + Poison:) > summary(lm.out) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < e-6 *** Poison e-05 *** Poison * e-05 *** e-06 *** * Poison: Poison: Poison: Poison: Poison: Poison: Signif. codes: 0 *** 0.00 ** 0.0 * Residual standard error: 0.9 on 6 degrees of freedom Multiple R-squared: 0.75,Adjusted R-squared: 0.65 F-statistic: 9.0 on and 6 DF, p-value:.986e-07 The overall F -test p-value is quite small, and we reject the overall F -test. At least one of the terms (Poison,, or Poison:) in the model is significant.

4 The summary() command also provides a test for each dummy regressor for each factor, but we re probably more interested in testing for each factor as a whole, initially. The function Anova (notice the capital A) is a function in the car library. We can use this to compute type III sums of squares, and do appropriate tests for the effects in the model. > Anova(lm.out,type="III") Anova Table (Type III tests) Response: Time (Intercept) <.e-6 *** Poison.00.7.e-07 *** e-06 *** Poison: Residuals Signif. codes: 0 *** 0.00 ** 0.0 * NOTE: We need to be careful using the anova function for unbalanced data, because it computes type I sums of squares (sequential sums of squares) for testing the significance of the terms. We want type III sums of squares. In models that include interaction (as this one does), a common first step is to test for the significance of the interaction. If there is no interaction, we perceive the true profile plot of the means to be parallel. mean of Time Poison The interaction is NOT significant in this model. Poison: So, these lines (connecting the cell means) are not significantly different than parallel. Amodellikethis,withnosignificantinteraction, is said to have only main effects. We can plot the main effects fitted model, which shows parallel lines: Main effects model (Survival Time vs. Poison by ) When the interaction is not significant, we can go back and fit the reduced model (without interaction), and use this reduced model for the rest of the analysis. > lm.no.interaction=lm(time ~ Poison + ) Survival Time Poisson TRT TRT TRT TRT > Anova(lm.no.interaction,type="III") Anova Table (Type III tests) Response: Time (Intercept) <.e-6 *** Poison e-07 *** e-06 *** Residuals.0509 Signif. codes: 0 *** 0.00 ** 0.0 * Since there is no significant interaction, we can say The effect is the same for all levels of Poison. Or, at each Poison level, the effect is the same. Effects model with only Main Effects: Y ijk = µ + α i + β j + ijk Both main effects are significant here. 5 6

5 We can get the parameter estimates like usual: > lm.no.interaction$coefficients (Intercept) Poison Poison You can also get the fitted values using the lm() output: > cbind(poison,,lm.no.interaction$fitted) Poison There will be unique fitted values in this -way ANOVA (one for each cell). 7 We could also plug-in the estimated parameters to the main effects model to get the fitted values... Ŷ ijk = ˆµ + ˆα i + ˆβ j To plot the main effects fitted model with a profile plot, we can get the fitted means using the parameters, and then connect the relevant fitted values. > coeff=lm.no.interaction$coefficients > coeff (Intercept) Poison Poison > ## Sum the relevant parameter estiamates to get the fitted values: > mu.=sum(coeff[c(,,)]) > mu.=sum(coeff[c(,,)]) > mu.=sum(coeff[c(,)])-sum(coeff[c(,)]) > mu.=sum(coeff[c(,,5)]) > mu.=sum(coeff[c(,,5)]) > mu.=sum(coeff[c(,5)])-sum(coeff[c(,)]) > mu.=sum(coeff[c(,,6)]) > mu.=sum(coeff[c(,,6)]) > mu.=sum(coeff[c(,6)])-sum(coeff[c(,)]) > mu.=sum(coeff[c(,)])-sum(coeff[c(,5,6)]) > mu.=sum(coeff[c(,)])-sum(coeff[c(,5,6)]) > mu.=coeff[]-sum(coeff[c(,)])-sum(coeff[c(,5,6)]) 8 ## Create vector of means for each treatment: > TRT..means=c(mu.,mu.,mu.) > TRT..means=c(mu.,mu.,mu.) > TRT..means=c(mu.,mu.,mu.) > TRT..means=c(mu.,mu.,mu.) > plot(c(0.5,.5),range(time),type="n",xlab="poisson", ylab="survival Time",axes=F,main="Main effects model (Survival Time vs. Poison by )") > axis() > axis(,at=:,c("","","")) > box() > poison.num=as.numeric(poison) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) > points(jitter(poison.num[==""],factor=0.5), Time[==""],pch="",col=) Survival Time Main effects model (Survival Time vs. Poison by ) Poisson TRT TRT TRT TRT > lines(c(,,),trt..means,col=) > lines(c(,,),trt..means,col=) > lines(c(,,),trt..means,col=) > lines(c(,,),trt..means,col=) > legend(,.,c("trt ","TRT ","TRT ","TRT "),col=c(,,,),lty=) 9 0

6 If there IS significant interaction, we shouldn t consider the tests for main effects (i.e. don t try to use the tests for Poison or in the Anova() output if Poison: was significant). Instead, we should test for the effect separately at each Poison level (because the effect depends on the level of Poison, and really, there isn t a common global effect to be testing all at once). mean of Time Poison Another option for testing for interaction is to use a Partial F -test. > lm.no.interaction=lm(time ~ Poison + ) > anova(lm.no.interaction,lm.out) Analysis of Variance Table Model : Time ~ Poison + Model : Time ~ Poison + + Poison: Res.Df RSS Df Sum of Sq F Pr(>F) This p-value is the same as the one seen in... > Anova(lm.out,type="III") Anova Table (Type III tests) Response: Time (Intercept) <.e-6 *** Poison.00.7.e-07 *** e-06 *** Poison: Residuals Signif. codes: 0 *** 0.00 ** 0.0 * Different types of interaction ( factors/ levels) Cells: Different types of interaction ( factors/ levels) Factor B is in same direction for all levels of Factor A: (but not parallel lines) No interaction: (parallel lines) Factor B is in different direction for different levels of Factor A: (lines cross)

7 Assumptions of two-way ANOVA independent simple random samples of size n ij from each of a b normal populations. constant variance across all populations ijk are normally distributed as N(0, σ ) A general ANOVA table for two-way ANOVA Source Sum of Squares df Mean Square F A SS A a- B SS B b- AB SS AB (a-)(b-) SS A a = MS A SS B b = MS B SS AB (a )(b ) = MS AB MS A MSE MS B MSE MS AB MSE Residuals RSS n-ab RSS n ab = MSE Total TSS N- Sums of squares We still have the same fundamental relationship that TSS = RegSS + RSS. The RegSS represents the variability explained by the model, which includes the terms of main effects for factor A and B and the interaction between A and B, in the full model. 5 If we have equal sample sizes in each cell (a balanced design), then TSS = SS A + SS B + SS AB + RSS But if we have an unbalanced design, then this equation does not hold (i.e. RegSS = SS A +SS B +SS AB ). In this case, we can still test for effects using the Type III sums of squares (as with the Anova() function) looking for a significant effect given all other factors are accounted for. 6