Unbalanced Data in Factorials Types I, II, III SS Part 1

Transcription

1 Unbalanced Data in Factorials Types I, II, III SS Part 1 Chapter 10 in Oehlert STAT:5201 Week 9 - Lecture 2 1 / 14

2 When we perform an ANOVA, we try to quantify the amount of variability in the data accounted for by a specific source using Sums of Squares (SS). For instance, SS A gives us an idea of the amount of variability in the data accounted for by factor A. For any model, we know SS total = SS model + SSE. In a balanced 2-way ANOVA, we have SS total = SS A + SS B + SS AB + SSE which means we can partition the SS model into non-overlapping SS as SS model = SS A + SS B + SS AB 2 / 14

3 We can also perceive the SS for a given source (like SS A ) to be the extra variability explained when the respective term is added to the model. And if the factorial is unbalanced, quantifying the extra variability that is explained by adding a term depends upon which terms have already been entered into the model. This is because the terms are not orthogonal. Thus, order of entry into the model matters. Suppose we have already fit an additive model with main effects of A and B and we re thinking of adding an interaction term. What is the SS AB? (This would be the last term entered into model) Recall that when the interaction is not in the model, then the SS AB is in the error term... SS AB = SSE A,B SSE A,B,AB NOTE: SSE A,B represents the sum of the squared errors for the main effects model. 3 / 14

4 And to test for AB interaction, we have F = ( SSEA,B ) SSE A,B,AB df for AB MSE A,B,AB = ( ) SSAB df AB MSE A,B,AB = MS AB MSE A,B,AB which follows a central F (df AB,dferror ) under the null being true. In a balanced design, we also have the nice formula SS AB = i j n(ȳ ij. ȳ i.. ȳ.j. + ȳ... ) 2 but in unbalanced designs, the formulas are not as clean and we usually think of SS in terms of how much more is explained by adding another term, given some previous model that was fit. 4 / 14

5 Recall the balanced 2-way ANOVA Animal Fattening Experiment: Antibiotics (0mg, 40mg) and Vitamin (0mg, 5mg) Example (R balanced factorial) # Set the dummy variable coding to sum-to-zero constraints > options(contrasts=c("contr.sum","contr.poly")) > attach(af) > table(vitamin, antibiotic) antibiotic vitamin / 14

6 Recall that the anova() function in R provides Type I SS, but all Types of SS are the same when you have a balanced design (i.e. orthogonality). Example (R balanced factorial) > anova(lm(gain~vitamin+antibiotic)) Analysis of Variance Table Response: gain Df Sum Sq Mean Sq F value Pr(>F) vitamin * antibiotic Residuals # SWITCHING THE ORDER OF ENTRY > anova(lm(gain~antibiotic+vitamin)) Analysis of Variance Table Response: gain Df Sum Sq Mean Sq F value Pr(>F) antibiotic vitamin * Residuals / 14

7 The SS AB can be calculated in two ways as SS AB = SSE A,B SSE A,B,AB and SS AB = i j n(ȳ ij. ȳ i.. ȳ.j. + ȳ... ) 2. Example (R balanced factorial) > (SSE.add <- anova(lm(gain~vitamin+antibiotic))[3,2]) [1] > lm.nonadd <- lm(gain~vitamin+antibiotic + vitamin:antibiotic) > (SSE.nonadd <- anova(lm.nonadd)[4,2]) [1] > (SS.interaction <- SSE.add-SSE.nonadd) [1] # USING THE FORMULA > vitamin.means <- tapply(gain,vitamin,mean) > antibiotic.means <-tapply(gain,antibiotic,mean) > y.bar <- mean(gain) > SS.interaction.by.formula <- sum((lm.nonadd$fitted-rep(vitamin.means,each=6)- rep(rep(antibiotic.means,each=3),2)+rep(y.bar,12))^2) > SS.interaction.by.formula [1] / 14

8 When we have unbalanced data, the SS for a term depends on the Type of SS being requested. This is because Type I, II, and III each use a different set of terms already being in the model before calculating how much the next variable entered explains. Again, in a balanced design, these SS are all the same because each term provides unique information and doesn t take away from what another term explains. In an orthogonal design, if you remove a term from the model, then that term s SS are added to the error. The other SS are not affected. 8 / 14

9 Types of sums of squares Type I SS In Type I SS, the order in which terms are entered into the model matters. These SS are called sequential sums of squares. For example, consider a regression scenario with the following undoubtedly correlated variables: X 1 triglyceride level X 2 cholesterol level X 3 age Y blood pressure In SAS, if we code the model: 1 model = X1 X2 X3; then the Type I SS for X 3 describes the variability explained by X 3 after accounting for X 1 & X 2. This is because X 1 & X 2 appear in the model statement before X 3. 9 / 14

10 1 model = X1 X2 X3; gives us the following Type I SS... Source Type I SS X 3 SS(X 3 1, X 1, X 2 ) = SSE 1,X1,X 2 SSE 1,X1,X 2,X 3 NOTE: the 1 represents accounting for the intercept These SS tell us how much more explained by the model when X 3 is entered sequentially after the other two terms. And the other Type I SS for 1 model... X 1 SS(X 1 1) = SSE 1 SSE 1,X1 (entered first) X 2 SS(X 2 1, X 1 ) = SSE 1,X1 SSE 1,X1,X 2 (entered second) 10 / 14

11 Suppose we write the model in SAS as: 2 model = X3 X1 X2; then Source Type I SS X 3 SS(X 3 1) = SSE 1 SSE 1,X3 (entered first) And the other Type I SS for 2 model... X 1 SS(X 1 1, X 3 ) = SSE 1,X3 SSE 1,X1,X 3 (entered second) X 2 SS(X 2 1, X 1, X 3 ) = SSE 1 SSE 1,X1,X 2,X 3 (entered last) 11 / 14

12 Thinking in terms of regression and the some general model with predictors X 1, X 2, X 3, when would the Type I SS for X 3 be approximately the same for models 1 and 2 above? (i.e. When would the conditioning statements not make a difference?) SS(X 3 1, X 1, X 2 ) added last? SS(X 3 1) added first The Type I SS are calculated based on the order of entry into the model. Sometimes this may be useful, like when choosing a polynomial model, but often, it s not the best option for testing the significance of a term. Do you want the order of entry to impact the significance? 12 / 14

13 When the factorial is balanced, the conditioning doesn t change the SS because the terms provide unique nonoverlapping information. SS(A 1) = SS(A 1, B) Recall in the balanced design, SS model = SS A + SS B + SS AB. In this 2x2 factorial example, the 3 d.f. for the model coincide with three 1 d.f. orthogonal contrasts. Let µ = (µ 11, µ 12, µ 21, µ 22 ) Contrast SS df c A = (1, 1, 1, 1) SS A 1 c B = (1, 1, 1, 1) SS B 1 c AB = (1, 1, 1, 1) +SS AB + 1 SS model 3 * NOTE: These are not orthogonal contrasts if unbalanced. 13 / 14

14 We can also see this orthogonality in the full rank design matrix X with columns labeled µ, A, B, AB... X = x4 The dot product of any two columns is zero. In this case of orthogonality, the SS isn t affect by which terms are entered first. Sometimes I think of it as the terms are orthogonal to each other, and what each term explains doesn t overlap with what another term explains. In contrast, when the terms are not orthogonal (i.e. the design is unbalanced), the terms share some information, or there is some overlap in what each term explains. 14 / 14