Conditional Effects Contrasts of interest main effects" for factor 1: μ i: μ:: i = 1; ; : : : ; a μ i: μ k: i = k main effects" for factor : μ :j μ::

Transcription

1 8. Two-way crossed classifications Example 8.1 Days to germination of three varieties of carrot seed grown in two types of potting soil. Soil Variety Type 1 1 Y 111 = Y 11 = 1 Y 11 = 1 Y 11 = 10 Y 1 = 1 Y 1 = Y 11 = 11 Y 11 = 1 Y 1 = 1 Y 1 = 18 Y 1 = 1 Y = 9 Y 1 = 19 Y = 1 Y 1 = 18 9 This is called an unbalanced" factorial experiment. Sample sizes Soil Variety type 1 1 n 11 = n 1 = n 1 = n 1 = n = 1 n = In general we have i = 1; ; : : : ; a j = 1; ; : : : ; b n ij > 0 levels for the first factor levels for the second factor observations at the i-th level of the first factor and the j-th level of the second factor 9 We will restrict our attention to normal-theory Gauss-Markov models. Cell means" model: where Y ijk = μ ij + ijk ijk ο NID(0; ff ) 8 >< >: i = 1; : : : ; a j = 1; : : : ; b k = 1; : : : ; n ij Clearly, E(Y ijk ) = μ ij is estimable if n ij > 0. 9 Overall mean response: μ:: = 1 ab ax bx μ ij i=1 j=1 Mean response at the i-th level of factor 1, averaging across the levels of factor. μ i: = 1 b bx j=1 μ ij Mean response at the j-th level of factor, averaging across the levels of factor 1 μ :j = 1 a ax i=1 μ ij 98

2 Conditional Effects Contrasts of interest main effects" for factor 1: μ i: μ:: i = 1; ; : : : ; a μ i: μ k: i = k main effects" for factor : μ :j μ:: j = 1; ; : : : ; b μ :j μ :` j = ` μ ij μ kj 8>< μ ij μ i` 8>< >: >: i = k j = 1; ; : : : ; b j = ` i = 1; ; : : : ; a Interaction Contrasts (μ ij μ kj ) (μ i` μ k`) = (μ ij μ i`) (μ kj μ k`) = μ ij μ kj μ i` + μ k` All of these contrasts are estimable when n ij > 0 for all (i; j) An effects" model Y ijk = μ + ff i + fi j + fl ij + ij because E( μ Y ij: ) = μ ij Any linear function of estimable functions is estimable where ijk ο NID(0; ff ) i = 1; ; : : : ; a j = 1; ; : : : ; b k = 1; ; : : : ; n ij >

3 Y Y 11 Y 11 Y 11 μ Y 1 ff 1 Y 11 ff Y 1 fi 1 Y 11 fi = Y 1 fi Y 1 fl 11 Y 1 fl 1 Y 1 fl 1 Y fl 1 Y fl fl Y The resulting restricted model is where Y ijk = μ + ff i + fi j + fl ij + ijk ijk ο NID(0; ff ) and ff 1 = 0 fi 1 = 0 8 >< >: i = 1; : : : ; a j = 1; : : : ; b k = 1; : : : ; n ij fl i1 = 0 for all i = 1; : : : ; a fl 1j = 0 for all j = 1; : : : ; b 0 We will call these baseline" restrictions. 0 Soil Type Soil Variety 1 Variety Variety Means 1 μ 11 = μ μ 1 = μ + fi μ 1 = μ + fi μ + fi1+fi μ 1 = μ + ff 1 μ = μ + ff μ = μ + ff μ + ff +fi + fl +fi + fl + fi+fi + fl+fl μ + ff μ + ff fl + fi + μ + ff Interpretation: μ = μ 11 = E(Y 11k ) + fi + fl is the mean response with both factors at the first level. ff i = μ i1 μ 11 = E(Y i1k ) E(Y 11k ) is the difference in mean responses between levels i and 1 of factor 1 when factor is at level 1. 0 Soil Type Soil Variety 1 Variety Variety Means 1 μ 11 = μ μ 1 = μ + fi μ 1 = μ + fi μ + fi1+fi μ 1 = μ + ff 1 μ = μ + ff μ = μ + ff μ + ff +fi + fl +fi + fl + fi+fi + fl+fl μ + ff μ + ff fl + fi + μ + ff Interpretation + fi + fl fi j = μ 1j μ 11 = E(Y 1jk ) E(Y 11k ) for j = 1; ; : : : ; b is the difference in the mean responses for levels j and 1 of factor when factor 1 is at level 1. 0

4 Soil Type Soil Variety 1 Variety Variety Means 1 μ 11 = μ μ 1 = μ + fi μ 1 = μ + fi μ + fi1+fi μ 1 = μ + ff 1 μ = μ + ff μ = μ + ff μ + ff +fi + fl +fi + fl + fi+fi + fl+fl μ + ff μ + ff + fi + fl μ + ff + fi + fl Interaction: fl ij = (μ ij μ ib ) (μ aj μ ab ) = (μ ij μ aj ) (μ ib μ ab ) Note that fl ij fl i` fl kj +fl k` = μ ij μ i` μ kj +μ k` for any (i; j) and (k; `) 0 Matrix formulation: Y 111 Y 11 Y 11 Y 11 Y 1 Y 11 Y 1 Y 11 Y 1 Y 1 Y 1 Y 1 Y 1 Y Y = μ ff fi fi fl fl " " Y = Xfi + where Y ο N(Xfi;ff I) 08 Least squares estimation: b = ^μ ^ff ^fi ^fi ^fl ^fl = (X T X) 1 X T Y = μy 11 μy 1 μ Y 11 μy 1 μ Y 11 μy 1 μ Y 11 μy μ Y 1 μ Y 1 + μ Y 11 μy μ Y 1 μ Y 1 + μ Y Restrictions must involve nonestimable" quantities for the unrestricted effects" model. Baseline restrictions: (SAS) ffa = 0 fi b = 0 fl ib = 0 for all i = 1; : : : ; a fl aj = 0 for all j = 1; : : : ; b Baseline restrictions: (S-PLUS) ff 1 = 0 fi 1 = 0 fl i1 = 0 for all i = 1; : : : ; a fl 1j = 0 for all j = 1; : : : ; b 10

5 ±-restrictions: Y ijk =! + fl i + ffi j + ij + ijk where - μ ij = E(Y ijk ) Variety 1 Variety Variety Means Soil μ 11 =! + fl 1 μ 1 =! + fl 1 μ 1 =! + fl 1 μ 1: =! + fl 1 type 1 +ffi ffi + 1 +ffi + 1 Soil μ 1 =! + fl μ =! + fl μ =! + fl μ : =! + fl type +ffi ffi + +ffi + means μ :1 =! + ffi 1 μ : =! + ffi μ : =! + ffi ijk ο NID(0; ff ) ax i=1 fl i = 0 bx j=1 ffi j = 0 ax i=1 ij = 0 for each j = 1; : : : ; b bx j=1 ij = 0 for each i = 1; : : : ; a Interpretation:! = 1 ab ax bx μ ij i=1 j=1 is the overall mean germination time, averaging across all soil types and all varieties used in this study Variety 1 Variety Variety Means Soil μ 11 =! + fl 1 μ 1 =! + fl 1 μ 1 =! + fl 1 μ 1: =! + fl 1 type 1 +ffi ffi + 1 +ffi + 1 Soil μ 1 =! + fl μ =! + fl μ =! + fl μ : =! + fl type +ffi ffi + +ffi + means μ :1 =! + ffi 1 μ : =! + ffi μ : =! + ffi Interpretation: and! + ffi j = μ :j ffi j = μ :j μ:: ffi j ffi k = (μ :j μ::) (μ :k μ::) = μ :j μ :k is the difference between mean germination times for varieties j and k, averaging across soil types. 1 Variety 1 Variety Variety Means Soil μ 11 =! + fl 1 μ 1 =! + fl 1 μ 1 =! + fl 1 μ 1: =! + fl 1 type 1 +ffi ffi + 1 +ffi + 1 Soil μ 1 =! + fl μ =! + fl μ =! + fl μ : =! + fl type +ffi ffi + +ffi + means μ :1 =! + ffi 1 μ : =! + ffi μ : =! + ffi Interpretation: fl 1 fl = μ 1: μ : is the difference in the mean germination times for different soil types, averaging across varieties. 1

6 Variety 1 Variety Variety Means Soil μ 11 =! + fl 1 μ 1 =! + fl 1 μ 1 =! + fl 1 μ 1: =! + fl 1 type 1 +ffi ffi + 1 +ffi + 1 Soil μ 1 =! + fl μ =! + fl μ =! + fl μ : =! + fl type +ffi ffi + +ffi + means μ :1 =! + ffi 1 μ : =! + ffi μ : =! + ffi Interaction: ij = μ ij (! + fl i + ffi j ) is a deviation from an additive model. Then, ij kj i` + k` = μ ij μ kj μ i` + μ k` Matrix formulation Y 111 Y 11 Y Y Y 11 1 Y Y 1! 1 Y 11 fl 1 11 = Y 1 ffi 1 1 Y 1 ffi + 11 Y Y Y Y 1 1 Y This uses the ±-restrictions to obtain fl = fl 1 ffi = ffi 1 ffi 1 = 11 1 = 11 1 = 1 = 1 = Least squares estimation b = ^! ^fl ^ffi 1 ^ffi ^ 11 ^ 1 = (X T X) 1 X T Y = 1 X X μy ij: i j 1 X j 1 X i μy 1j: 1 μy i1: 1 X X μy ij: X X μy ij: = X X μy ij: 1 X μy i: 1 i μy 11: ^! ^fl 1 ^ffi 1 μy 1: ^! ^fl 1 ^ffi 1:8 :1 : : 0: : If restrictions are placed on nonestimable" functions of parameters in the unrestricted effects" model, then The resulting models are reparameterizations of each other. 1 18

7 The solution to the normal equations ^Y = P X Y e = Y ^Y = (I P X )Y SSE = e T e = Y T (I P X )Y ^Y T ^Y = Y T PX Y SS model = Y T (P X P 1 )Y are the same for any set of restrictions. b = (X T X) 1 X T Y and interpretations of the corresponding parameters will not be the same for all such sets of restrictions. If you were to place restrictions on estimable functions of parameters in Y ijk = μ + ff 1 + fi j + fl ij + ijk then you would change 19 Analysis of variance rank(x) Y T Y = Y T PμY + Y T (Pμ;ff Pμ)Y X space spanned by the columns of ^Y = X(X T X) X T Y and OLS estimators of other estimable quantities. +Y T (P μ;ff;fi Pμ;ff)Y +Y T (P X P μ;ff;fi )Y +Y T (I P X )Y = R(μ) + R(ffjμ) + R(fijμ; ff) +R(fljμ; ff; fi) + SSE 0

8 Y 111 Y 11 Y 11 Y 11 Y 1 Y 11 Y 1 Y 11 = Y 1 Y 1 Y 1 Y 1 Y 1 Y Y μ ff 1 ff fi 1 fi fi fl 11 + fl 1 fl fl 1 fl fl " - - " call this call this call this call this X μ X ff X fi X fl Define: X μ = X μ X μ;ff = [X μ jx ff ] X μ;ff;fi = [X μ jx ff jx fi ] X = [X μ jx ff jx fi jx fl ] P μ = X μ (X T μ X μ) 1 X T μ P μ;ff = X μ;ff (X T μ;ff X μ;ff) X T μ;ff P μ;ff;fi = X μ;ff;fi (X T μ;ff;fi X μ;ff;fi) X T μ;ff;fi P X = X(X T X) X T 1 The following three model matrices correspond to reparameterizations of the same model: μ ff 1 ff fi 1 fi fi fl 11 fl 1 fl 1 fl 1 fl fl R(μ) = Y T P μ Y is the same for all three models μ ff 1 ff fi 1 fi fi fl 11 fl 1 fl fl 1 fl fl

9 R(μ; ff) = Y T P μ;ff Y is the same for all three models and so is R(ffjμ) = R(μ; ff) R(μ) μ ff 1 ff fi 1 fi fi fl 11 fl 1 fl fl 1 fl fl R(μ; ff; fi) = Y T P μ;ff;fi Y is the same for all three models and so is R(fijff) = R(μ; ff; fi) R(μ; ff) μ ff 1 ff fi 1 fi fi fl 11 fl 1 fl fl 1 fl fl R(μ; ff; fi; fl) = Y T P X Y is the same for all three models and so is R(fljμ; ff; fi) = R(μ; ff; fi; fl) R(μ; ff; fi) μ ff 1 ff fi 1 fi fi fl 11 fl 1 fl fl 1 fl fl Consequently, the partition Y T Y = Y T PμY + Y T (P μ;fi Pμ)Y +Y T (P μ;ff;fi P μ;fi )Y +Y T (P X P μ;ff;fi )Y +Y T (I P X )Y = R(μ) + R(fijμ) + R(ffjμ; fi) +R(fljμ; ff; fi) + SSE is the same for all three models. 8

10 Normal Theory Gauss-Markov Model ijk ο NID(0; ff ) By Cochran's Theorem, these quadratic forms (sums of squares) have independent chi-square distributions with 1; b 1; a 1; (a 1)(b 1); and n ab degrees of freedom, respectively, when n ij > 0 for all (i; j). Using result., we have also shown earlier that SSE = a X n ij X bx (Y ijk Y μ ij ) i=1 j=1 k=1 = Y T (I P X )Y ο χ n ab 9 0 The `m( ) function in S-PLUS: To allow the `m( ) function to fit a model involving classification variables, create factors. > carrot ψ read.table("carrots.dat", col.names=c("soil","variety","days")) > carrot$soil ψ as.factor(carrot$soil) > carrot$variety ψ as.factor(carrot$variety) > options(contrasts= c( contr.sum","contr.poly")) Produce ANOVA tables: > `m.out1 ψ `m(days ο soil*variety, data=carrot) > anova (`m.out1) R(ffjμ) R(fijμ; ff) R(fljμ; ff; fi) SSE > `m.out ψ `m(days ο variety*soil, data=carrot) > anova (`m.out) R(fijμ) R(ffjμ; fi) R(fljμ; ff; fi) SSE 1

11 There are four options for creating columns in the model matrix for classification variables: ># This file is posted as carrots.ssc > carrot <- read.table("c: carrots.dat", + col.names=c("soil","variety","days")) > carrot$soil <- as.factor(carrot$soil) > carrot$variety <- as.factor(carrot$variety) > carrot contr: helmert contr: treatment sets ff 1 = 0 fi 1 = 0 fl 1j = 0 for all j fl i1 = 0 for all i contr:sum contr:poly ± constraints orthogonal polynomial contrasts equal spacing equal sample sizes Soil Variety Days # Set up the axes and title of the # profile plot. # Compute sample means of germination # times for all combinations of soil # type and varieties of carrot seeds # and make a profile plot. > means <- tapply(carrot$days, list(carrot$variety,carrot$soil),mean) > means > par(fin=c(,),cex=1.,lwd=,mex=1.) > x.axis <- unique(carrot$variety) > matplot(c(1,,1), c(0,0,10), type="n", xlab="variety", ylab="mean Time", main= "Average Time to Carrot Seed Germination") # Add a profile for each soil type > matlines(x.axis,means,type='l', lty=c(1,),lwd=) # Plot points for the observations > matpoints(x.axis,means, pch=c(1,1)) # Add a legend to the plot > legend(.,8., legend=c('soil Type 1','Soil Type '), lty=c(1,),bty='n')

12 Mean Time Average Time to Carrot Seed Germination Variety Soil Type 1 Soil Type # Fit a model with main effects and interaction # Compute both sets of Type I sums of squares > options(contrasts= c(``contr.sum'',''contr.poly'')) > lm.out1 <- lm(days~soil*variety,data=carrot) > anova(lm.out1) Analysis of Variance Table Response: Days Terms added sequentially (first to last) Df Sum Sq Mean Sq F Value Pr(F) Soil Variety Soil:Variety Residuals # Create a data frame containing the original # data and the residuals and estimated means > lm.out <- lm(days~variety*soil,data=carrot) > anova(lm.out) > data.frame(carrot$soil,carrot$variety, carrot$days, Pred=lm.out1$fitted, Resid=round(lm.out1$resid,)) Analysis of Variance Table Response: Days Terms added sequentially (first to last) Df Sum Sq Mean Sq F Value Pr(F) Variety Soil Variety:Soil Residuals X1 X X Pred Resid

13 # Create residual plots frame( ) par(cex=1.0,mex=1.0,lwd=,pch=, mkh=0.1,fig=c(0,1,.1,1), pty='m') plot(lm.out1$fitted, lm.out1$resid, xlab="estimated Means", ylab="residuals") abline(h=0, lty=, lwd=) par(fig=c(0, 1, 0, 0.9), pty='s') qqnorm(lm.out1$resid) qqline(lm.out1$resid) Residuals Estimated Means lm.out1$resid Quantiles of Standard Normal 1 # Create plots for studentized residuals # You must attach the MASS library # to have access to the studres( ) # function that computes studentized # residuals in the following code library(mass) frame( ) par(cex=1.0,mex=1.0,lwd=,pch=, mkh=0.1,fin=c(.,.)) plot(lm.out1$fitted, studres(lm.out1), xlab="estimated Means", ylab="studentized Residuals", main="studentized Residual Plot") abline(h=0, lty=, lwd=) Studentized Residuals Studentized Residual Plot qqnorm(studres(lm.out1), main="studentized Residuals") qqline(studres(lm.out1)) Estimated Means

14 # Compute Type III sums of squares and F-tests. # First create the model matrix for # the cell means model. studres(lm.out1) Studentized Residuals Quantiles of Standard Normal > cb <- as.factor(10*as.numeric(carrot$soil) + as.numeric(carrot$variety)) > lm.out <- lm(carrot$days ~ cb - 1) > D <- model.matrix(lm.out) > D cb11 cb1 cb1 cb1 cb cb # Compute the sample means > y <- matrix(carrot$days,ncol=1) > b <- solve(crossprod(d)) %*% crossprod(d,y) > b [,1] cb11 9 cb1 1 cb1 18 cb1 1 cb 1 cb 1 # Generate an identity matrix and # a vector of ones Iden <- function(n) diag(rep(1,n)) one <- function(n) matrix(rep(1,n),ncol=1) # Compute Type III sums of squares and # related F-tests > s <- length(unique(carrot$soil)) > t <- length(unique(carrot$variety)) > yhat <- D %*% b > sse <- crossprod(y-yhat) > df <- nrow(y) - s*t > c1 <- kronecker( cbind(iden(s-1),-one(s-1)), t(one(t)) ) > q1 <- t(b) %*% t(c1)%*% solve( c1 %*% solve(crossprod(d)) %*% t(c1))%*% c1 %*% b > df1<- s-1 > f <- (q1/df1)/(sse/df) > p <- 1-pf(f,df1,df) 8

15 > c1 [,1] [,] [,] [,] [,] [,] [1,] > data.frame(ss=q,df=df1,f.stat=f,p.value=p) SS df F.stat p.value > data.frame(ss=q1,df=df1,f.stat=f,p.value=p) SS df F.stat p.value > c <- kronecker( t(one(s)), cbind(iden(t-1), -one(t-1)) > q <- t(b) %*% t(c)%*% solve( c %*% solve(crossprod(d)) %*% t(c))%*% c %*% b > df1<- t-1 > f <- (q/df1)/(sse/df) > p <- 1-pf(f,df1,df) > c [,1] [,] [,] [,] [,] [,] [1,] [,] > c <- kronecker( cbind(iden(s-1),-one(s-1)), cbind(iden(t-1),-one(t-1)) ) > q <- t(b) %*% t(c)%*% solve( c %*% solve(crossprod(d)) %*% t(c))%*% c %*% b > df1<- (s-1)*(t-1) > f <- (q/df1)/(sse/df) > p <- 1-pf(f,df1,df) > c [,1] [,] [,] [,] [,] [,] [1,] [,] > data.frame(ss=q,df=df1,f.stat=f,p.value=p) SS df F.stat p.value What null hypotheses are tested by F-tests derived from such ANOVA tables? Consider Type I sums of squares: R(μ) = Y T P 1 Y = Y T P 1 P 1 Y = (P 1 Y) T (P 1 Y) 1 ff R(μ) ο χ 1 (ffi ) and F = where = ( μ Y ::: 1) T ( μ Y :::1) = n :: μy ::: R(μ) SSE=(n :: ab) ο F (1;n :: ab) (ffi ) ffi = 1 ff fit X T P 1 Xfi = 1 ff (fit X T P 1 )(P 1 Xfi) = 1 ff (P 1Xfi) T (P 1 Xfi) For the carrot seed germination study: P 1 Xfi = 1 n :: 1 1 T Xfi = 1 n :: 1[n :: ; n 1: ; n : ; n :1 ; n : ; n : ; n 11 ; n 1 ; n 1 ; n 1 ; n ; n ] fi = 1 n :: 1 n :: μ + a X + a X i=1 bx fl ij j=1 The null hypothesis is H 0 : 0 = n :: μ+ a X i=1 i=1 n i: ff i + b X j=1 n i: ff i + With respect to the cell means b X j=1 n :j fi j + X i E(Y ijk ) = μ ij = μ + ff i + fi j + fl ij this null hypothesis is X H 0 : 0 = a bx i=1 j=1 n ij n :: μ ij n :j fi j X j n ij fl ij 1

16 Consider and F = Here, R(ffjμ) = Y T (P μ;ff P μ )Y R(ffjμ)=(a 1) MSE ο F (a 1;n:: ab) (ffi ) 1 ff R(ffjμ) ο χ a 1 (ffi ) where a 1 = rank(x μ;ff) rank(x μ ) and ffi = 1 ff fit X T (P μ;ff P μ )Xfi = 1 ff [(P μ;ff P μ )Xfi] T [(P μ;ff P μ )Xfi] For the general effects model for the carrot seed germination study: P μ;ff X = X μ;ff (X T μ;ff X μ;ff) X T μ;ff X = X μ;ff n :: n 1: n : n 1: n 1: 0 n : 0 n : n :: n 1: n : n :1 n : n : n 11 n 1 n 1 n 1 n n n 1: n 1: 0 n 11 n 1 n 1 n 11 n 1 n n : 0 n : n 1 n n n 1 n n = X μ;ff n 0 1: n : # = n11 n1 n1 n11 n1 n n1: n1: n1: n :: n :: n :: n11 n n n1 n n n1: n: n: n1: n: n1: Then, the first seven rows of (P μ;ff P μ )Xfi are» μ + ff1 + P b j=1 n 1j n 1: (fi j + fl 1j )» μ+ a X i=1 n i: n :: ff i + b X j=1 n :j n :: fi j + X i X j n ij n :: fl ij The last eight rows of (P μ;ff P μ )Xfi are» μ + ff + P b j=1 n j n : (fi j + fl j )» μ+ a X i=1 n i: n :: ff i + b X j=1 n :j n :: fi j + X i X j n ij n :: fl ij The null hypothesis is X H 0 : ff i + b n ij (fi j=1 n j + fl ij ) i: are all equal (i = 1; : : : ; a) with respect to the cell means model, μ ij = E(Y ijk = μ + ff i + fi j + fl ij ; this null hypothesis is bx n H ij 0 : μ j=1 n ij are all equal (i = 1; : : : ; a): i: