STAT22200 Spring 2014 Chapter 8A

STAT22200 Spring 2014 Chapter 8A Yibi Huang May 13, 2014 81-86 Two-Way Factorial Designs Chapter 8A - 1

Problem 81 Sprouting Barley (p166 in Oehlert) Brewer s malt is produced from germinating barley, so brewers like to know under what conditions they should germinate their barley The following is part of an experiment on barley germination 30 lots of barley seeds, 100 seeds per lot, are randomly divided into 10 groups of 3 lots Each group receives a treatment according to water amount used in germination 4 ml or 8 ml age of seeds in weeks after harvest 1, 3, 6, 9, or 12 Response: # of seeds germinating Age of Seeds (weeks) water 1 3 6 9 12 11 7 9 13 20 4(ml) 9 16 19 35 37 6 17 35 28 45 8 1 5 1 11 8(ml) 3 7 9 10 15 3 3 9 9 25 Chapter 8A - 2

Basic Terminology The sprouting barley experiment has 10 treatments The 10 treatments has a factorial structure A factor is an experimentally adjustable variable, eg water amount used in germination, age of seeds in weeks after harvest, Factors have levels, eg water amount is a factor with 2 levels (4 ml or 8 ml) age of seeds is a factor with 5 levels (1, 3, 6, 9, 12 weeks) A treatment is a combination of factors In the barley experiment, the treatments are the 2 5 combinations of the possible levels of the two factors (4ml, 1 wk) (4ml, 3 wks) (4ml, 6 wks) (4ml, 9 wks) (4ml, 12 wks) (8ml, 1 wk) (8ml, 3 wks) (8ml, 6 wks) (8ml, 9 wks) (8ml, 12 wks) Chapter 8A - 3

Full k-way Factorial Design Consider k factors with respectively L 1, L 2,, L k levels, a full k-way factorial design include all the L 1 L 2 L k combination of the k factors as treatments A factorial design is said to be balanced if all the treatment groups have the same number of replicates Otherwise, the design is unbalanced Question: How many units are there in a 3 2 design with 4 replicates? Balanced designs have many advantages, but not always necessary sometimes if a unit fails (ex, a test tube gets dropped) we might end up with unbalanced results even if the original design was balanced Chapter 8A - 4

Data for a Two-Way a b Design with n Replicates B-level 1 B-level 2 B-level b A-level 1 A-level 2 A-level a y 111 y 112 y 11n y 211 y 212 y 21n y a11 y a12 y a1n y 121 y 122 y 12n y 221 y 222 y 22n y a21 y a22 y a2n Chapter 8A - 5 y 1b1 y 1b2 y 1bn y 2b1 y 2b2 y 2bn y ab1 y ab2 y abn

Display of Data from Two Way Factorial Designs Age of Seeds (weeks) y ijk 1 3 6 9 12 11 7 9 13 20 water 4(ml) 9 16 19 35 37 6 17 35 28 45 8 1 5 1 11 water 8(ml) 3 7 9 10 15 3 3 9 9 25 Age of Seeds (weeks) Row means Cell means y ij 1 3 6 9 12 y i water 4(ml) 867 1333 2100 2533 3400 2047 water 8(ml) 467 367 767 667 1700 793 Column means y j 667 850 1433 1600 2550 y = 142 overall mean Does water have an effect on gemination? Does the age of seeds have an effect? Chapter 8A - 6

How to Get Cell Means in R? To get cell means (average of the 3 values in each cell) > barley = readtable("sproutingbarleytxt",header=t) > attach(barley) > tapply(y,list(water,week),mean) 1 3 6 9 12 4 8666667 13333333 21000000 25333333 34 8 4666667 3666667 7666667 6666667 17 To get the row means > tapply(y,water,mean) 4 8 20466667 7933333 To get the column means > tapply(y,week,mean) 1 3 6 9 12 6666667 8500000 14333333 16000000 25500000 Chapter 8A - 7

Graphical Display of Data Interaction Plots cell means Age of Seeds (weeks) y ij 1 3 6 9 12 water 4(ml) 867 1333 2100 2533 3400 water 8(ml) 467 367 767 667 1700 Interaction plots: plotting cell means (y ij ) against levels of one factor (A or B), with different lines for the other factor (B or A) mean of y 5 10 20 30 1 1 2 2 1 1 2 2 1 2 water 1 2 4 8 mean of y 5 10 20 30 5 4 3 2 1 5 34 12 week 5 3 4 1 2 12 6 9 1 3 1 3 6 9 12 Age of Seed (week) 4 8 Water Amount (ml) > interactionplot(week,water,y,type="b",xlab="age of Seed (week)") > interactionplot(water,week,y,type="b",xlab="water Amount (ml)") Chapter 8A - 8

Means Model for a Two-Way Factorial Design For a a b two-way factorial experiment with n replicates means model : y ijk = µ ij + ε ijk for { i = 1,, a, j = 1,, b, k = 1,, n y ijk = the kth replicate in the treatment formed from the ith level of factor A and jth level of factor B ε ijk s are iid N(0, σ 2 ) µ ij = the mean response in the treatment formed from the ith level of factor A and jth level of factor B The means model regards the 2-way factorial design as a CRD with a b treatments, ignoring the factorial structure of the treatments Chapter 8A - 9

Main Effects (1) Though the means model y ijk = µ ij + ε ijk ignores factorial structure of the treatments, one can use appropriate contrasts to explore the effects of the two factors Eg, if one wants to compare the effects of level 1 and 2 of factor A, one can use the contrast C = µ 11 + µ 12 + µ 1b b µ 21 + µ 22 + µ 2b b = µ 1 µ 2 In general, if one wants to compare between levels of factor A, the contrasts are all of the form C = µ i1 µ i2 Observe µ i = mean response at the ith level of factor A, averaged over all levels of factor B We call µ i, i = 1,, a, as the main effects of factor A Similarly, µ j, j = 1,, b, are called the main effects of factor B Chapter 8A - 10

Main Effects (2) As µ i s and µ j s are important parameters We thus give them new notations µ i = µ + α i = overall effect + effect due to factor A µ j = µ + β j = overall effect + effect due to factor B in which, µ = µ is the overall average of all µ ij s We also call α i s as the main effect of factor A, and β j s as the main effect of factor B Observe that only a 1 of the α i s can be arbitrary since α i = i=1 µ i aµ = i=1 i=1 y i ( {}}{ 1 ) b b µ ij aµ j=1 = 1 b µ a ab µ = 0 Similarly, one can show that b j=1 β j = 0 Chapter 8A - 11

Interaction (1) Two factors A and B are said to have a two-way interaction if the effects of factor A change with the levels of factor B To be more specific, based on the means model y ijk = µ ij + ε ijk, the effect of changing factor A from level i 1 to level i 2 is { µ i2 j 1 µ i1 j 1 if factor B is fixed at level j 1 If µ i2 j 2 µ i1 j 2 if factor B is fixed at level j 2 µ i2 j 2 µ i1 j 2 (µ i2 j 1 µ i1 j 1 ) = µ i1 j 1 µ i1 j 2 µ i2 j 1 + µ i2 j 2 = 0, then level (i 1, i 2 ) of factor A doesn t interact with level (j 1, j 2 ) of factor B If none of the levels of factor A interact the levels of factor B, ie, µ i1 j 1 µ i1 j 2 µ i2 j 1 + µ i2 j 2 = 0, for all i 1, i 2, j 1, j 2, then we say factor A and factor B have no interaction Chapter 8A - 12

Interaction (2) If factor A and factor B have no interaction, we claim that µ ij = µ + α i + β j, for all i, j, in which µ = µ, α i = µ i µ, and β i = µ j µ Proof µ ij µ α i β j = µ ij µ i µ j + µ = µ ij 1 µ im 1 m µ lj + 1 b a ab = 1 ab = 1 ab m=1 l=1 m=1 l=1 m=1 µ ij 1 ab l=1 l=1 m=1 l=1 m=1 µ im 1 ab (µ ij µ im µ lj + µ lm ) = 0 }{{} = 0, since no interaction Chapter 8A - 13 µ lm l=1 m=1 µ lj + 1 ab l=1 m=1 µ lm

Interaction (3) In view of the result on the previous slide, we define the interaction terms of factor A and factor B as αβ ij def = µ ij µ α i β j, for all i = 1,, a, j = 1,, b The interaction terms αβ ij s have the following properties αβ ij = 0, for all i, j if the two factors do not interact a i=1 αβ ij = 0 for all j and b j=1 αβ ij = 0 for all i In other words, the row sums and column sums of the array below are all 0 αβ 11 αβ 12 αβ ab αβ 21 αβ 22 αβ 2b αβ a1 αβ a2 αβ ab See the next slide for proofs Chapter 8A - 14

Interaction (4) αβ ij = i=1 (µ ij µ i µ j + µ ) i=1 ( 1 ) = µ j b µ i aµ j + aµ i=1 which is valid for all j = 1,, b = µ j 1 b µ a a µ j + a ab µ = 0, HW today: Show that a j=1 αβ ij = 0 for all i Chapter 8A - 15

Main-Effect-Interaction Model for 2-Way Factorial Designs The main-effect-interaction model for a two-way factorial design is y ijk = µ + α i + β j + αβ ij + ε ijk for { i = 1,, a, j = 1,, b, k = 1,, n Unlike the means model y ijk = µ ij + ε ijk that there is no constraints on the parameter µ ij s, the main-effect-interaction model has several constraints α i = i=1 β j = j=1 αβ ij = i=1 αβ ij = 0, for all i, j The means model and the main-effect-interaction model are related as follows j=1 µ = µ, α i = µ i µ, β j = µ j µ αβ ij = µ ij µ i µ j + µ Chapter 8A - 16

Main-Effect-Interaction Model for 2-Way Factorial Designs µ 11 µ 12 µ ab µ 21 µ 22 µ 2b µ a1 µ a2 µ ab = µ + α 1 α 2 α a + β 1 β 2 β b + αβ 11 αβ 12 αβ ab αβ 21 αβ 22 αβ 2b αβ a1 αβ a2 αβ ab Chapter 8A - 17

Additive Model A model is said to be additive if all interaction terms are 0 y ijk = µ + α i + β j + ε ijk for { i = 1,, a, j = 1,, b, k = 1,, n In other words, additive models for two-way factorial designs assume no interactions between the two factors An additive model also has constraints on parameters α i = i=1 β j = 0 j=1 Chapter 8A - 18

Interaction Plot Revisit Plot cell means (y ij ) against levels of one factor (i or j), with different lines for the other factor (j or i) > barley = readtable("sproutingbarleytxt",header=t) > attach(barley) > interactionplot(week,water,y,type="b",xlab="age of Seed (week)") > interactionplot(water,week,y,type="b",xlab="water Amount (ml)") mean of y 5 10 20 30 1 1 2 2 1 1 2 2 1 2 water 1 2 4 8 mean of y 5 10 20 30 5 4 3 2 1 5 34 12 week 5 3 4 1 2 12 6 9 1 3 1 3 6 9 12 Age of Seed (week) 4 8 Water Amount (ml) Chapter 8A - 19

Interaction Plots Revisit(2) Parallel lines indicate no interaction Interaction No Interaction Mean of Y 0 1 2 3 4 5 Factor B = 1 Factor B = 2 Mean of Y 0 1 2 3 4 5 Factor B = 1 Factor B = 2 1 2 3 1 2 3 Factor A Factor A Chapter 8A - 20

Interaction Plot Revisit (3) What does the interaction plot below tell us? Mean of Y 0 1 2 3 4 5 Factor B = 1 Factor B = 2 1 2 3 Factor A Chapter 8A - 21

Estimation of Parameters (1) Parameter estimation in a balanced factorial design is straightforward For the means model and the effects model, y ijk = µ ij + ε ijk (means model) y ijk = µ + α i + β j + αβ ij + ε ijk (main-effect-interaction model) the parameter estimates are µ ij = y ij µ = y, α i = y i y, β j = y j y αβ ij = y ij y i y j + y Observe the estimates also satisfy the zero-sum constraints: α i = i=1 β j = j=1 αβ ij = i=1 αβ ij = 0, for all i, j j=1 Chapter 8A - 22

Estimation of Parameters (2) Since the design is balanced, for any of the reduced models below, y ijk = µ + ε ijk (no main effects, no interaction) y ijk = µ + α i + ε ijk (main effects of A only) y ijk = µ + β j + ε ijk (main effects of B only) y ijk = µ + α i + β j + ε ijk (additive model) the estimates of µ, α i s, and β j s are the same as in the main-effects-interaction model: µ = y, α i = y i y, βj = y j y If NOT balanced, the estimate will change with the model Recall in a regression model, the estimate of a coefficient will change with the model formula Chapter 8A - 23

Fitted Values for a Main-Effect-Interaction Model For a main-effect-interaction model, the fitted value for y ijk is simply y ij because ŷ ijk = µ + α i + β j + αβ ij = y + (y i y ) + (y j y ) + (y ij y i y j + y ) = y ij = cell mean Chapter 8A - 24

Fitted Values for an Additive Model For an additive model (no interaction), the fitted value for y ijk is ŷ ijk = µ + α i + β j = y + (y i y ) + (y j y ) = y i + y j y = row mean + column mean overall mean Chapter 8A - 25

Example 86 Bacteria in Cheese (p178 in Oehlert) Factor A: Bacteria R50#10, added or not Factor B: Bacteria R21#2, added or not 3 replicates Response: total free amino acids in cheddar cheese after 56 days of ripening No R21 R21 added No 1697 2211 R50 1601 1673 1830 1973 R50 2032 2091 added 2017 2255 2409 2987 Is there interaction? No R21 R21 added No R50 y 11 = 1709 y 12 = 1952 R50 added y 21 = 2153 y 22 = 2444 mean of y 16 20 24 R21 added No R21 No R50 Chapter 8A - 26 R50 added

Example 86 Bacteria in Cheese (p178 in Oehlert) B-level 1 B-level 2 row mean A-level 1 y 11 = 1709 y 12 = 1952 y 1 = 1831 A-level 2 y 21 = 2153 y 22 = 2444 y 2 = 2299 column mean y 1 = 1931 y 2 = 2198 y = 2065 µ = y = 2065 α 1 = y 1 y = 1831 2065 = 0234 β 1 = y 1 y = 1931 2065 = 0134 αβ 11 = y 11 y 1 y 1 + y = 1709 1831 1931 + 2065 = 0012 The estimates of all other parameters can be computed using the zero-sum constraints α 1 + α 2 = 0 α 2 = α 1 = 0234 β 1 + β 2 = 0 β 2 = β 2 = 0134 αβ 11 + αβ 12 = 0 αβ 12 = αβ 11 = 0012 αβ 11 + αβ 21 = 0 αβ 21 = αβ 11 = 0012 αβ 12 + αβ 22 = 0 αβ 22 = αβ 12 = 0012 Chapter 8A - 27

Sum of Squares for Balanced 2-Way Factorial Designs (1) An balanced a b two-way factorial design with n replicates is also a CRD with ab treatments, so the sum of squares identity is still valid SST = SS trt + SSE where SST = SS trt = n i=1 j=1 k=1 i=1 j=1 n (y ijk y ) 2 and (y ij y ) 2, SS E = i=1 j=1 k=1 df for SST = total # of observations 1 = abn 1 df for SS trt = # of treatments 1 = ab 1 n (y ijk y ij ) 2 df for SSE = total # of observations # of treatments = abn ab = ab(n 1) Chapter 8A - 28

Sum of Squares for Balanced 2-Way Factorial Designs (2) As the ab treatments have a factorial structure, SS trt can be decomposed further in which SS trt = SS A + SS B + SS AB SS formula df SS A SS B SS AB SS trt n a n a n a n a b i=1 b i=1 b i=1 b i=1 j=1 (y i y ) 2 a 1 j=1 (y j y ) 2 b 1 j=1 (y ij y i y j +y ) 2 (a 1)(b 1) j=1 (y ij y ) 2 ab 1 Observe the df for the SS of the main effects or interactions = (number of parameters) (number of constraint(s)) Chapter 8A - 29

Sum of Squares for Balanced 2-Way Factorial Designs (3) In summary SST = SS A + SS B + SS AB + SSE SST = SS A = SS B = SS AB = SS E = i=1 j=1 k=1 i=1 j=1 k=1 i=1 j=1 k=1 i=1 j=1 k=1 i=1 j=1 k=1 n (y ijk y ) 2 n (y i y }{{ ) } α i n (y j y ) }{{} β j 2 = bn 2 = an i=1 j=1 α 2 i β 2 j n (y ij y i y j + y ) }{{} αβ ij n (y ijk y ij ) 2 Chapter 8A - 30 2 = n i=1 j=1 αβ 2 ij

Expected Values for the Mean Squares Just like CRD, the mean squares for factorial design are the sum of squares divided by the corresponding df MS A = SS A a 1, MS B = SS B b 1, MS AB = SS AB (a 1)(b 1), MSE = Under the effects model for a balanced two-way factorial, y ijk = µ + α i + β j + αβ ij + ε ijk ε ijk s are iid N(0, σ 2 ) one can show that E(MS A ) = σ 2 + E(MS AB ) = σ 2 + bn a 1 i=1 n (a 1)(b 1) α 2 i, E(MS B ) = σ 2 + an b 1 i=1 j=1 Again the MSE is an unbiased estimator of σ 2 Chapter 8A - 31 SSE ab(n 1) j=1 β 2 j αβij, 2 E(MSE) = σ 2

ANOVA Table for Balanced Two-Way Factorial Designs Source df SS MS F Factor A a 1 SS A MS A = SS A a 1 Factor B b 1 SS B MS B = SS B b 1 SS AB F A = MS A MSE F B = MS B MSE AB Interaction (a 1)(b 1) SS AB MS AB = (a 1)(b 1) F AB = MS AB MSE Error ab(n 1) SSE MSE = SSE ab(n 1) Total abn 1 SST Chapter 8A - 32

Questions of Interest in a 2-Way Factorial Design 1 Does factor A has an effect on the response? Eg does the age of seeds has an effect on germination? { H 0 : α 1 = =α a =0 F A = MS A H a : not all α i s = 0, MSE F a 1, ab(n 1) under H 0 2 Does factor B has an effect on the response? Eg does the water amount has an effect on germination? { H 0 : β 1 = =β b =0 F B = MS B H a : not all β i s = 0, MSE F b 1, ab(n 1) under H 0 3 Does the effect of factor A interact with that of factor B? Eg, does the effect of age change with water amount? { H 0 : αβ ij =0 for all i, j H a : αβ ij 0 for some i, j F AB = MS AB MSE F (a 1)(b 1), ab(n 1) under H 0 Chapter 8A - 33

Example 86 Bacteria in Cheese (p178 in Oehlert) SS A = bn a i=1 α2 i = 2 3 [( 0234) 2 + 0234 2 ] = 0656 SS B = an b β j 2 = 2 3 [( 0134) 2 + 0134 2 ] = 0214 SS AB = n a j=1 b i=1 j=1 αβ 2 ij = 3 [0012 2 4] 00017 Computing SSE needs more work It is easier to compute the SST: b n SST = a (y ijk y i=1 j=1 k=1 ) 2 = (1697 2065) 2 + (1601 2065) 2 + (1830 2065) 2 Then we can get + + (2987 2065) 2 = 1598 SSE = SST SS A SS B SS AB = 1598 0656 0214 00018 = 0726 Chapter 8A - 34

Example 86 Bacteria in Cheese ANOVA table Source df SS MS F -value P-value A(R50) 1 0656 0656 723 0028 B(R21) 1 0214 0214 236 016 AB interaction 1 00017 00017 0019 089 Error 8 0726 0091 Total 11 1598 Only main effect A (Bacteria R50) is moderately significant Main effect B and interaction are not One can also get the ANOVA table in R as follows > lmcheese = lm(y ~ r50 + r21 + r50*r21, data=cheese) > anova(lmcheese) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) r50 1 065614 065614 72335 002752 * r21 1 021440 021440 23636 016275 r50:r21 1 000178 000178 00196 089217 Residuals 8 072566 009071 Chapter 8A - 35

Advantage and Disadvantage of Factorial Designs Advantage: Factorial design is superior to one-at-a-time designs that change only one factor at a time because factorial design can test the effects of both factors at once more efficient than one-at-a-time design, taking fewer units to attain the same power; one can investigate interaction of factors in a factorial design, but not in one-at-a-time experiments By trying a wide range of treatments we broaden the inductive base for generalizing our results Disadvantage: If there are many factors or many levels, the size of the experiment grows rapidly Remedy: fractional factorial designs (Chapter 18) If blocking is necessary, natural block size may not be big enough to run a full factorial in one block Chapter 8A - 36

Q1: Does the age of seeds has an effect on germination? We can ignore water amount, and analyze the data as a CRD of 5 treatments Though the units are watered at two different levels, this is still a fair comparison because we ensure that half of the units get 4ml and half 8ml for each of the 5 Age of Seeds (weeks) 1 3 6 9 12 11 7 9 13 20 9 16 19 35 37 6 17 35 28 45 8 1 5 1 11 3 7 9 10 15 3 3 9 9 25 > barley = readtable("sproutingbarleytxt",header=t) > lmage = lm(y ~ asfactor(week), data=barley) > anova(lmage) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) asfactor(week) 4 13211 33028 31984 002987 * Residuals 25 25817 10327 Age of seeds has a significant effect on germination Chapter 8A - 37

Q2: Does the water amount has an effect on germination? We can ignore seed age, and analyze the data as a CRD of 2 treatments Though the seeds different in age, this is still a fair comparison because the 2 treatment groups have the same distribution in seed age > lmwater = lm(y ~ asfactor(water), data=barley) > anova(lmwater) Analysis of Variance Table water Response 11 7 9 13 20 4(ml) 9 16 19 35 37 6 17 35 28 45 8 1 5 1 11 8(ml) 3 7 9 10 15 3 3 9 9 25 Response: y Df Sum Sq Mean Sq F value Pr(>F) asfactor(water) 1 11781 117813 12107 0001663 ** Residuals 28 27247 9731 Water amount also has a significant effect on germination Chapter 8A - 38

You might have realized that a better idea to test the age effect is to block on water amount and analyze like a > lmboth = lm(y ~ asfactor(week) + asfactor(water), data=barley) > anova(lmboth) Analysis of Variance Table Response: y Df Sum Sq Mean Sq F value Pr(>F) asfactor(week) 4 13211 33028 56477 00023801 ** asfactor(water) 1 11781 117813 201457 00001525 *** Residuals 24 14035 5848 By blocking on water amount, the age effect becomes more significant Conversely, one can also regard water amount as treatment and seed age as block Water effect also become more significant after blocking on seed age Chapter 8A - 39