Two-Way ANOVA (Two-Factor CRD)

Two-Way ANOVA (Two-Factor CRD) STAT:5201 Week 5: Lecture 1 1 / 29

Factorial Treatment Structure A factorial treatment structure is simply the case where treatments are created by combining factors. We can generically refer to the factors with letters like A, B, C, etc. and the number of levels in are a, b, c, etc. A two-factor factorial has g = ab treatments, a three-factor factorial has g = abc treatments and so forth. We have a completely randomized design with N total number of experiment units. As mentioned earlier, we can think of factorials as a 1-way ANOVA with a single superfactor (levels as the treatments), but in most cases, it is beneficial to consider the factorial nature of the design. 2 / 29

Two-Way ANOVA (Factorial): Balanced Design Two factors: A with a levels, and B with b levels. g = a b treatments altogether, where the treatments are the combinations of the levels of the two factors. Completely randomized design with treatments randomly assigned to the g treatments. No blocking. No nesting. Example (Factors of DayLength and Climate with a = b = 2) Four treatments arising from the combination of the two factors g = 4. Treatments randomly assigned to hamsters, with two hamsters in each treatment cell or n i = 4 for all i, and N = 8. DayLength long short Climate cold warm 3 / 29

Two-Way ANOVA: Balanced Design Full Model (includes interaction): Y ijk = µ + α i + β j + (αβ) ij + ɛ ijk with ɛ ijk iid N(0, σ 2 ) for i = 1,..., a and j = 1,..., b and k = 1,..., n Total number of observations N = nab. As with the 1-way ANOVA effects model, we have an overparameterization (we have 1 + a + b + ab parameters for the mean structure, but only ab distinct groups), so we need constraints to make the parameters interpretable. Sum-to-zero restrictions on parameters: 0 = a α i = i=1 b β j = j=1 a (αβ) ij = i=1 b (αβ) ij j=1 4 / 29

Two-Way ANOVA: Balanced Design Full Model (includes interaction): Y ijk = µ + α i + β j + (αβ) ij + ɛ ijk with ɛ ijk iid N(0, σ 2 ) for i = 1,..., a and j = 1,..., b and k = 1,..., n Parameters α i, β j and (αβ) ij are fixed, unknown constants. The sum-to-zero restrictions give µ as the overall mean, or the expected value of response averaged across all treatments. The term α i is called the main effect of A at level i. The term β j is called the main effect of B at level j. The term (αβ) ij is the interaction effect of A and B in the ij treatment. The interaction effect is a measure of how far the treatment mean differs from the additive model. If all (αβ) ij = 0, then we are reduced to the additive model. 5 / 29

Two-Way ANOVA: Balanced Design Example (Animal-fattening experiment) Two primary factors: Vitamin B12 (0mg, 5mg) & Antibiotics (0mg, 40mg) Three animals randomly assigned to each of 4 treatments. Response was weight gain in lbs/week. Because we can not assume that the two factors do not interact, we should include interaction in the model. G.W. Cobb (1998). Introduction to Design & Analysis of Experiments. 6 / 29

Two-WayGtANOVA: z 5 (. '2. 2 Balanced I.S-f '-,."J Design ~ i3f< ' Example (Animal Fattening example) ~/~ 8(L ~;:b'~h( J 0 0 2 0 0 3 0 0 Lj 0 <.fo S- D yo fo 0 40 ":J- '5 o Y S- O ~ S- (o 0 5" 40 l~ C;; L{D /2- - ) L(O #~ (r~{c;or An fl'bld?'c W(,&hi- ) ~GU/\. C Ibs/Uld,.30 (. I ~ I. Do /.0,:) it o 0 l. 0 r- I I 2lo I. 2-1 I.. ( ~ [. 52- (. SlP [. <)''1 We will fit a model that includes interaction, also known as the full model. 7 / 29

Two-Way ANOVA: Parameter Estimates Our full-model fitted values: Ŷ ijk = ˆµ + ˆα i + ˆβ j + (αβ) ˆ ij Of course all observations in the same cell have the same fitted value. Our estimates for µ, α i, and β j are similar to what we saw in the additive two-factor CRD... ˆµ = Ȳ... Overall mean ˆα i = Ȳ i.. Ȳ... main effects of factor A level i ˆβ j = Ȳ.j. Ȳ... main effects of factor B level j To get our estimator for (αβ) ij, we should note that the full model (with interaction) allows for effects due to individual combinations of A and B. 8 / 29

Two-Way ANOVA: Full-Model Fitted Values The fitted values Ŷijk in the full model ARE the cell means (i.e. just the mean of all observations in the cell). Example (Animal Fattening example) The cell means for the animal fattening example give us the fitted values for the full model, or the two-way ANOVA with interaction: Ȳ 11. = Ŷ11k = 1.19 Ȳ 21. = Ŷ21k = 1.22 Ȳ 12. = Ŷ 12k = 1.03 Ȳ 11. = Ŷ 22k = 1.54 for k = 1, 2, 3 9 / 29

Two-Way ANOVA: Parameter Estimates The estimates for the interaction effects, or the (αβ) ˆ ij values, quantify how far the full-model fitted values are from the additive-model fitted values (or reduced model fitted values). Thus, we need the additive-model fitted values (we already know how to get these) and the full-model fitted values (just the cell means) to estimate the (αβ) ij values. Additive model fitted values: Ŷ ijk = ˆµ + ˆα i + ˆβ j Later, when we test for an interaction effect, relatively large ˆ (αβ) ij ˆ (αβ) ij values are all values suggest there is an interaction present. If very small, then no interaction is present, or the additive model is sufficient. 10 / 29

Two-Way ANOVA: Parameter Estimates Example (Animal Fattening example) Two primary factors: Vitamin B12 (0mg, 5mg) & Antibiotics (0mg, 40mg) What are the additive-model fitted values? Ŷ ijk = ˆµ + ˆα i + ˆβ j 11 / 29

Two-Way ANOVA: Parameter Estimates Example (Animal Fattening example) The additive model (reduced model) finds the best fit (smallest SSE) such that the interaction plot has parallel lines. l-d The estimate for each (αβ) ˆ ij can be found by subtracting the additive fit for cell ij from the full-model fit for that cell. 12 / 29

Two-Way ANOVA: Parameter Estimates Example (Animal Fattening example) The estimate for each (αβ) ˆ ij can be found by subtracting the additive fit for cell ij from the full model fit for that cell, or (αβ) ˆ ij =Ŷ ij,full Ŷ ij,additive S;m; l~ ) r-. qj(b t \.::: l, (<1 -- I. 0 "7-0, I L- A o(p'2-1 ~ I, 2 2. ~ (. 3 Y -==-- - O. /2- A crfo L 2- :::- i, ")1 -- I. C/2::: O. I L 13 / 29

o(p'2-1 ~ I, 2 2. ~ (. 3 Y -==-- - O. /2- Two-Way A ANOVA: Parameter Estimates crfo L 2- :::- i, ")1 -- I. C/2::: O. I L -- v.. - v y. --y. 111~ I' -. + cj I".. 'J «: ~Po 14 / 29

Two-Way ANOVA: Parameter Estimates -- v.. - v y. --y. 111~ I' -. + cj I".. 'J «: ~Po 15 / 29

Two-Way ANOVA: Parameter Estimates Example (Animal Fattening example) Use the parameter estimates to show that fitted is truly the same as the cell mean. Y es~ ~ ~. s~ a4lj t, 1';, k- -=- Y ~ CJl.fJ2 V>'U2a/Yl -3-1/ 3 -r - ~'~ ~/V~ S~/rt ~ ~ 6C2/~ JYz ~ (C S to?? - Iv -?::c r: 0 ~" r:ie»i-r- I C!;'on 5, 16 / 29

Two-Way ANOVA: Restrictions on Parameters The estimators we ve shown here are based on the sum-to-zero restrictions. The animal fattening experiment has 4 cell means, and only 4 parameters are needed to describe the mean structure (i.e. µ 1, µ 2, µ 3, µ 4 ) But the full effects model utilizes 9 parameters to describe the 4 means: µ, α 1, α 2, β 1, β 2, (αβ) 11, (αβ) 12, (αβ) 21, (αβ) 22 So, we utilize restrictions to make the parameters uniquely estimable (and interpretable). 17 / 29

Two-Way ANOVA: Restrictions on Parameters Sum-to-zero restrictions on parameters: a b a b 0 = α i = β j = (αβ) ij = (αβ) ij i=1 j=1 i=1 j=1 SAS uses a different constraint or restriction, and this affects the interpretation or the parameters (we will see this soon). The other parameter to be estimated is σ 2 in the full model: Lito O~ jjta/(~ :h k d~ co a: L c:., It, _ "Z.. D- 2-= fylse zz: '0[;;:. _:- z (Y~~ - Y:;L~) N -~~S ) N - (~~ ) Example (Animal Fattening example) For the full model with the animal fattening data, ˆσ 2 = SSE 12 4 = 0.029 8 = 0.0036 and ˆσ = 0.0602 18 / 29

Two-Way ANOVA: SS for Balanced Design 1~/1''')a.. I 2. h v,j k '" Ai + G-f -' ~. +fy# )'" + ~I~I~.., b A 2 i d ~ AN'OIJIt f~ J" IwD-~ tr.{) (6.f-tt ld. r 5J;t.Jf,Idr) I I T '/,I I t ;f~sj~/~ bj~ dh/~ Lu~'II~ e~ ~ ~~ d-f: 5S ;\II S E(MS) A - 8 b -I 19 / 29

Two-Way ANOVA: SS for Balanced Design Oehlert provides the table with slightly Factorial different Treatment notation, Structure using the effects notation: Term Sum of Squares Degrees of Freedom A B AB Error Total a bn( α i ) 2 a 1 i=1 b an( β j ) 2 b 1 j=1 a,b i=1,j=1 a,b,n i=1,j=1,k=1 a,b,n i=1,j=1,k=1 n( αβ ij ) 2 (a 1)(b 1) (y ijk y ij ) 2 ab(n 1) (y ijk y ) 2 abn 1 20 / 29

Two-Way ANOVA: SS for Balanced Design dels with Parameters Reminder of estimators, shown in Oehlert. µ = y α i = y i µ = y i y β j = y j µ = y j y αβ ij = y ij µ α i β j = y ij y i y j + y Display 8.2: Estimators for main effects and interactions in a two-way factorial. of them can vary freely; there are a 1 degrees of freedom for factor ilarly, the β j values must sum to 0, so at most b 1 of them can vary 21 / 29

Two-Way ANOVA: Two-Factor SAS Example Experiment Example (Animal Fattening example) +r6yjt ett( 1(' tryfi I( -h'{o/fu e Two Primary Factors: Vitamin B12 (Omgand 5 mg) and Antibioitic (Omg and 40 mg) Response: Weight gain in lbs/week Three animals are randomly assigned to each of the 4 treatments as a completely randomized design / (CRD). SAS Program Part 1: /*Fit the full model (with interaction)*/ proc glm data=anfat plot=diagnostics; class animal antibiotic vitamin; model gain=vitamin antibiotic vitamin*antibiotic; output out=diagnostics p=predicted r=residual; run; Ani, b,ot/l ~rn SAS Output from Part 1: The GLM Procedure Dependent Variable: gain Source Model Error Corrected R-Square 0.934046 Total Coeff Var 4.835982 Sumof DF / Squares 3 0.41070000 8 0.02900000 11 0.43970000 ~ Root MSE~ gain Mean 0.060208 1.245000 Mean Square F Value Pr > F 0.13690000 0.00362500 37.77 <.0001 (}1t~~ F--6.4l " ~I 22 / 29

0.934046 4.835982 0.060208 1.245000 Two-Way ANOVA: SAS Example Source DF Type I SS Mean Square F Value Pr > F vitamin 1 0.21870000 0.21870000 60.33 <.0001 antibiotic 1 0.01920000 0.01920000 5.30 0.0504 vitamin*antibiotic 1 0.17280000 0.17280000 47.67 0.0001 \~;:- DF Type III SS Mean Square F Value Pr > F \ vitamin 1 0.21870000 0.21870000 60.33 <.0001 antibiotic 1 0.01920000 0.01920000 5.30 0.0504, vit ami.n=arrtabf otlc 1 0.17280000 0.17280000 47.67 0.0001 Example (Animal Fattening example)! - -------- ~---~--- ~-~..-~----~- --- ANDVA kb/e 1 2 - -fug-lvr-s wiftt Wt Ie (tl-it5f1- We start by testing the highest-order interaction term, if that interaction is not significant, then we move to lower-order terms. Here, the interaction is significant (p=0.0001), so we will NOT consider the tests for main effects. The hierarchy principle says they must now remain in the model regardless of their significance. Remember to check all assumptions for the model using diagnostics. 23 / 29

Two-Way ANOVA: SAS Example Example (Animal Fattening example) See handout on PROC GLM, two-way ANOVA 24 / 29

Two-Way ANOVA: Overparameterization Y ijk = µ + α i + β j + (αβ) ij + ɛ ijk with ɛ ijk iid N(0, σ 2 ) Writing the factor effects model (a = 2, b = 2, n = 2) as a linear model Y = X β + ɛ µ 1 1 0 1 0 1 0 0 0 1 1 0 1 0 1 0 0 0 α 1 ɛ 111 1 1 0 0 1 0 1 0 0 α 2 ɛ 112 Y = 1 1 0 0 1 0 1 0 0 β 1 ɛ 121 1 0 1 1 0 0 0 1 0 β 2 + ɛ 122 1 0 1 1 0 0 0 1 0 (αβ) 11 ɛ 211 1 0 1 0 1 0 0 0 1 (αβ) 12 ɛ 212 (αβ) 1 0 1 0 1 0 0 0 1 21 ɛ 221 ɛ (αβ) 222 22 But the model is overparametrized and the X matrix is not of full rank. The OLS estimates could be found by using a generalized inversed: ˆβ = (X X ) XY, but we can also fix this by imposing restrictions on the parameters. 25 / 29

Two-Way ANOVA: Re-written X design matrix Using sum-to-zero restrictions and re-writing as a full rank design matrix: 1 1 1 1 ɛ 111 1 1 1 1 ɛ 112 1 1 1 1 µ ɛ 121 Y = 1 1 1 1 α 1 1 1 1 1 β 1 + ɛ 122 ɛ 211 1 1 1 1 (αβ) 11 ɛ 212 1 1 1 1 ɛ 221 1 1 1 1 ɛ 222 Other parameters determined by model restrictions: α 2 = α 1, β 2 = β 1, (αβ) 12 = (αβ) 21 = (αβ) 11, (αβ) 22 = (αβ) 21 26 / 29

Two-Way ANOVA: the Cell Means version of the model Though not commonly used, on occasion we may write the two-way ANOVA model using the cell means model notation: Y ijk = µ ij + ɛ ijk with ɛ ijk iid N(0, σ 2 ) where i = 1,..., a; j = 1,... b, k = 1..., n ij - µ ij is the mean of all units given level i of factor A and level j of factor B. - µ i. = - µ.j = - µ.. = j µ ij b is the mean response at level i of factor A. i µ ij a is the mean response at level j of factor B. i j µ ij ab is the overall mean response. 27 / 29

".. -,.",'" -,------ "--'''''-'--'''~'~''''-''-- -. Two-Way ANOVA: the Cell Means version of the model Example (Drill Speed and Feed Rate) ~f'u>1-u: rljuul ~ 7 4. divu /1.uY. A mechanical engineer is studying the thrust force developed by a drill press. Two primary factors are investigated; Drill Speed (125, 200) and Feed Rate (0.02, A IN.MM1I~ 0.03, 0.05, ~/~ 0.06). Two ;; runs J~fi7 will be performed Ik ~ at each combination performed in random order. JOLU- ~ 'c a- ~ I'/'~ F~rd Drii{s~ tun 6. ()'5. - o. o~ D. 0(, /2- C; z. 7-0 2. 'I') I 2. wo 1-2. "15 2. n I 2, f<j 2,?- z. I 2, '8"~ 1'-----, -+ "..,-_"..--,. 0.' I 200 2. g-~ I 2.3'') 2. "Z? 2.11'" _--- 2, Y/.p I 2, 70 2 -?7-2 ' 77 b.f. ij1ntl;~ {UJt>~} D~~ l<~~ IY{ I:~/~. W,'l d :04. rl/~ ~~: D.C. Montgomery (2005). Design and Analysis of Experiments. Wiley :USA 28 / 29

Two-Way ANOVA: the Cell Means version of the model Example (Drill Speed and Feed Rate) Thought of as cell means using cell means notation below: /2) e'lho q And the design matrix for X in the cell means model is again of full rank and no restrictions are needed for parameter estimation. 29 / 29