Design and Analysis of

Design and Analysis of Multi-Factored Experiments Module Engineering 7928-2 Two-level Factorial Designs L. M. Lye DOE Course 1 The 2 k Factorial Design Special case of the general factorial design; k factors, all at two levels The two levels are usually called low and high (they could be either quantitative or qualitative) Very widely used in industrial experimentation Form a basic building block for other very useful experimental designs (DNA) Special (short-cut) methods for analysis We will make use of Design-Expert for analysis L. M. Lye DOE Course 2 1

Chemical Process Example A = reactant concentration, B = catalyst amount, y = recovery L. M. Lye DOE Course 3 The Simplest Case: The 2 2 - and + denote the low and high levels of a factor, respectively Low and high are arbitrary terms Geometrically, the four runs form the corners of a square Factors can be quantitative or qualitative, although their treatment in the final model will be different L. M. Lye DOE Course 4 2

Estimating effects in two-factor two-level experiments Estimate of the effect of A a 1 b 1 -a 0 b 1 estimate of effect of A at high B a 1 b 0 -a 0 b 0 estimate t of effect of A at low B sum/2 estimate of effect of A over all B Or average of high As average of low As. Estimate of the effect of B a 1 b 1 - a 1 b 0 estimate of effect of B at high A a 0 b 1 -a 0 b 0 estimate of effect of B at high A sum/2 estimate of effect of B over all A Or average of high Bs average of low Bs L. M. Lye DOE Course 5 Estimating effects in two-factor two-level experiments Estimate the interaction of A and B a 1 b 1 -a 0 b 1 estimate of effect of A at high B a 1b 0 -a 0b 0 estimate of effect of A at low B difference/2 estimate of effect of B on the effect of A called as the interaction of A and B a 1 b 1 -a 1 b 0 a 0 b 1 -a 0 b 0 difference/2 estimate of effect of B at high A estimate of effect of B at low A estimate of the effect of A on the effect of B Called the interaction of B and A Or average of like signs average of unlike signs L. M. Lye DOE Course 6 3

Estimating effects, contd... Note that the two differences in the interaction estimate are identical; by definition, the interaction of A and B is the same as the interaction of B and A. In a given experiment one of the two literary statements of interaction may be preferred by the experimenter to the other; but both have the same numerical value. L. M. Lye DOE Course 7 Remarks on effects and estimates Note the use of all four yields in the estimates of the effect of A, the effect of B, and the effect of the interaction of A and B; all four yields are needed and are used in each estimates. Note also that the effect of each of the factors and their interaction can be and are assessed separately, this in an experiment in which both factors vary simultaneously. Note that with respect to the two factors studied, the factors themselves together with their interaction are, logically, all that can be studied. These are among the merits of these factorial designs. L. M. Lye DOE Course 8 4

Remarks on interaction Many scientists feel the need for experiments which will reveal the effect, on the variable under study, of factors acting jointly. This is what we have called interaction. The simple experimental design discussed here evidently provides a way of estimating such interaction, with the latter defined in a way which corresponds to what many scientists have in mind when they think of interaction. It is useful to note that interaction was not invented by statisticians. It is a joint effect existing, often prominently, in the real world. Statisticians have merely provided ways and means to measure it. L. M. Lye DOE Course 9 Symbolism and language A is called a main effect. Our estimate of A is often simply written A. B is called a main effect. Our estimate of B is often simply written B. AB is called an interaction effect. Our estimate of AB is often simply written AB. So the same letter is used, generally without confusion, to describe the factor, to describe its effect, and to describe our estimate of its effect. Keep in mind that it is only for economy in writing that we sometimes speak of an effect rather than an estimate of the effect. We should always remember that all quantities formed from the yields are merely estimates. L. M. Lye DOE Course 10 5

The following table is useful: Table of signs A B AB a 0 b 0 (1) - - + a 0 b 1 (b) - + - a 1 b 0 (a) + - - a 1 b 1 (ab) + + + Notice that in estimating A, the two treatments with A at high level are compared to the two treatments with A at low level. Similarly B. This is, of course, logical. Note that the signs of treatments in the estimate of AB are the products of the signs of A and B. Note that in each estimate, plus and minus signs are equal in number L. M. Lye DOE Course 11 Example 1 15 Example 1 Example 2 B Example 2 B 14 A + 15 13 Low High Low High 14 B + A=2.5 13 12 B=2 Y11 10 12 10 15 10 A - 12 Y11 B - Low 9 10 (1) b Low (1) b 9-2 -1 0 1 A B 13 15 A High 15 15-2 -1 0 1 A Example 1 High 15 B + a ab a ab 14 13 A=3 11 12 Y B - 10 9 Example 3 B Example 4 B 15 Example 4-2 -1 0 1 Low High Low High 14 A 13 B 12 12 -, 12 Y Example 3 Low Low 15 (1) b 10 11 B + 14 B + 9 A A 13 B - -2-1 0 1 12 A Y11 High High 10 9-2 -1 0 1 A 10 (1) 13 a Example A B AB 1 3* 2 0 2 2.5 2.5-2.5 3 0 0-3 4 0 0 0 13 b 10 ab 12 a 12 ab Discussion of examples: Notice that in examples 2 & 3 interaction is as large as or larger than main effects. *A = [-(1) - b + a + ab]/2 = [-10-12 + 13 + 15]/2 L. M. Lye DOE Course = 3 12 6

Change of scale, by multiplying each yield by a constant, multiplies each estimate by the constant but does not affect the relationship of estimates to each other. Addition of a constant to each yield does not affect the estimates. The numerical magnitude of estimates is not important here; it is their relationship to each other. L. M. Lye DOE Course 13 Modern notation and Yates order Modern notation: a 0 b 0 = 1 a 0 b 1 = b a 1 b 0 = a a 1 b 1 = ab We also introduce Yates (standard) order of treatments and yields; each letter in turn followed by all combinations of that letter and letters already introduced. This will be the preferred order for the purpose of analysis of the yields. It is not necessarily the order in which the experiment is conducted; that will be discussed later. For a two-factor two-level factorial design, Yates order is 1 a b ab Using modern notation and Yates order, the estimates of effects become: A = (-1 + a - b + ab)/2 B = (-1 - a + b +ab)/2 AB = (1 -a - b + ab)/2 L. M. Lye DOE Course 14 7

Three factors each at two levels Example: The variable is the yield of a nitration process. The yield forms the base material for certain dye stuffs and medicines. Low high A time of addition of nitric acid 2 hours 7 hours B stirring time 1/2 hour 4 hours C heel absent present Treatments (also yields) Yates order: 1 a b ab c ac bc abc L. M. Lye DOE Course 15 Estimating effects in three-factor two-level designs (2 3 ) Estimate of A (1) a - 1 estimate of A, with B low and C low (2) ab - b estimate of A, with B high and C low (3) ac - c estimate of A, with B low and C high (4) abc - bc estimate of A, with B high and C high = (a+ab+ac+abc - 1-b-c-bc)/4, = (-1+a-b+ab-c+ac-bc+abc)/4 (in Yates order) L. M. Lye DOE Course 16 8

Estimate of AB Effect of A with B high - effect of A with B low, all at C high plus effect of A with B high - effect of A with B low, all at C low Note that interactions are averages. Just as our estimate of A is an average of response to A over all B and all C, so our estimate of AB is an average response to AB over all C. AB = {[(4)-(3)] + [(2) - (1)]}/4 = {1-a-b+ab+c-ac-bc+abc)/4, in Yates order or, = [(abc+ab+c+1) - (a+b+ac+bc)]/4 L. M. Lye DOE Course 17 Estimate of ABC interaction of A and B, at C high minus interaction of A and B at C low ABC = {[(4) - (3)] - [(2) - (1)]}/4 =(-1+a+b-ab+c-ac-bc+abc)/4, in Yates order or, =[abc+a+b+c - (1+ab+ac+bc)]/4 L. M. Lye DOE Course 18 9

This is our first encounter with a three-factor interaction. It measures the impact, on the yield of the nitration process, of interaction AB when C (heel) goes from C absent to C present. Or it measures the impact on yield of interaction AC when B (stirring time) goes from 1/2 hour to 4 hours. Or finally, it measures the impact on yield of interaction BC when A (time of addition of nitric acid) goes from 2 hours to 7 hours. As with two-factor two-level factorial designs, the formation of estimates in three-factor two-level factorial designs can be summarized in a table. L. M. Lye DOE Course 19 Sign Table for a 2 3 design A B AB C AC BC ABC 1 - - + - + + - a + - - - - + + b - + - - + - + ab + + + - - - - c - - + + - - + ac + - - + + - - bc - + - + - + - abc + + + + + + + L. M. Lye DOE Course 20 10

Example Yield of nitration process discussed earlier: 1 a b ab c ac bc abc Y = 72 7.2 84 8.4 20 2.0 30 3.0 67 6.7 92 9.2 34 3.4 37 3.7 A = main effect of nitric acid time = 1.25 B = main effect of stirring time = -4.85 AB = interaction of A and B = -0.60 C = main effect of heel = 0.60 AC = interaction of A and C = 0.15 BC = interaction i of B and C = 0.45 ABC = interaction of A, B, and C = -0.50 NOTE: ac = largest yield; AC = smallest effect L. M. Lye DOE Course 21 We describe several of these estimates, though on later analysis of this example, taking into account the unreliability of estimates based on a small number (eight) of yields, some estimates may turn out to be so small in magnitude as not to contradict the conjecture that the corresponding true effect is zero. The largest estimate is -4.85, the estimate of B; an increase in stirring time, from 1/2 to 4 hours, is associated with a decline in yield. The interaction AB = -0.6; an increase in stirring time from 1/2 to 4 hours reduces the effect of A, whatever it is (A = 1.25), on yield. Or equivalently L. M. Lye DOE Course 22 11

an increase in nitric acid time from 2 to 7 hours reduces (makes more negative) the already negative effect (B = -485) of stirring time on yield. Finally, ABC = -0.5. Going from no heel to heel, the negative interaction effect AB on yield becomes even more negative. Or going from low to high stirring time, the positive interaction effect AC is reduced. Or going from low to high nitric acid time, the positive interaction effect BC is reduced. All three descriptions of ABC have the same numerical value; but the chemist would select one of them, then say it better. L. M. Lye DOE Course 23 Number and kinds of effects We introduce the notation 2 k. This means a factor design with each factor at two levels. The number of treatments in an unreplicated 2 k design is 2 k. The following table shows the number of each kind of effect for each of the six two-level designs shown across the top. L. M. Lye DOE Course 24 12

2 2 2 3 2 4 2 5 2 6 2 7 Main effect 2 3 4 5 6 7 2f factor interaction ti 1 3 6 10 15 21 3 factor interaction 1 4 10 20 35 4 factor interaction 1 5 15 35 5 factor interaction 1 6 21 6 factor interaction 7 factor interaction 1 7 1 3 7 15 31 63 127 In a 2 k design, the number of r-factor effects is C k r = k!/[r!(k-r)!] L. M. Lye DOE Course 25 Notice that the total number of effects estimated in any design is always one less than the number of treatments In a 2 2 design, there are 2 2 =4 treatments; we estimate 2 2-1 = 3 effects. In a 2 3 design, there are 2 3 =8 treatments; we estimate 2 3-1 = 7 effects One need not repeat the earlier logic to determine the forms of estimates in 2 k designs for higher values of k. A table going up to 2 5 follows. L. M. Lye DOE Course 26 13

2 E f f e c t s 2 23 24 25 A B AB C AC BC ABC D AD BD ABD CD ACD BCD ABCD E AE BE ABE CE ACE BCE ABCE DE ADE BDE ABDE CDE ACDE BCDE ABCDE T r e a t m e n t s 1 - - + - + + - - + + - + - - + - + + - + - - + + - - + - + + - a + - - - - + + - - + + + + - - - - + + + + - - + + - - - - + + b - + - - + - + - - + + - - + - - + - + + - + - + - + - - + - + ab + + + - - - - - - - - + + + + - - - - + + + + + + + + - - - - c - - + + - - + - + + - - + + - - + + - - + + + + - - + + - - + ac + - - + + - - - - + + - - + + - - + + - - + + + + - - + + - - bc - + - + - + - - + - + - + - + - + - + - + - + + - + - + - + - abc + + + + + + + - - - - - - - - - - - - - - - - + + + + + + + + d - - + - + + - + - - + - + + - - + + - + - - + - + + - + - - + ad + - - - - + + + + - - - - + + - - + + + + - - - - + + + + - - bd - + - - + - + + - + - - + - + - + - + + - - - + - + + - + - abd + + + - - - - + + + + - - - - - - - - + + + + - - - - + + + + cd - - + + - - + + - - + + - - + - + + - - + + - - + + - - + + - acd + - - + + - - + + - - + + - - - - + + - - + + - - + + - - + + bcd - + - + - + - + - + - + - + - - + - + - + - + - + - + - + - + abcd + + + + + + + + + + + + + + + - - - - - - - - - - - - - - - - e - - + - + + - - + + - + - - + + - - + - + + - - + + - + - - + ae + - - - - + + - - + + + + - - + + - - - - + + - - + + + + - - be - + - - + - + - + - + + - + - + - + - - + - + + + - + + - + - abe + + + - - - - - - - - + + + + + + + + - - - - - - - - + + + + ce - - + + - - + - + + - - + + - + - - + + - - + - + + - - + + - ace + - - + + - - - - + + - - + + + + - - + + - - - - + + - - + + bce - + - + - + - - + - + - - + + - + - + - + - + - + - + - + abce + + + + + + + - - - - - - - - + + + + + + + + - - - - - - - - de - - + - + + - + - - + - + + - + - - + - + + - + - - + - + + - ade + - - - - + + + + - - - - + + + + - - - - + + + + - - - - + + bde - + - - + - + + - + - - + - + + - + - - + - + + - + - - + - + abde + + + - - - - + + + + - - - - + + + + - - - - + + + + - - - - cde - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + acde + - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - bcde - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - abcde + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + L. M. Lye DOE Course 27 Main effects in the face of large interactions Several writers have cautioned against making statements about main effects when the corresponding interactions are large; interactions describe the dependence of the impact of one factor on the level of another; in the presence of large interaction, main effects may not be meaningful. L. M. Lye DOE Course 28 14

Factors not studied In any experiment, factors other than those studied may be influential. Their presence is sometimes acknowledged under the dubious title experimental error. They may be neglected, but the usual cost of neglect is high. For they often have uneven impact, systematically affecting some treatments more than others, and thereby seriously confounding inferences on the studied factors. It is important to deal explicitly with them; even more, it is important to measure their impact. How? L. M. Lye DOE Course 29 1. Hold them constant. 2. Randomize their effects. 3. Estimate t their magnitude by replicating the experiment. 4. Estimate their magnitude via side or earlier experiments. 5. Argue (convincingly) that the effects of some of these non-studied d factors are zero, either in advance of the experiment or in the light of the yields. 6. Confound certain non-studied factors. L. M. Lye DOE Course 30 15

Simplified Analysis Procedure for 2-level Factorial Design Eti Estimate t factor effects Formulate model using important effects Check for goodness-of-fit of the model. Interpret results Use model for Prediction L. M. Lye DOE Course 31 Example: Shooting baskets Consider an experiment with 3 factors: A, B, and C. Let the response variable be Y. For example, Y = number of fbaskets made out of 10 Factor A = distance from basket (2m or 5m) Factor B = direction of shot (0 or 90 ) Factor C = type of shot (set or jumper) Factor Name Units Low Level (-1) High Level (+1) A Distance m 2 5 B Direction Deg. 0 90 C Shot type Set Jump L. M. Lye DOE Course 32 16

Treatment Combinations and Results Order A B C Combination Y 1-1 -1-1 (1) 9 2 +1-1 -1 a 5 3-1 +1-1 b 7 4 +1 +1-1 ab 3 5-1 -1 +1 c 6 6 +1-1 +1 ac 5 7-1 +1 +1 bc 4 8 +1 +1 +1 abc 2 L. M. Lye DOE Course 33 Estimating Effects Order A B AB C AC BC ABC Comb Y 1-1 -1 +1-1 +1 +1-1 (1) 9 2 +1-1 -1-1 -1 +1 +1 a 5 3-1 +1-1 -1 +1-1 +1 b 7 4 +1 +1 +1-1 -1-1 -1 ab 3 5-1 -1 +1 +1-1 -1 +1 c 6 6 +1-1 -1 +1 +1-1 -1 ac 5 7-1 +1-1 +1-1 +1-1 bc 4 8 +1 +1 +1 +1 +1 +1 +1 abc 2 Effect A = (a + ab + ac + abc)/4 - (1 + b + c + bc)/4 = (5 + 3 + 5 + 2)/4 - (9 + 7 + 6 + 4)/4 = -2.75 L. M. Lye DOE Course 34 17

Effects and Overall Average Using the sign table, all 7 effects can be calculated: Effect A = -2.75 Effect B = -2.25 225 Effect C = -1.75 Effect AC = 1.25 Effect AB = -0.25 Effect BC = -0.25 Effect ABC = -0.25 The overall average value = (9 + 5 + 7 + 3 + 6 + 5 + 4 + 2)/8 = 5.13 L. M. Lye DOE Course 35 Formulate Model The most important effects are: A, B, C, and AC Model: Y = β 0 + β 1 X 1 + β 2 X 2 + β 3 X 3 + β 13 X 1 X 3 β 0 = overall average = 5.13 β 1 = Effect [A]/2 = -2.75/2 = -1.375 β 2 = Effect [B]/2 = -2.25/2 = -1.125 β 3 = Effect [C]/2 = -1.75/2 = - 0.875 β 13 = Effect [AC]/2 = 1.25/2 = 0.625 Model in coded units: Y = 5.13-1.375 X 1-1.125 X 2-0.875 X 3 + 0.625 X 1 X 3 L. M. Lye DOE Course 36 18

Checking for goodness-of-fit Actual Predicted Value Value 9.00 9.13 5.00 5.13 7.00 6.88 3.00 2.87 6.00 6.13 5.00 4.63 400 4.00 388 3.88 2.00 2.37 DESIGN-EXPERT Plot Baskets Predicted 9.13 7.34 5.56 3.78 2.00 Predicted vs. Actual 2.00 3.78 5.56 7.34 9.13 Amazing fit!! Actual L. M. Lye DOE Course 37 Interpreting Results # out of f10 10 8 (9+5+6+5)/4=6.25 6 4 2 Effect of B=4-6.25= -2.25 (7+3+4+2)/4=4 0 90 B # out of 10 10 8 6 4 2 C: Shot type C(-1) C (+1) 2m 5m A Interaction of A and C = 1.25 At 5m, Jump or set shot about the same BUT at 2m, set shot gave higher values compared to jump shots L. M. Lye DOE Course 38 19

Statistical Details: Errors of estimates in 2 k designs 1. Meaning of σ 2 Assume that each treatment has variance σ 2. This has the following meaning: consider any one treatment and imagine many replicates of it. As all factors under study are constant throughout these repetitions, the only sources of any variability in yield are the factors not under study. Any variability in yield is due to them and is measured by σ 2. L. M. Lye DOE Course 39 Errors of estimates in 2 k designs, Contd.. 2. Effect of the number of factors on the error of an estimate What is the variance of an estimate of an effect? In a 2 k design, 2 k treatments go into each estimate; the signs of the treatments are + or -, depending on the effect being estimated. Note: σ 2 (kx) = k 2 σ 2 (x) So, any estimate = 1/2 k-1 [generalized (+ or -) sum of 2 k treatments] σ 2 (any estimate) = 1/2 2k-2 [2 k σ 2 ] = σ 2 /2 k-2 ; The larger the number of factors, the smaller the error of each estimate. L. M. Lye DOE Course 40 20

Errors of estimates in 2 k designs, Contd.. 3. Effect of replication on the error of an estimate What is the effect of replication on the error of an estimate? Consider a 2 k design with each treatment replicated n times. 1 a b abc d - - - - - - - - - - - - --- - - - - - - - - --- L. M. Lye DOE Course 41 Errors of estimates in 2 k designs, Contd.. Any estimate = 1/2 k-1 [sums of 2 k terms, all of them means based on samples of size n] σ 2 (any estimate) = 1/2 2k-2 [2 k σ 2 /n] = σ 2 /(n2 k-2 ); The larger the replication per treatment, the smaller the error of each estimate. L. M. Lye DOE Course 42 21

So, the error of an estimate depends on k (the number of factors studied) and n (the replication per factor). It also (obviously) depends on σ 2. The variance σ 2 can be reduced holding some of the non-studied factors constant. But, as has been noted, this gain is offset by reduced generality of any conclusions. L. M. Lye DOE Course 43 Effects, Sum of Squares and Regression Coefficients Contrast Effect = k 1 n2 SS = [ Contrast] n2 k 2 β 0 = grandmean β 1 Effect = 2 i L. M. Lye DOE Course 44 22

Judging Significance of Effects a) p- values from ANOVA MSi F i = MSE Compute p-value of calculated F. IF p < α, then effect is significant. b) Comparing std. error of effect to size of effect Contrast V(effect) = V = k 1 n2 V(Contrast) = n2 k σ 2 (n2 1 k 1 ) 2 [ ] V Contrast t L. M. Lye DOE Course 45 Hence 1 V(Effect) = k (n2 se(effect) = 1 (n2 ) 1 2 k 2 n2 k σ 2 MSE ) 1 = k (n2 2 σ ) 2 If effect ± 2 (se), contains zero, then that effect is not significant. These intervals are approximately the 95% CI. e.g. 3.375 ± 1.56 (significant) 1.125 ± 1.56 (not significant) L. M. Lye DOE Course 46 23

c) Normal probability plot of effects Significant effects are those that do not fit on normal probability plot. i. e. non-significant effects will lie along the line of a normal probability plot of the effects. Good visual tool - available in Design-Expert software. L. M. Lye DOE Course 47 Design and Analysis of Multi-Factored Experiments Examples of Computer Analysis L. M. Lye DOE Course 48 24

Analysis Procedure for a Factorial Design Estimate factor effects Formulate model With replication, use full model With an unreplicated design, use normal probability plots Statistical testing (ANOVA) Refine the model Analyze residuals (graphical) Interpret results L. M. Lye DOE Course 49 Chemical Process Example A = reactant concentration, B = catalyst amount, y = recovery L. M. Lye DOE Course 50 25

Estimation of Factor Effects A = (a + ab - 1 - b)/2n = (100 + 90-60 - 80)/(2 x 3) = 8.33 B = (b + ab - 1 - a)/2n The effect estimates are: = -5.00 = 8.33, B = -5.00, AB = 1.67 C = (ab + 1 - a - b)/2n = 1.67 Design-Expert analysis A L. M. Lye DOE Course 51 Statistical Testing - ANOVA Response:Conversion ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 291.67 3 97.22 24.82 0.0002 A 208.33 1 208.33 53.19 < 0.0001 B 75.00 1 75.00 19.15 0.0024 AB 8.33 1 8.33 2.13 0.1828 Pure Error 31.33 8 3.92 Cor Total 323.00 11 Std. Dev. 1.98 R-Squared 0.9030 Mean 27.50 Adj R-Squared 0.8666 C.V. 7.20 Pred R-Squared 0.7817 PRESS 70.50 Adeq Precision 11.669 The F-test for the model source is testing the significance of the overall model; that is, is either A, B, or AB or some combination of these effects important? L. M. Lye DOE Course 52 26

Refine Model Response:Conversion ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 283.33 2 141.67 32.14 < 0.0001 A 208.33 1 208.33 47.27 < 0.0001 B 75.00 1 75.00 17.02 0.0026 Residual 39.67 9 4.41 Lack of Fit 8.33 1 8.33 2.13 0.1828 Pure Error 31.33 8 3.92 Cor Total 323.00 11 Std. Dev. 2.10 R-Squared 0.8772 Mean 27.50 Adj R-Squared 0.8499 C.V. 7.63 Pred R-Squared 0.7817 PRESS 70.52 Adeq Precision 12.702 There is now a residual sum of squares, partitioned into a lack of fit component (the AB interaction) and a pure error component L. M. Lye DOE Course 53 Regression Model for the Process Coefficient Standard 95% CI 95% CI Factor Estimate DF Error Low High VIF Intercept 27.5 1 0.60604 26.12904 28.87096 A-Concent 4.166667 1 0.60604 2.79571 5.537623 1 B-Catalyst -2.5 1 0.60604-3.87096-1.12904 1 Final Equation in Terms of Coded Factors: Conversion = 27.5 4.166667 * A -2.5 * B Final Equation in Terms of Actual Factors: Conversion = 18.33333 0.833333 * Concentration -5 * Catalyst L. M. Lye DOE Course 54 27

Residuals and Diagnostic Checking DESIGN-EXPERT Plot Conversion Normal plot of residuals DESIGN-EXPERT Plot Conversion Residuals vs. Predicted 2.16667 99 Normal % probability 95 90 80 70 50 30 20 10 5 Residuals 0.916667-0.333333-1.58333 1 2-2.83333-2.83333-1.58333-0.333333 0.916667 2.16667 20.83 24.17 27.50 30.83 34.17 Residual Predicted L. M. Lye DOE Course 55 t The Response Surface Conversion 3 3 2.00 n 23.0556 34.1667 1.75 30.8333 27.5 B: Catalyst 1.50 25.2778 27.5 29.7222 Conversion 24.1667 20.8333 1.25 31.9444 3 3 1.00 15.00 17.50 20.00 22.50 25.00 A: Concentration 2.00 1.75 1.50 B: Catalyst 1.25 1.00 25.00 22.50 20.00 17.50 A: Concentration 15.00 L. M. Lye DOE Course 56 28

An Example of a 2 3 Factorial Design A = carbonation, B = pressure, C = speed, y = fill deviation L. M. Lye DOE Course 57 Estimation of Factor Effects Term Effect SumSqr % Contribution Model Intercept Error A 3 36 46.1538 Error B 2.25 20.25 25.9615 Error C 1.75 12.25 15.7051 Error AB 0.75 2.25 2.88462 Error AC 0.25 0.25 0.320513 Error BC 0.5 1 1.28205 Error ABC 0.5 1 1.28205 Error LOF 0 Error P Error 5 6.41026 Lenth's ME 1.25382 Lenth's SME 1.88156 L. M. Lye DOE Course 58 29

ANOVA Summary Full Model Response:Fill-deviation ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 73.00 7 10.43 16.69 0.0003 A 36.00 1 36.00 57.60 < 0.0001 B 20.25 1 20.25 32.40 0.0005 C 12.25 1 12.25 19.60 0.0022 AB 2.25 1 2.25 3.60 0.0943 AC 0.25 1 0.25 0.40 0.5447 BC 1.00 1 1.00 1.60 0.2415 ABC 1.00 1 1.00 1.60 0.2415 Pure Error 5.00 8 0.63 Cor Total 78.00 15 Std. Dev. 0.79 R-Squared 0.9359 Mean 1.00 Adj R-Squared 0.8798 C.V. 79.06 Pred R-Squared 0.7436 PRESS 20.00 Adeq Precision 13.416 L. M. Lye DOE Course 59 Refine Model Remove Nonsignificant Factors Response: Fill-deviation ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 70.75 4 17.69 26.84 < 0.0001 A 36.00 1 36.00 54.62 < 0.0001 B 20.25 1 20.25 30.72 0.0002 C 12.25 1 12.25 18.59 0.0012 AB 2.25 1 2.25 3.41 0.0917 Residual 7.25 11 0.66 LOF 2.25 3 0.75 1.20 0.3700 Pure E 5.00 8 0.63 C Total 78.00 15 Std. Dev. 0.81 R-Squared 0.9071 Mean 1.00 Adj R-Squared 0.8733 C.V. 81.18 Pred R-Squared 0.8033 PRESS 15.34 Adeq Precision 15.424 L. M. Lye DOE Course 60 30

Model Coefficients Reduced Model Coefficient Standard 95% CI 95% CI Factor Estimate DF Error Low High Intercept 1.00 1 0.20 0.55 1.45 A-Carbonation 1.50 1 0.20 1.05 1.95 B-Pressure 1.13 1 0.20 0.68 1.57 C-Speed 0.88 1 0.20 0.43 1.32 AB 0.38 1 0.20-0.072 0.82 L. M. Lye DOE Course 61 Model Summary Statistics R 2 and adjusted R 2 2 SS 73.00 R = Model = = 0.9359 SS 78.00 R 2 Adj T SSE / dfe 5.00/8 = 1 = 1 = 0.8798 SS / df 78.00 /15 T T R 2 for prediction i (based on PRESS) 2 PRESS 20.00 RPred = 1 = 1 = 0.7436 SS 78.00 T L. M. Lye DOE Course 62 31

Model Summary Statistics Standard error of model coefficients 2 ˆ ˆ σ MSE 0.625 se( β) = V ( β) = = = = 0.20 k k n2 n2 2(8) Confidence interval on model coefficients ˆ β t se( ˆ β) β ˆ β + t se( ˆ β) α/2, df α/2, df E E L. M. Lye DOE Course 63 The Regression Model Final Equation in Terms of Coded Factors: Fill-deviation = +1.00 +1.50 * A +1.13 * B +0.88 * C +0.38 * A * B Final Equation in Terms of Actual Factors: Fill-deviation = +9.62500 962-2.62500 * Carbonation -1.20000 * Pressure +0.035000 * Speed +0.15000 * Carbonation * Pressure L. M. Lye DOE Course 64 32

Residual Plots are Satisfactory DESIGN-EXPERT Plot Fill-deviation Normal plot of residuals 99 95 Normal % probability 90 80 70 50 30 20 10 5 1-1.67-0.84 0.00 0.84 1.67 Studentized Residuals L. M. Lye DOE Course 65 Model Interpretation DESIGN-EXPERT Plot Fill-deviation X = A: Carbonation Y = B: Pressure B- 25.000 B+ 30.000 Actual Factor C: Speed = 225.00 Fill-deviation 6 3.75 1.5 Interaction Graph B: Pressure Moderate interaction ti between carbonation level and pressure -0.75-3 10.00 10.50 11.00 11.50 12.00 A: Carbonation L. M. Lye DOE Course 66 33

Model Interpretation DESIGN-EXPERT Plot Fill-deviation X = A: Carbonation Y = B: Pressure Z = C: Speed B: Pressure B+ -0.63 Cube Graph Fill-deviation 1.13 4.88 Cube plots are often useful visual displays of 3.13 experimental results -0.37 1.88 C+ C: Speed B- -2.13 0.12 C- A- A+ A: Carbonation L. M. Lye DOE Course 67 Contour & Response Surface Plots Speed at the High Level DESIGN-EXPERT Plot Fill-deviation X = A: Carbonation Y = B: Pressure Design Points Actual Factor C: Speed = 250.00 B: Pressure Fill-deviation 2 2 30.00 28.75 3.125 2.25 27.50 1.375 4.875 3.5625 2.25 Fill-deviation 0.9375-0.375 26.25 0.5 2 2 25.00 10.00 10.50 11.00 11.50 12.00 A: Carbonation 30.00 28.75 27.50 B: Pressure 26.25 25.00 12.00 11.50 11.00 10.50 A: Carbonation 10.00 L. M. Lye DOE Course 68 34

Design and Analysis of Multi-Factored Experiments Unreplicated Factorials L. M. Lye DOE Course 69 Unreplicated 2 k Factorial Designs These are 2 k factorial designs with one observation at each corner of the cube An unreplicated 2 k factorial design is also sometimes called a single replicate of the 2 k These designs are very widely used Risks if there is only one observation at each corner, is there a chance of unusual response observations spoiling the results? Modeling noise? L. M. Lye DOE Course 70 35

Spacing of Factor Levels in the Unreplicated 2 k Factorial Designs If the factors are spaced too closely, it increases the chances that the noise will overwhelm the signal in the data More aggressive spacing is usually best L. M. Lye DOE Course 71 Unreplicated 2 k Factorial Designs Lack of replication causes potential problems in statistical testing Replication admits an estimate of pure error (a better phrase is an internal estimate of error) With no replication, fitting the full model results in zero degrees of freedom for error Potential solutions to this problem Pooling high-order h interactions i to estimate error Normal probability plotting of effects (Daniels, 1959) L. M. Lye DOE Course 72 36

Example of an Unreplicated 2 k Design A2 4 factorial was used to investigate the effects of four factors on the filtration rate of a resin The factors are A = temperature, B = pressure, C = mole ratio, D= stirring rate Experiment was performed in a pilot plant L. M. Lye DOE Course 73 The Resin Plant Experiment L. M. Lye DOE Course 74 37

The Half-Normal Probability Plot DESIGN-EXPERT Plot Filtration Rate Half Normal plot A: Temperature B: Pressure C: Concentration D: Stirring Rate Half Norm al % probability 99 97 95 90 85 80 70 60 40 C D AC AD A 20 0 0.00 5.41 10.81 16.22 21.63 E ffe ct L. M. Lye DOE Course 75 ANOVA Summary for the Model Response:Filtration Rate ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob >F Model 5535.81 5 1107.16 56.74 < 0.0001 A 1870.56 1 1870.56 95.86 < 0.0001 C 390.06 1 390.06 19.99 0.0012 D 855.56 1 855.56 43.85 < 0.0001 AC 1314.06 1 1314.06 67.34 < 0.0001 AD 1105.56 1 1105.56 56.66 < 0.0001 Residual 195.12 10 19.51 Cor Total 5730.94 15 Std. Dev. 4.42 R-Squared 0.9660 Mean 70.06 Adj R-Squared 0.9489 C.V. 6.30 Pred R-Squared 0.9128 PRESS 499.52 Adeq Precision 20.841 L. M. Lye DOE Course 76 38

The Regression Model Final Equation in Terms of Coded Factors: Filtration Rate = +70.06250 +10.81250 * Temperature +4.93750 * Concentration +7.31250 * Stirring Rate -9.06250 * Temperature * Concentration +8.31250 * Temperature * Stirring Rate L. M. Lye DOE Course 77 Model Residuals are Satisfactory DESIGN-EXPERT Plot Filtration Rate Normal plot of residuals 99 95 Normal % probability 90 80 70 50 30 20 10 5 1-1.83-0.96-0.09 0.78 1.65 Studentized Residuals L. M. Lye DOE Course 78 39

Model Interpretation Interactions DESIGN-EXPERT Plot Filtration Rate X = A: Temperature Y = C: Concentration C- -1.000 C+ 1.000 Actual Factors B: Pressure = 0.00 D: Stirring Rate = 0.00 Filtration Rate 104 72.8851 Interaction Graph C: Concentration DESIGN-EXPERT Plot Filtration Rate X = A: Temperature Y = D: Stirring Rate 88.4426 D- -1.000 D+ 1.000 Actual Factors B: Pressure = 0.00 C: Concentration = 0.00 Filtration Rate 104 88.75 73.5 Interaction Graph D: Stirring Rate 57.3277 58.25 41.7702 43-1.00-0.50 0.00 0.50 1.00-1.00-0.50 0.00 0.50 1.00 A: Temperature A: Temperature L. M. Lye DOE Course 79 Model Interpretation Cube Plot DESIGN-EXPERT Plot Filtration Rate X = A: Temperature Y = C: Concentration Z = D: Stirring Rate Actual Factor B : P re ssu re = 0.00 C: Concentration C C+ 74.25 Cube Graph Filtration Rate 72.25 92.38 61.13 44.25 C- 46.25 69.38 D- A- A+ A: Temperature 100.63 D+ D: Stirring Rate If one factor is dropped, the unreplicated 2 4 design will project into two replicates of a 2 3 Design projection is an extremely useful property, carrying over into fractional factorials L. M. Lye DOE Course 80 40

Model Interpretation Response Surface Plots DESIGN-EXPERT Plot Filtration Rate X = A: Temperature Y = D: Stirring Rate Actual Factors B: Pressure = 0.00 C: Concentration = -1.00 1.00 0.50 Filtration Rate 90.125 83.75 DESIGN-EXPERT Plot Filtration Rate X = A: Temperature Y = D: Stirring Rate Actual Factors 100.625 B: Pressure = 0.00 C: Concentration = -1.00 86.5313 D: Stirring Rate 0.00 51.9395 56.935 64.625 71 77.375 72.4375 Filtration Rate 58.3438 44.25-0.50-1.00-1.00-0.50 0.00 0.50 1.00 A: Temperature 1.00 0.50 0.00 D: Stirring Rate -0.50-1.00 1.00 0.50 0.00-0.50 A: Temperature -1.00 With concentration at either the low or high level, high temperature and high stirring rate results in high filtration rates L. M. Lye DOE Course 81 The Drilling Experiment A = drill load, B = flow, C = speed, D = type of mud, y = advance rate of the drill L. M. Lye DOE Course 82 41

Effect Estimates - The Drilling Experiment Term Effect SumSqr % Contribution Model Intercept Error A 0.9175 3.36722 1.28072 Error B 6.4375 165.766 63.0489 Error C 3.2925 43.3622 16.4928 Error D 2.29 20.9764 7.97837 Error AB 0.59 1.3924 0.529599 Error AC 0.155 0.0961 0.0365516 Error AD 0.8375 2.80563 1.06712 Error BC 1.51 9.1204 3.46894 Error BD 1.5925 10.1442 3.85835 Error CD 0.4475 0.801025 0.30467 Error ABC 0.1625 0.105625 0.0401744 Error ABD 076 0.76 2.3104 0.87876 Error ACD 0.585 1.3689 0.520661 Error BCD 0.175 0.1225 0.0465928 Error ABCD 0.5425 1.17722 0.447757 Lenth's ME 2.27496 Lenth's SME 4.61851 L. M. Lye DOE Course 83 Half-Normal Probability Plot of Effects DESIGN-EXPERT Plot adv._rate Half Normal plot A: load B: flow C: speed e D: mud Half Normal % probability 99 97 95 90 85 80 70 60 40 BD BC D C B 20 0 0.00 1.61 3.22 4.83 6.44 Effect L. M. Lye DOE Course 84 42

ot Normal plot of residuals Residual Plots t Residuals vs. Predicted 2.58625 99 Normal % probability 95 90 80 70 50 30 20 10 5 Residuals 1.44875 0.31125-0.82625 1-1.96375-1.96375-0.82625 0.31125 1.44875 2.58625 1.69 4.70 7.70 10.71 13.71 Residual Predicted L. M. Lye DOE Course 85 Residual Plots The residual plots indicate that there are problems with the equality of variance assumption The usual approach to this problem is to employ a transformation on the response Power family transformations are widely used * y y λ Transformations are typically performed to Stabilize variance Induce normality Simplify the model = L. M. Lye DOE Course 86 43

Selecting a Transformation Empirical selection of lambda Prior (theoretical) knowledge or experience can often suggest the form of a transformation Analytical selection of lambda the Box-Cox (1964) method (simultaneously estimates the model parameters and the transformation parameter lambda) Box-Cox method implemented in Design-Expert L. M. Lye DOE Course 87 The Box-Cox Method DESIGN-EXPERT Plot adv._rate Lambda Current = 1 Best = -0.23 Low C.I. = -0.79 High C.I. = 0.32 Recommend transform: Log (Lambda = 0) Ln(ResidualSS) Box-Cox Plot for Power Transforms 6.85 A log transformation is recommended 5.40 The procedure provides a confidence interval on the transformation 3.95 parameter lambda If unity is included in the 2.50 confidence interval, no transformation would be needed 1.05-3 -2-1 0 1 2 3 Lambda L. M. Lye DOE Course 88 44

Effect Estimates Following the Log Transformation DESIGN-EXPERT Plot Ln(adv._rate) A: load B: flow C: speed D: m ud Half Norm al % probability Half Normal plot 99 Three main effects are 97 large 95 90 85 80 70 60 40 20 D C B No indication of large interaction effects What happened to the interactions? 0 0.00 0.29 0.58 0.87 1.16 Effect L. M. Lye DOE Course 89 ANOVA Following the Log Transformation Response: adv._rate Transform: Natural log Constant: 0.000 ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 7.11 3 2.37 164.82 < 0.0001 B 5.35 1 5.35 371.49 < 0.0001 C 1.34 1 1.34 93.05 < 0.0001 D 0.43 1 0.43 29.92 0.0001 Residual 0.17 12 0.014 Cor Total 7.29 15 Std. Dev. 0.12 R-Squared 0.9763 Mean 1.60 Adj R-Squared 0.9704 C.V. 7.51 Pred R-Squared 0.9579 PRESS 0.31 Adeq Precision 34.391 L. M. Lye DOE Course 90 45

Following the Log Transformation Final Equation in Terms of Coded Factors: Ln(adv._rate) = +1.60 +0.58 * B +0.29 * C +0.16 * D L. M. Lye DOE Course 91 Following the Log Transformation DESIGN-EXPERT Plot Ln(adv._rate) Normal plot of residuals DESIGN-EXPERT Plot Ln(adv._rate) Residuals vs. Predicted 0.194177 99 Normal % probability 95 90 80 70 50 30 20 10 5 Residuals 0.104087 0.0139965-0.0760939 1-0.166184-0.166184-0.0760939 0.0139965 0.104087 0.194177 0.57 1.08 1.60 2.11 2.63 Residual Predicted L. M. Lye DOE Course 92 46

The Log Advance Rate Model Is the log model better? We would generally prefer a simpler model in a transformed scale to a more complicated model in the original metric What happened to the interactions? Sometimes transformations provide insight into the underlying mechanism L. M. Lye DOE Course 93 Design and Analysis of Multi-Factored Experiments Center points L. M. Lye DOE Course 94 47

Addition of Center Points to a 2 k Designs Based on the idea of replicating some of the runs in a factorial ldesign Runs at the center provide an estimate of error and allow the experimenter to distinguish between two possible models: 0 k k k First-order model (interaction) y = β + β x + β x x + ε 0 i i ij i j i= 1 i= 1 j> i k k k k 2 i i ij i j ii i i= 1 i= 1 j> i i= 1 Second-order model y = β + β x + β x x + β x + ε L. M. Lye DOE Course 95 y F = y C no "curvature" The hypotheses are: SS Pure Quad H 0 1 k : β = 0 i= 1 k i= 1 ii H : β 0 ii nn F C( yf yc) = n + n This sum of squares has a single degree of freedom F C 2 L. M. Lye DOE Course 96 48

Example n C = 5 Usually between 3 and 6 center points will work well Design-Expert provides the analysis, including the F-test for pure quadratic curvature L. M. Lye DOE Course 97 ANOVA for Example Response: yield ANOVA for Selected Factorial Model Analysis of variance table [Partial sum of squares] Sum of Mean F Source Squares DF Square Value Prob > F Model 2.83 3 0.94 21.92 0.0060 A 2.40 1 2.40 55.87 0.0017 B 0.42 1 0.42 9.83 0.0350 AB 2.500E-003 1 2.500E-003 0.058 0.8213 Curvature 2.722E-003 1 2.722E-003 0.063 0.8137 Pure Error 0.17 4 0.043 Cor Total 3.00 8 Std. Dev. 0.21 R-Squared 0.9427 Mean 40.44 Adj R-Squared 0.8996 C.V. 0.51 Pred R-Squared N/A PRESS N/A Adeq Precision 14.234 L. M. Lye DOE Course 98 49

If curvature is significant, augment the design with axial runs to create a central composite design. The CCD is a very effective design for fitting a second-order response surface model L. M. Lye DOE Course 99 Practical Use of Center Points Use current operating conditions as the center point Check for abnormal conditions during the time the experiment was conducted Check for time trends Use center points as the first few runs when there is little or no information available about the magnitude of error Can have only 1 center point for computer experiments hence requires a different type of design L. M. Lye DOE Course 100 50