Chapter 30 Design and Analysis of

Chapter 30 Design and Analysis of 2 k DOEs Introduction This chapter describes design alternatives and analysis techniques for conducting a DOE. Tables M1 to M5 in Appendix E can be used to create test trials. 1

30.1 2-level DOE Design Alternatives It was illustrated how a saturated fractional factorial design could be created from a full factorial design. However, there are other alternatives between full and saturated fractional factorial design. The concern is how to match the factors to the interaction columns so that there is minimal confounding. Tables M1 to M5 manages this issue by providing the column selections for the practitioner, while Tables N1 to N3 shows the confounding with 2-factor interaction. Table 30.1 and Table M1 indicate test possibilities for 4, 8, 16, 32, and 64 2-level factor designs with resolution V+, V, IV, and III. 30.1 2-level DOE Design Alternatives Table 30.1 (M1) Number of 2-level Factor Considerations Possible for Various Full and Fractional Factorial Design Alternatives in Table M Number of Trials Experiment Resolution V+ V IV III 4 2 3 8 3 4 5~7 16 4 5 6~8 9~15 32 5 6 7~16 17~31 64 6 7~8 9~32 33~63 V+: Full 2-level factorial V: All main effects and 2-factor interactions are unconfounded with main effects or 2- factor interactions. IV: All main effects are un-confounded by 2-factor interactions. 2-factor interactions are confounded with each other. III: Main effects confounded with 2-factor interactions. 2

30.2 Designing a 2-level Fractional Experiment Using Tables M and N In Tables M1 to M5, the rows of the matrix define the trial configurations. The columns are used to define the 2-level states of the factors for each trial, where the level designations are + or. Step-by-step descriptions for creating an experiment design using these tables are provided in Table M1. After the number of factors, resolution, and number of trials are chosen, a design can then be determined from the tables by choosing columns from left to right using those identified by an asterisk ( ) and the numbers sequentially in the header, until the number of columns equals the number of factors in the experiment. 30.2 Designing a 2-level Fractional Experiment Using Tables M and N The contrast column numbers are then assigned sequential alphabetic characters from left to right. The numbers from the original matrix are noted and crossreferenced with Tables N1 to N3 if information is desired about 2-factor interactions and 2-factor interaction confounding. 3

30.3 Determining Statistically Significant Effects and Probability Plotting Procedure ANOVA techniques has traditionally been used to determine the significant effects in a factorial experiment. The t-test for assessing significance gives the same results as ANOVA techniques, but can be more appealing because the significance assessment is made against the magnitude of the effect. DOE techniques are often conducted with a small number of trials to save time and resources. Experimental trials are often not replicated, which leads to no knowledge about pure experimental error. One approach is to use non-significant interaction (or main effect) terms to estimate error for these significance tests. 30.3 Determining Statistically Significant Effects and Probability Plotting Procedure The formal significance test uses a probability plot of the contrast column. For 2-level factorial designs, a contrast column effect,, can be determined as, = n high x highi n high i=1 n low x lowi n low i=1 where x highi and x lowi are the response values from n trials for high and low factor-level conditions. 4

30.3 Determining Statistically Significant Effects and Probability Plotting Procedure Main effect or interaction effect is said to be statistically significant if its magnitude is large relative to other contrast column effects. When the plot position of an effect is beyond the bounds of a straight line through the non-significant contrast column effects, this effect is thought to be statistically significant. The contrast columns not found to be statistically significant can then be combined to give an estimate of experimental error for a significance test of the other factors. 30.4 Modeling Equation Format for a 2-level DOE If the situation is the lower (higher) is always better, the choice of the statistically significant factor level to use either in a confirmation or follow-up experiment may be obvious. In some situations, a mathematical model is needed for the purpose of estimating the response as a function of the factor-level considerations. For a 7-factor 2-level test, the modeling equation (without interaction terms) would initially take the form, y = b 0 + b 1 x 1 + b 2 x 2 + b 3 x 3 + b 4 x 4 + b 5 x 5 + b 6 x 6 + b 7 x 7 where y is the response and b 0 is the average of all trials, b 1 to b 7 are half of the calculated effects of factor x 1 to x 7, noting that x 1 to x 7 would take on values of -1 or +1. 5

30.4 Modeling Equation Format for a 2-level DOE This equation form assumes that factor levels have a linear relationship with the response. Center points may have been included in the basic experiment design to check this assumption. The results from 2-level experiments might lead a practitioner from considering many factors initially to considering a few factors that may need to be analyzed further using response surface techniques. Interaction terms in a model are added as the product of the factors, y = b 0 + b 1 x 1 + b 2 x 2 + b 12 x 1 x 2 30.4 Modeling Equation Format for a 2-level DOE Interaction terms in a model are added as the product of the factors, y = b 0 + b 1 x 1 + b 2 x 2 + b 12 x 1 x 2 If an interaction term is found statistically significant, the hierarchy rule states that all main factors and lower interaction terms that are part of the statistically significant interaction should be included in the model. 6

The settle-out time of a stepper motor was a critical item in the design of a document printer. The product development group proposed a change to the stepping sequence algorithm that they believed would improve the settle-out characteristics of the motor. Approach 1: To manufacture several motors and monitor their settle-out time. Confidence interval on the average settle-out time. Percent of population characteristics by probability plot. Approach 2: To perform a comparison test between the old design and the new design. Paired comparison. Approach 3: To conduct a fractional factorial experiment. Brainstorming session determined the factors to be considered: Factors ( ) Level (+) Level A: Motor temperature (mot_temp) Cold Hot B: Algorithm (algor) Current design Proposed design C: Motor adjustment (mot_adj) Low tolerance High tolerance D: External adjustment (ext_adj) Low tolerance High tolerance E: Supply voltage (sup_volt) Low tolerance High tolerance Team agreed to evaluate these five 2-level factors in a resolution V design (also called half-fraction) 7

Approach 3: (Fractional factorial experiment) Table M1 (or Table 30.1) shows that 16 test trials are needed. Number of Trials Experiment Resolution V+ V IV III 4 2 3 8 3 4 5~7 16 4 5 6~8 9~15 32 5 6 7~16 17~31 64 6 7~8 9~32 33~63 Table M3 (or Table 30.2) shows the design matrix 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 V+ * * * 4 V * * * * 5 IV * * * * * 6 7 8 III * * * * * * * * 9 10 11 12 13 14 15 1 + - - - + - - + + - + - + + + 2 + + - - - + - - + + - + - + + 3 + + + - - - + - - + + - + - + 4 + + + + - - - + - - + + - + - 5 - + + + + - - - + - - + + - + 6 + - + + + + - - - + - - + + - 7 - + - + + + + - - - + - - + + 8 + - + - + + + + - - - + - - + 9 + + - + - + + + + - - - + - - 10 - + + - + - + + + + - - - + - 11 - - + + - + - + + + + - - - + 12 + - - + + - + - + + + + - - - 13 - + - - + + - + - + + + + - - 14 - - + - - + + - + - + + + + - 15 - - - + - - + + - + - + + + + 16 - - - - - - - - - - - - - - - 8

Table 30.3 Test Design with Trial Responses A B C D E Output mot_temp algor mot_adj ext_adj sup_volt Timing 1 + - - - + 5.6 2 + + - - - 2.1 3 + + + - + 4.9 4 + + + + - 4.9 5 - + + + + 4.1 6 + - + + + 5.6 7 - + - + - 1.9 8 + - + - - 7.2 9 + + - + + 2.4 10 - + + - - 5.1 11 - - + + - 7.9 12 + - - + - 5.3 13 - + - - + 2.1 14 - - + - + 7.6 15 - - - + + 5.5 16 - - - - - 5.3 Approach 3: To conduct a fractional factorial experiment. From Table N, all contrast columns contain either a main or 2-factor interaction effect. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 *A *B *C *D AB BC CD ABD AC BD ABC BCD ABCD ACD AD CE DE AE *E BE The factors high-lighted with * and the higher order terms are also given. 9

Minitab: Stat DOE Factorial Analyze Factorial Design Graph Effects Plots: Normal Minitab: Stat DOE Factorial Analyze Factorial Design Graph Pareto 10

Analysis with all 5 factors Factorial Fit: Timing versus mot_temp, algor, mot_adj, ext_adj, sup_volt Estimated Effects and Coefficients for Timing (coded units) Term Effect Coef SE Coef T P Constant 4.844 0.1618 29.95 0.000 mot_temp -0.187-0.094 0.1618-0.58 0.575 algor -2.813-1.406 0.1618-8.69 0.000 mot_adj 2.138 1.069 0.1618 6.61 0.000 ext_adj -0.287-0.144 0.1618-0.89 0.395 sup_volt -0.238-0.119 0.1618-0.73 0.480 S = 0.647012 PRESS = 10.7168 R-Sq = 92.36% R-Sq(pred) = 80.44% R-Sq(adj) = 88.54% Analysis of Variance for Timing (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 5 50.613 50.613 10.1226 24.18 0.000 Residual Error 10 4.186 4.186 0.4186 Total 15 54.799 Unusual Observations for Timing Obs StdOrder Timing Fit SE Fit Residual St Resid 6 6 5.60000 6.96250 0.39621-1.36250-2.66R R denotes an observation with a large standardized residual. 11

Analysis with only 2 factors Factorial Fit: Timing versus algor, mot_adj Estimated Effects and Coefficients for Timing (coded units) Term Effect Coef SE Coef T P Constant 4.844 0.1532 31.61 0.000 algor -2.813-1.406 0.1532-9.18 0.000 mot_adj 2.137 1.069 0.1532 6.98 0.000 S = 0.612883 PRESS = 7.39692 R-Sq = 91.09% R-Sq(pred) = 86.50% R-Sq(adj) = 89.72% Analysis with only 2 factors Analysis of Variance for Timing (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 49.9163 49.9163 24.9581 66.44 0.000 Residual Error 13 4.8831 4.8831 0.3756 Lack of Fit 1 0.9506 0.9506 0.9506 2.90 0.114 Pure Error 12 3.9325 3.9325 0.3277 Total 15 54.7994 Unusual Observations for Timing Obs StdOrder Timing Fit SE Fit Residual St Resid 6 6 5.60000 7.31875 0.26539-1.71875-3.11R R denotes an observation with a large standardized residual. 12

Analysis with only 2 factors Estimated Coefficients for Timing using data in uncoded units Term Coef Constant 4.84375 algor -1.40625 mot_adj 1.06875 Minitab: Stat DOE Factorial Factorial Plot Main Effects 13

Minitab: Stat DOE Factorial Analyze Factorial Graph Effects Plots: Residual vs fits Minitab: Stat DOE Factorial Analyze Factorial Graph Effects Plots: Normal 14

Analysis with only 2 factors with Observation #6 Removed Factorial Fit: Timing versus algor, mot_adj Estimated Effects and Coefficients for Timing (coded units) Term Effect Coef SE Coef T P Constant 4.976 0.08364 59.49 0.000 algor -3.077-1.538 0.08364-18.39 0.000 mot_adj 2.402 1.201 0.08364 14.36 0.000 S = 0.322401 PRESS = 1.96115 R-Sq = 97.70% R-Sq(pred) = 96.38% R-Sq(adj) = 97.31% Analysis with only 2 factors with Observation #6 Removed Analysis of Variance for Timing (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 2 52.9420 52.9420 26.4710 254.67 0.000 Residual Error 12 1.2473 1.2473 0.1039 Lack of Fit 1 0.2156 0.2156 0.2156 2.30 0.158 Pure Error 11 1.0317 1.0317 0.0938 Total 14 54.1893 Estimated Coefficients for Timing using data in uncoded units Term Coef Constant 4.97596 algor -1.53846 mot_adj 1.20096 15

Minitab: Stat DOE Factorial Analyze Factorial Graph Effects Plots: Normal Minitab: Stat DOE Factorial Analyze Factorial Graph Effects Plots: Residual vs fits 16

Probability Plot with all 16 Observations Probability Plot with all 16 Observations by 2 Factors 17

algor mot_adj - - - + + - + + 30.6 DOE Alternatives This section provides 3 examples of DOE alternatives for 16 trial tests. Situation X: 5-factor 16-trial experiment Situation Y: 8-factor 16-trial experiment Situation Z: 15-factor 16-trial experiment 18

30.6 DOE Alternatives (Situation X 5 Factors) Table M3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 V+ * * * 4 V * * * * 5 IV * * * * * 6 7 8 III * * * * * * * * 9 10 11 12 13 14 15 1 + - - - + - - + + - + - + + + 2 + + - - - + - - + + - + - + + 3 + + + - - - + - - + + - + - + 4 + + + + - - - + - - + + - + - 5 - + + + + - - - + - - + + - + 6 + - + + + + - - - + - - + + - 7 - + - + + + + - - - + - - + + 8 + - + - + + + + - - - + - - + 9 + + - + - + + + + - - - + - - 10 - + + - + - + + + + - - - + - 11 - - + + - + - + + + + - - - + 12 + - - + + - + - + + + + - - - 13 - + - - + + - + - + + + + - - 14 - - + - - + + - + - + + + + - 15 - - - + - - + + - + - + + + + 16 - - - - - - - - - - - - - - - 30.6 DOE Alternatives (Situation X 5 Factors) From Table N, all contrast columns contain either a main or 2-factor interaction effect. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 *A *B *C *D AB BC CD ABD AC BD ABC BCD ABCD ACD AD CE DE AE *E BE The factors high-lighted with * and the higher order terms are also given. 19

30.6 DOE Alternatives (Situation Y 8 Factors) Table M3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 V+ * * * 4 V * * * * 5 IV * * * * * 6 7 8 III * * * * * * * * 9 10 11 12 13 14 15 1 + - - - + - - + + - + - + + + 2 + + - - - + - - + + - + - + + 3 + + + - - - + - - + + - + - + 4 + + + + - - - + - - + + - + - 5 - + + + + - - - + - - + + - + 6 + - + + + + - - - + - - + + - 7 - + - + + + + - - - + - - + + 8 + - + - + + + + - - - + - - + 9 + + - + - + + + + - - - + - - 10 - + + - + - + + + + - - - + - 11 - - + + - + - + + + + - - - + 12 + - - + + - + - + + + + - - - 13 - + - - + + - + - + + + + - - 14 - - + - - + + - + - + + + + - 15 - - - + - - + + - + - + + + + 16 - - - - - - - - - - - - - - - From Table N 30.6 DOE Alternatives (Situation Y 8 Factors) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 *A *B *C *D AB BC CD ABD AC BD ABC BCD ABCD ACD AD DE AF EF *E BF AE *F *G CE *H BE CF DG BG EG CG DF FG GH EH AH DH FH AG CH BH All contrast columns either have one main effect or 2- factor interactions. 20

30.6 DOE Alternatives (Situation Z 15 Factors) Table M3 (Situation Z) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 V+ * * * 4 V * * * * 5 IV * * * * * 6 7 8 III * * * * * * * * 9 10 11 12 13 14 15 1 + - - - + - - + + - + - + + + 2 + + - - - + - - + + - + - + + 3 + + + - - - + - - + + - + - + 4 + + + + - - - + - - + + - + - 5 - + + + + - - - + - - + + - + 6 + - + + + + - - - + - - + + - 7 - + - + + + + - - - + - - + + 8 + - + - + + + + - - - + - - + 9 + + - + - + + + + - - - + - - 10 - + + - + - + + + + - - - + - 11 - - + + - + - + + + + - - - + 12 + - - + + - + - + + + + - - - 13 - + - - + + - + - + + + + - - 14 - - + - - + + - + - + + + + - 15 - - - + - - + + - + - + + + + 16 - - - - - - - - - - - - - - - From Table N 30.6 DOE Alternatives (Situation Z 15 Factors) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 *A *B *C *D AB BC CD ABD AC BD ABC BCD ABCD ACD AD BE AE BF CG *E *F *G DE EF FG CE DF EG AG BH CI CF DG EH DH EI FJ *H *I AH AF BG CH FH GI HJ DJ AI BJ FI GJ HK AJ BK *J GH HI IJ DI EJ FK IK EK FL CK AK BL GK HL CL BI CJ DK JK KL LM GL JL KM GM DL EM IL JM IM *K *L AL EL FM GN MN HM IN LN HN AN CM DN KN DM AM *M BM CN DO HO NO AO JO MO IO FN GO EO JN EN BN *N *O BO LO KO FO CO 21

30.7 Example 30.2: A DOE Development Test A computer manufacturer determines that no trouble found (NTF) is the largest category of returns from customers. Further investigation determines that there was a heat problem in the system. The problem was design-related. The objective is to develop a strategy that identifies both the problem and risk of failure early in the product development.. The direction will be first to identify the worst-case configuration using DOE techniques, and then stress a sample of these configured machines to failure to determine the temperature guardband. 30.7 Example 30.2: A DOE Development Test Brainstorming session determined the factors to be considered: Factors ( 1) Level (+1) Level A: System type (sys_type) New Old B: Processor speed (proc-spd) Fast Slow C: Hard drive size (hd_size) Large Small D: Card (card) No card 1 card E: Memory module (m_mod) 2 extra 0 extra F: Test case (tst_case) Test case 1 Test case 2 G: Battery state (btry_st) Full charge Charging 22

30.7 Example 30.2: A DOE Development Test sys_type proc_spd hd_size card m_mod tst_case btry_st temp_proc temp_hd temp_vc 1-1 -1-1 -1-1 -1-1 76.0 58.5 72.8 2 1 1-1 -1-1 1-1 73.7 63.3 71.3 3-1 -1 1-1 -1 1 1 73.8 67.2 75.2 4 1 1 1-1 -1-1 1 74.8 58.3 73.2 5 1-1 -1 1-1 1 1 81.3 66.2 70.9 6-1 1-1 1-1 -1 1 67.0 56.1 69.1 7 1-1 1 1-1 -1-1 84.1 61.1 69.7 8-1 1 1 1-1 1-1 67.5 63.6 71.7 9 1-1 -1-1 1-1 1 79.4 58.2 65.5 10-1 1-1 -1 1 1 1 65.6 62.3 69.6 11 1-1 1-1 1 1-1 78.7 59.2 68.1 12-1 1 1-1 1-1 -1 68.6 61.3 71.5 13-1 -1-1 1 1 1-1 71.6 64.6 74.5 14 1 1-1 1 1-1 -1 73.7 56.8 69.8 15-1 -1 1 1 1-1 1 74.4 64.2 74.2 16 1 1 1 1 1 1 1 72.3 57.4 69.5 30.7 Example 30.2: A DOE Development Test 23

30.7 Example 30.2: A DOE Development Test 30.7 Example 30.2: A DOE Development Test 24

30.7 Example 30.2: A DOE Development Test Factorial Fit: temp_proc versus sys_type, proc_spd,... Estimated Effects and Coefficients for temp_proc (coded units) Term Effect Coef SE Coef T P Constant 73.906 0.2493 296.46 0.000 sys_type 6.688 3.344 0.2493 13.41 0.000 proc_spd -7.013-3.506 0.2493-14.06 0.000 hd_size 0.738 0.369 0.2493 1.48 0.177 card 0.163 0.081 0.2493 0.33 0.753 m_mod -1.738-0.869 0.2493-3.48 0.008 tst_case -1.688-0.844 0.2493-3.38 0.010 btry_st -0.663-0.331 0.2493-1.33 0.221 S = 0.997184 PRESS = 31.82 R-Sq = 98.06% R-Sq(pred) = 92.26% R-Sq(adj) = 96.37% 30.7 Example 30.2: A DOE Development Test Analysis of Variance for temp_proc (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 7 403.094 403.094 57.5849 57.91 0.000 Residual Error 8 7.955 7.955 0.9944 Total 15 411.049 Estimated Coefficients for temp_proc using data in uncoded units Term Coef Constant 73.9063 sys_type 3.34375 proc_spd -3.50625 hd_size 0.368750 card 0.081250 m_mod -0.868750 tst_case -0.843750 btry_st -0.331250 25

30.7 Example 30.2: A DOE Development Test Factorial Fit: temp_proc versus sys_type, proc_spd, m_mod, tst_case Estimated Effects and Coefficients for temp_proc (coded units) Term Effect Coef SE Coef T P Constant 73.906 0.2610 283.14 0.000 sys_type 6.688 3.344 0.2610 12.81 0.000 proc_spd -7.013-3.506 0.2610-13.43 0.000 m_mod -1.738-0.869 0.2610-3.33 0.007 tst_case -1.688-0.844 0.2610-3.23 0.008 S = 1.04411 PRESS = 25.3712 R-Sq = 97.08% R-Sq(pred) = 93.83% R-Sq(adj) = 96.02% Analysis of Variance for temp_proc (coded units) Source DF Seq SS Adj SS Adj MS F P Main Effects 4 399.06 399.06 99.764 91.51 0.000 Residual Error 11 11.99 11.99 1.090 Total 15 411.05 30.7 Example 30.2: A DOE Development Test Unusual Observations for temp_proc Obs StdOrder temp_proc Fit SE Fit Residual St Resid 6 6 67.0000 68.7687 0.5837-1.7687-2.04R R denotes an observation with a large standardized residual. Estimated Coefficients for temp_proc using data in uncoded units Term Coef Constant 73.9063 sys_type 3.34375 proc_spd -3.50625 m_mod -0.868750 tst_case -0.843750 26

30.7 Example 30.2: A DOE Development Test The worst-case levels and temperatures are: Constant 73.9063 sys_type 3.34375 proc_spd -3.50625 m_mod -0.868750 tst_case -0.843750 Factors Level Contribution Constant 73.9063 A: System type (sys_type) 1 (Old) 3.34375 B: Processor speed (proc-spd) -1 (Fast) 3.50625 E: Memory module (m_mod) -1 (2 extra) 0.868750 F: Test case (tst_case) -1 (Test case 1) 0.843750 82.4688 27