Design of Experiments

6s ANALYZE 6s IMPROVE Chapter 2-4 Design of Experiments 2-4-1

Definition Design of Experiments (DOE) DOE is a systematic approach to investigation of a system or process. A series of structured tests are designed in which planned changes are made to the input variables of a process or system. The effects of these changes on a pre-defined output are then assessed. 2-4-2

Experiments The term experiment is defined as the systematic procedure carried out under controlled conditions in order to discover an unknown effect, to test or establish a hypothesis, or to illustrate a known effect. When analyzing a process, experiments are often used to evaluate which process inputs have a significant impact on the process output, and what the target level of those inputs should be to achieve a desired result (output). 2-4-3

Hypothesis Sometimes when we test hypotheses we have more than two samples. Three different lengths of heat treating are evaluated for hardness. Is one length better than the others? To test this using conventional methods would involve several null hypotheses: H H H 01 02 03 : x : x : x 1 1 2 x x x 2 3 3 For large number of alternatives this would be a tedious process. 2-4-4

Analysis of Variance Analysis of Variance is a widely used statistical technique that partitions the total variability in our data into components of variability that are used to test hypotheses. In One-way ANOVA, we wish to test the hypothesis: against: H 0 : 1 = 2 = = k H 1 : Not all population means are the same

Analysis of Variance In ANOVA, we compare the between-group variation with the within-group variation to assess whether there is a difference in the population means. Thus by comparing these two measures of variance (spread) with one another, we are able to detect if there are true differences among the underlying group population means.

ANOVA This technique is one of the most powerful of statistical methods. It provides the basis for determining whether several sample means differ significantly. The methodology is best introduced via an example. 2-4-7

Example One Way ANOVA Ten service stations are rated in terms of overall quality and effectiveness. More than 20 factors are considered in computing the rating score. A sample of five ratings is made for each station and the results are shown below. The distributor wishes to determine if the stations differ significantly in the rating score or whether the variation in average scores can be attributed to chance (common cause variation.) Use a 95% confidence level. Adapted from Statistical Analysis for Administrative Decisions, by Clark and Schkade, 1974 2-4-8

Data for Example Service Station 1 2 3 4 5 6 7 8 9 10 Totals 99 70 90 99 65 85 75 70 85 92 830 96 65 80 95 70 88 70 51 84 91 790 95 60 48 87 48 75 71 93 80 93 750 98 65 70 95 67 82 73 94 86 90 820 97 65 62 99 60 80 76 92 90 89 810 Total 485 325 350 475 310 410 365 400 425 455 4000 Mean 97 65 70 95 62 82 73 80 85 91 800 Data Set 2-4-10A 2-4-9

Example Continued In this example the observations of quality ratings are grouped so that each service station may be viewed as a class. The classes for grouping or classifying data are sometimes called treatments. The hypothesis that is tested is as follows: 2-4-10

One Way ANOVA Table The results are typically expressed in what is called an ANOVA table. 2-4-11

One Way ANOVA Calculations We perform the calculations of the SS (sums of squares) as follows: The ANOVA table is on the next page. r is number of rows and c number of columns 2-4-12

ANOVA Table Since the F Ratio (F test ) is greater than the F table value we reject the null hypothesis and state that there is a difference in treatments. 2-4-13

ANOVA Practice Problem Three specimens of each of five different metals were immersed in a highly corrosive solution and their corrosion rates were measured with the following results shown on the next page. At the.01 level is there a significant difference between the corrosion rates of the metals? 2-4-14

Data Metal Corrosion Rates Aluminum 0.5 0.4 0.6 Stainless 0.6 0.7 0.6 Carbon Steel 0.5 0.7 1.3 Enamel Coated Steel 0.8 0.6 0.8 Nickel Alloy 4.1 3.5 3.1 Data Set 2-4-16 2-4-15

ANOVA Table C = 23.563 SStr = 20.251 SST = 21.157 SSE = 0.9067 Source of Variation SS df MS F P-value F crit Metals 20.25 4 5.063 55.84 8E-07 5.994 Error 0.907 10 0.091 Total 21.16 14 2-4-16

Excel 2-4-17

Excel 2 2-4-18

Excel 3 2-4-19

Practice Problem Random samples of 4 brands of tires required the following braking distances while going 30 mph. Is there a significant difference at the.1 level? Brand A Brand B Brand C Brand D 27 25 27 26 30 20 31 26 25 22 30 25 26 21 32 23 2-4-20

Two Way ANOVA Often it is desirable to test hypotheses concerning two variables. These two variables may be referred to as row effects and column effects. A similar method is used. 2-4-21

Two Way ANOVA H H H H 0 1 0 1 : No difference in Rows : Difference in Rows : No difference in Columns : Difference in Columns 2-4-22

Calculation Factors 2-4-23

Example Problem Let us use the example of providing quality ratings for five service stations. One question that may be asked is if there is a difference between ratings of the different service stations. A second question that could be asked, is, if there were multiple raters, is there a significant difference between raters at the.05 level. If the data were set up with rows being raters and columns being ratings we could test the following hypotheses: 2-4-24

Hypotheses H H H H 0 1 0 1 : No difference in Rows : Difference in Rows : No difference in Columns : Difference in Columns 2-4-25

Data Data Set 2-4-10B 2-4-26

Solution 2-4-27

Practice Problem Let us look at the stopping data again, but this time note that we used four different cars. Can we say, with 95% confidence, that there is no difference between tire brands as well as no difference between vehicle types? Vehicle Brand A Brand B Brand C Brand D Fusion 27 25 27 26 Camry 30 20 31 26 Charger 25 22 30 25 C-240 26 21 32 23 Data Set 2-4-29 2-4-28

Hypotheses H H H H 0 1 0 1 : No difference in Vehicles : Difference in Vehicles : No difference in Brands : Difference in Brands We will first do it manually then use Excel. 2-4-29

2-4-30

2-4-31

Two Way ANOVA Replications In testing hypotheses concerning two variables, two possible models may be considered. First, in the previous model in which there was only one observation per cell, the column and row effects were assumed to be independent and additive. That is to say the service station was independent of the rater. It was assumed that the rater exercised no bias either for or against a particular station but made a completely objective rating based on what she found at the station. 2-4-32

Two Way ANOVA Replications However, it is possible that there is not complete independence and the variables may interact with or on one another. In this example, the rating of service stations by raters, each rater might rate each service station several times, making it possible to test for any interaction between raters and stations. In this set up if the stations were columns and the raters rows, we might have multiple measurements within each cell. In this instance we would have the following hypotheses to test: 2-4-33

Two Way ANOVA Replications 2-4-34

Computations and Example 2-4-35

Example Data 2-4-36

ANOVA Table We reject all three null hypotheses. 2-4-37

Practice Problem To find the best arrangement of instruments on a control panel, four different displays were tested by simulating an emergency condition and observing the reaction time required to correct this condition. The experiment was conducted three times. At the.01 level is there a significant difference between 1. Subjects 2. Displays 3. Interaction 2-4-38

Hypotheses 2-4-39

Data Subject A Subject B Subject C Display 1 9 17 14 16 16 18 16 15 16 Display 2 19 13 16 14 15 10 12 14 15 Display 3 14 11 14 Data Set 2-4-42 14 17 16 16 16 12 Display 4 15 16 12 15 13 13 14 13 14 2-4-40

Excel Application 2-4-41

No significant differences. 2-4-42

Latin Squares When the experimenter wishes to study the effects of three variables on the basis of relatively few observations he may turn to the use of the Latin Square Design. The experiment is performed by arranging the levels of one factor which are denoted by the letters A, B, C, etc. into an array so that every letter appears once and only once in every row and column. 2-4-43

Latin Square One Latin Square with for levels for one variable might look like this: B A C D D B A C C D B A A C D B 2-4-44

Latin Squares For example, the study group engaged in rating service stations might be concerned with another variable, the time of day when the rating was made. The group may feel that during very busy times of the day the service station operator is not able to give as good service as at other times and that this might influence the rating of the station. The letters A, B, C, and D might be used to refer to four different levels of business activity. These different levels are commonly called treatments. The size of the Latin square is determined by the number of treatments. 2-4-45

Hypotheses 2-4-46

LS ANOVA Table 2-4-47

Computations 2-4-48

Example We will work this on manually. Swap the C and D to agree with the table on page 2-4-44. 2-4-49

Solution No significant difference for rows. Significant difference for columns and treatments. 2-4-50

DOE Experimental design can be used at the point of greatest leverage to reduce design costs by speeding up the design process, reducing late engineering design changes, and reducing product material and labor complexity. Designed Experiments are also powerful tools to achieve process cost savings by minimizing process variation and reducing rework, scrap, and the need for inspection. 2-4-51

Review: A process is In (Statistical) Control if it is Stable or Predictable An In Control Process 103 93 83 73 63 53 43 33 23 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 2-4-52

But It Is Not Producing The Results That I Want! An In Control Process 103 93 83 73 63 53 43 33 23 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 2-4-53

If That s Not What You Want!! Then. CHANGE The PROCESS! (Using..) 2-4-54

Experimental Design (DOE) Uses Scientific Methods to Find Out: What Helps (25%) What Hurts (25%) What Has No Effect (50%) 2-4-55

Wrong Ways 1. One thing at a time Takes too long May Sub-Optimize Misses Interactions 2-4-56

Wrong Ways 2. Everything at Once Which Input Change Effected The Output Change? Pluses May Be Canceled by Minuses? 2-4-57

Using Regression in Six Sigma How Is Regression Used Here? The State Agricultural Extension Service has hired you, as a Six Sigma Black Belt to help improve a process. They desire to help soybean farmers increase the yield (in bushels per acre) from their fields based on certain easily measurable and controllable factors. We know that fertilizer sells for $4 per 100 pounds. Water costs $.18 per gallon. Lime costs $1.75 per 100 pounds. Soybeans sell for $1.42 a bushel. What would be the best combination of fertilizer, water, and lime to maximize the profit for the farmer? Use the data collected on the following page. 2-4-58

Is it one thing at a time? everything at once?...a systematic pre-arrangement of combinations? Bushels per Acre Pounds Fertilizer Gallons Water Pounds Lime Bushels per Acre Pounds Fertilizer Gallons Water Pounds Lime 21.0 500 50 200 40.2 900 100 600 20.0 500 50 200 40.7 900 100 600 21.0 500 50 200 38.3 900 100 600 24.0 600 50 300 43.5 1000 100 700 22.4 600 50 300 42.9 1000 100 700 29.4 600 50 300 43.8 1000 100 700 26.1 700 75 400 50.1 1200 125 800 26.0 700 75 400 50.6 1200 125 800 27.4 700 75 400 50.5 1200 125 800 32.1 800 75 500 39.9 1500 125 900 32.2 800 75 500 46.8 1500 125 900 32.4 800 75 500 43.3 1500 125 900 2-4-59

Using Regression in Six Sigma One thing at a time? No- Fertilizer, water and lime are all factors Everything at once? Yes- Fertilizer, water and lime are all factors A systematic pre-arrangement of combinations? Not really- Fertilizer, water and lime are haphazardly set To be systematic, choose and keep to levels, e.g., Fertilizer: 500 and 1500; Water: 50 and 100; Lime: 200 and 600 2-4-60

Using Regression and Correlation Studies in Six Sigma Useful for analyzing existing data Priority application in the Define and Measure stages, e.g., Define: Select the factors with the greatest effects on the process Measure: Identify how the greatest factors impact the process, e.g., use to develop Y=f(X1, X2, ) in SIPOC Not necessary to be systematic in pre-arranging the factors, but it is better if so 2-4-61

All Good DOEs Require the collection of data through measuring responses Involve systematic manipulation of variables or factors of interest in pre-arranged ways (called experiment designs) Require control or accounting for all other variables or factors Therefore, not all opportunities to collect or use data are good DOE s, but all DOE s are opportunities to collect and use data 2-4-62

The Right Way Design of Experiments (DOE) Test Everything in pre-arranged combinations Measure the Effects of Each and Interactions 2-4-63

Factorial Design A factorial experiment is the name commonly given to an experiment in which several factors are investigated at each of two or more levels. The information obtained by looking at experimental variables used simultaneously is more complete than that obtained from a series of single factor experiments. In factorial experiments an effect attributable to the combination of variables can be measured when this would not be possible if the variables were considered singly. 2-4-64

Example-Improve Green Belt Exam Scores Brainstorming Session Results Work More Problems (Factor A) Use A Different Text (Factor B) Study in Morning or at Night (Factor C) But What Really Affects The Scores? Run Some Experiments!!! 2-4-65

Define Factors Design Considerations Problems Fewer = - More = + Design an Experiment to Include all of the Possibilities Text Present = - New = + Time Morning = + Evening = - Factors Trial (Run) Problems Text Time 1 - - - 2 - - + 3 - + - 4 - + + 5 + - - 6 + - + 7 + + - 8 + + + Run the Experiment 2-4-66

Example-Improve Green Belt Exam Scores Factors Results A B C 1 - - - 76 2 - - + 65 3 - + - 65 4 - + + 72 5 + - - 93 6 + - + 91 7 + + - 86 8 + + + 92 Sum + 362 315 320 Sum - 278 325 320 Diff 84-10 0 Avg 21-2.5 0 80 Coeff 10.5-1.25 0 Helps Hurts No Effect 2-4-67

Example-Improve Green Belt Exam Scores Prediction Formula: OR OR Yˆ 80 10.5A 1.25B 0C Estimated Test Score 80 10.5A 1.25B 0C Estimated Test Score 80 10.5A 1.25B 0C Note: Factor impacts are referenced to the mean of 80. 2-4-68

Benefits of Experimental Design Identifies Beneficial Changes - Quickly - Inexpensively Avoids Harmful Changes Identifies Interactions 2-4-69

An example of Interaction Univ of Western Ontario Research Grapefruit Juice reacts with drugs that are largely broken down in the small intestine before they get to blood stream Result: People who take these drugs with grapefruit juice get a higher dosage than their doctors prescribed Reportedly as much as 3 times prescribed dosage! 2-4-70

Detailed DOE Course Content History of DOE and Fundamental Terminology Contingency Table Analysis One Factor Analysis- Control versus Experimental Comparisons One Way ANOVA Two Factor Analysis- Two Way ANOVA Two Way ANOVA with Replication Three or More Factors- Latin Square for Three Factors Design of Experiments or DOE (Multivariate Testing) Intent Is to Gain Understanding and Control of Process Two or More Factors Use Scientific Investigation Method Controlled Data Collection Designed Experimental Sequence All Other Factors Assumed to Be Held Constant Regression Based Analysis Prediction Equation 2-4-71

History of DOE Sir Ronald Fisher in 1920 s Agricultural origins Long use in food and drug research Strategic competitive weapon Statistical basis 2-4-72

Design of Experiments Introductive Definitions Selected terms Factor Block Treatment Level Response Replication I II III 3 8 1 6 2 9 4 7 5 2-4-73

DOE Terms Factor: An independent variable that may affect a response. Block: A factor used to account for variables that the experimenter wishes to avoid or separate out during analysis. Treatment: Factor levels to be examined during experimentation. Levels: Given treatment or setting for an input factor. Response: The result of a single run of an experiment at a given setting (or given combination of settings when more than one factor is involved). Replication (Replicate): Repeated run(s) of an experiment at a given setting (or given combination of settings when more than one factor is involved). 2-4-74

Contingency Table Analysis Deal with two qualitative or categorical factors Data are counts in a table with one factor in the rows and the other in the columns One pre-factor and the other an outcome, e.g., training level versus success or no success Association of two factors, e.g., time-of-day versus choice of self-, chat-, or phone-assistance TOH can be done with the Chi-Square distribution, which tests for null hypothesis of independence between the two factors 2-4-75

Contingency Table Analysis Pre-Factor and Outcome Factor Pre-Factor: Training or no training Outcome Factor: Task Success Use c 2 {alpha,(r-1)(c-1)} where df=(rows-1)(columns-1) The contribution to c 2 from SPC-XL helps indicate where major dependencies lie Note: These data are fictitious Untrained Trained Successful 143 281 Not Successful 128 38 2-4-76

Hypotheses Null: There is no difference Alternative: There is a significant difference 2-4-77

Use SPC XL, Analysis Tools, Independence Test Matrix 2-4-78

SPCXL 2-4-79

Result The p value tells the story. In this case training makes a difference. 2-4-80

Contingency Table- Independent and Outcome Factor 5-Step Hypothesis Test Hypothesis: = 0.05 H Independent H : 0 : 1 Dependent Test statistic: c 2 = 90.40 Rejection Region: c 2 table= c 2 {alpha,(r-1)(c-1)} = c2 {.05,1} = 3.84 Interpretation: Reject H 0 Since c 2 test = 90.40 > c 2 crit = c 2 table = 3.84 Conclusion: Success significantly depends on training level at 95% confidence 2-4-81

Using Contingency Table Analysis in Six Sigma Useful for analyzing whether two factors are independent or associated based on cross-tabulated counts. Priority application in the Measure or Improve stages, e.g., Measure: Test association between two categorical factors, e.g., experience-novice versus success-failure. Improve: Determine whether an improvement is statistically significant using cross-tabulated field counts Must use systematic arrangement of the factors into a cross-table of counts. 2-4-82

One Factor Analysis- Matched comparison Treatments only All yields are in columns Also known as: Pre-test vs. post-test Control vs. experimental comparison within subjects Paired or correlated observations Balanced (same number of responses for each treatment) Assumption: Two responses come from same individual and the difference is approximately Normally distributed. Samples of individuals are random and independent. 2-4-83

1 Factor Analysis- Matched Comparison 1) Pre-test 2) Train 3) Post-test (20 Questions) Pre-Test Post-Test Difference 17 17 0 12 19 7 13 16 3 16 18 2 7 19 12 16 20 4 14 19 5 15 18 3 13 17 4 20 16-4 14.3 17.9 3.6 Mean 3.47 1.37 4.2 std dev 10 10 10 n 2-4-84

1 Factor Analysis- Matched Comparison t x post x pre s difference / n df n 1 2-4-85

1 Factor Analysis- Matched Comparison 5- Step Hypothesis Test Hypothesis: = 0.05 H o difference Test statistic: t = 2.71 = : x 0 H : xdifference 0 (17.9 14.3) / (4.195 * sqrt(10)) Rejection Region: t table = t {alpha,n-1} = t {.05,9} = 1.83 a Interpretation: Reject H 0 Since t test = 2.71 > t crit = t table = 1.83 Conclusion: Post-test performance is significantly greater than pre-test performance at 95% confidence 2-4-86

Using 1 Factor Analysis- Matched Comparison in Six Sigma Useful for analyzing whether training or a new procedure improves time or accuracy of personnel Priority application in the Improve stage, e.g., Improve: Select a response, e.g., time or accuracy, to determine whether an improvement is statistically significant using a pre-test and post-test experimental design on a sample of personnel with a 1-tailed test of hypothesis Must use systematic pre-test versus post-test arrangement of the factors and then sample the difference scores with a 1-tailed test 2-4-87

One Factor Analysis- Control Versus Experimental Comparison Treatments only All yields are in columns Also known as: Comparing two sample means with known sample standard deviations Balanced versus unbalanced Assumption: Treatment population variances equal and approximately Normally distributed. Samples are random and independent. 2-4-88

1 Factor Analysis- Control Vs. Experimental Help-Desk Assistance Time With and Without Script Control (Without Script) Experimental (With Script) 1.32 0.57 1.74 0.92 1.44 1.03 1.97 1.11 2.11 0.76 1.67 0.88 1.55 0.39 1.32 1.21 0.77 0.62 1.64 0.826 Mean 0.291 0.255 s 8 10 n 2-4-89

1 Factor Analysis- Control Vs. Experimental 2 2 1) ( 1) ( 1 1 2 2 b a b a b b a a b a b a n n df n n s n s n n n x x t 2-4-90

1 Factor Analysis- Control Vs. Experimental 5-Step Hypothesis Test Hypothesis: = 0.05 H Test statistic: t = 6.32, df=16 o : x1 x2 Ha : x1 x2 Rejection Region: t table = t {alpha,n1+n2-2} = t {.05,16} = 1.745 Interpretation: Reject H 0 Since t test = 6.32 > t crit = t table = 1.745 Conclusion: Assistance time is significantly greater without a script to guide the help-desk than with the script at 95% confidence 2-4-91

Using 1 Factor Analysis- Control Vs. Experimental - in Six Sigma Useful for analyzing whether two means are the same or different. Priority application in the Measure, Analyze or Improve stages, e.g., Measure and Analyze: Compare two means, e.g., men vs. women or experience vs. novice Improve: Determine whether an improvement is statistically significant using a comparison with and without the improvement Must use systematic control versus experimental arrangement of the factors 2-4-92

Using One-Way ANOVA in Six Sigma (Calculations covered earlier) Useful for analyzing whether two or more means are all the same or some different. Priority application in the Measure, Analyze or Improve stages, e.g., Measure and Analyze: Compare multiple means, e.g., four levels of fertilizer or five days of week Improve: Determine whether multiple improvements are statistically significant Must use systematic arrangement of the factors to manipulate one factor and control, counterbalance or randomize the remaining factors 2-4-93

Linear Model for Randomized Design Alternative to ANOVA Analysis of variance produced through regression techniques Consider the single factor dyes example with three treatments from earlier Linear regression model of yields OR 3 3 2 2 1 1 0 ˆ x x x Y 3 3 2 2 1 1 0 ˆ x x x Y 2-4-94

Linear Model Coding Techniques For Treatments x 1 1 0 if y value otherwise is from treatment I x 2 1 0 if y value otherwise is from treatment II x 3 1 0 if y value otherwise is from treatment III 2-4-95

Single Factor Exercise Exercise: Three dye formulas for a certain synthetic fiber were under consideration. The data in the following table was collected. At 0.01 significance level is there a difference between the dyes? (Assume the higher the value the better the performance.) Dyes A B C 4 11 25 8 16 29 6 20 20 12 23 17 7 3 13 2-4-96

Single Factor Exercise Dyes Exercise: Three dye formulas for a certain synthetic fiber were under consideration. The data in the following table was collected. At 0.01 significance level is there a difference between the dyes? (Assume the higher the value the better the performance.) A B C 4 11 25 8 16 29 6 20 20 12 23 17 7 Avg = 7.4 3 Avg = 14.6 13 Avg = 20.8 2-4-97

Model of Yield Matrix y x1 x2 x3 4 1 0 0 8 1 0 0 6 1 0 0 12 1 0 0 7 1 0 0 11 0 1 0 16 0 1 0 20 0 1 0 23 0 1 0 3 0 1 0 25 0 0 1 29 0 0 1 20 0 0 1 17 0 0 1 13 0 0 1 2-4-98

Regression Setup REMEMBER from earlier, we would like to have the formula referenced to the mean, SO transform the data to y - ybar 3 3 2 2 1 1 0 ˆ x x x Y 3 3 2 2 1 1 0 ˆ x x x Y 2-4-99

Regression Setup Create a new variable y y y Now regress the 3 dummy variables against This compares each to y y 2-4-100

Regression Table y ybar y-ybar x1 x2 x3 4 14.26667-10.2667 1 0 0 8 14.26667-6.26667 1 0 0 6 14.26667-8.26667 1 0 0 12 14.26667-2.26667 1 0 0 7 14.26667-7.26667 1 0 0 11 14.26667-3.26667 0 1 0 16 14.26667 1.733333 0 1 0 20 14.26667 5.733333 0 1 0 23 14.26667 8.733333 0 1 0 3 14.26667-11.2667 0 1 0 25 14.26667 10.73333 0 0 1 29 14.26667 14.73333 0 0 1 20 14.26667 5.733333 0 0 1 17 14.26667 2.733333 0 0 1 13 14.26667-1.26667 0 0 1 2-4-101

Regression Setup Use all 3 dummy independent variables. Use the transformed y-ybar as the dependent variable. Force the constant to be 0. 2-4-102

Regression Setup 2-4-103

Regression Output 2-4-104

Regression Equation Noting that x 2 is likely insignificant yˆ y 6.867x 1 6.533x 3 yˆ 14.267 6.867x 1 6.533x 3 yˆ 14.267 6.867x 1 6.533x 3 2-4-105

Regression Equation yˆ 14.267 6.867x 1 6.533x 3 Let x 1 1, x 2 0, x 3 0 : yˆ 14.267 6.867 7.4 Let x 1 0, x 2 1, x 3 0 : yˆ 14.267 Let x 1 0, x 2 0, x 3 1: yˆ 14.267 6.533 20.8 2-4-106

Comparison of EXCEL Outputs ANOVA vs. Regression What sections are the same? What s different? Why use regression instead of plain ANOVA? The regression equation is important Allows predictions Regression analysis shows where differences lie, not just that the treatments are different Coding and regression are the concepts used in factorial designs 2-4-107

Using One-Factor Experiment Using Regression in Six Sigma Useful for analyzing whether two or more means are all the same or some are different and obtaining a model Priority application in the Measure, Analyze or Improve stages, e.g., Measure and Analyze: Compare strength of multiple factors and model their effects, e.g., four levels of fertilizer or five days of week Improve: Determine whether multiple improvements are statistically significant and also predict the improvements Must use systematic arrangement of the factors to manipulate one factor and control, counterbalance or randomize the remaining factors and a dummy coding of factors in the regression analysis (our software will do this for us) 2-4-108

Using Two-Way ANOVA in Six Sigma (Calculations covered earlier) Useful for analyzing whether two factors are significant or not. Priority application in the Measure, Analyze or Improve stages, e.g., Measure and Analyze: Compare two factors, e.g., four levels of fertilizer and five levels of lime Improve: Determine whether multiple improvements developed through changing two factors are statistically significant Must use systematic arrangement of the factors to manipulate two factors and control, counterbalance or randomize the remaining factors 2-4-109

How Is a Third Factor Added? Two Way ANOVA with interaction Also known as replication Latin Squares Factorial Designs 2 or more factors 2-4-110

Using Two-Way ANOVA With Replication in Six Sigma (Calculations covered earlier) Useful for analyzing whether two factors and their interaction are significant. Priority application in the Measure, Analyze or Improve stages, e.g., Measure and Analyze: Investigate effects of two factors, e.g., experience of customer representative and complexity of call Improve: Determine whether two improvements and their interaction are statistically significant Must use systematic arrangement of the factors to manipulate two factors and control, counterbalance or randomize the remaining factors all while replicating the experiment 2-4-111

Using Three-Factor ANOVA with Latin Squares in Six Sigma Useful for efficiently analyzing whether three factors have main effects Priority application in the Measure or Analyze stages, e.g., Measure and Analyze: Compare multiple means, e.g., four levels of fertilizer, four levels of water and four levels of lime Must use systematic arrangement of the three factors to manipulate all three factors while controlling the remaining factors Can not analyze interaction with any variant of Latin Square design 2-4-112

Factorial Designs Framework Deal simultaneously with qualitative and quantitative factors Change individual factors while observing combinations of others Allows blocking and randomization Allows replication 2-4-113

Factorial Designs Framework Allows linear mathematical models Coding based on a binomial counting scheme Easy to set up and carry out Simple to analyze and interpret Simple to explain to others 2-4-114

Two-level Factorial Designs Full factorial design (2 k ) k is the number of factors and can be 2 or greater Each factor has two levels Responses at every level Random order of runs Yields listed in binomial order but yield data is actually captured randomly 2-4-115

Two-level Factorial Designs Things Not studied in this course Extensive study of fractional factorial designs Only one level for any factor(s) Incomplete Designs (Missing responses) 2-4-116

Two-level Factorial Designs for Two Factors 2 2 design for two factors at two levels each x2 High Conceptually has a total of four responses One for each factor at each level Low Low High x1 2-4-117

Two-level Factorial Designs for Three Factors 2 3 design for three factors at two levels each Conceptually has a total of eight responses One for each factor at each level x2 x3 x1 2-4-118

2 2 Full Factorial Design Statistical Analysis Runs Factor A Factor B Yield 1 low low 55 2 high low 70 3 low high 65 4 high high 45 2-4-119

2 2 Full Factorial Example Factor A Average Yields Factor A low settings average = 60 Average of 55 and 65 Factor A high settings average = 57.5 Average of 70 and 45 Factor A effect is 2.5 Low to high average difference Trial Factor A Factor B Yield 1 Low Low 55 2 High Low 70 3 Low High 65 4 High High 45 2-4-120

2 2 Full Factorial Example Factor B Average Yields Factor B low settings average = 62.5 Average of 55 and 70 Factor B high settings average = 55 Average of 65 and 45 Factor B effect is 7.5 Low to high average difference Trial Factor A Factor B Yield 1 Low Low 55 2 High Low 70 3 Low High 65 4 High High 45 2-4-121

Yield Yield 2 2 Full Factorial Example Factor Effects Factor A Effect Factor B Effect 64 62 60 58 56 54 60-2.5 57.5 64 62 60 58 56 54 62.5-7.5 55 Low Settings High Low Settings High 2-4-122

2 2 Full Factorial Example Test of Interaction Factors A and B Show Interaction Factors Show Interaction 70 65 60 55 50 45 65 55 70 45 B Low B High 70 65 60 55 50 45 70 55 65 45 A Low A High Lo Factor A Hi Factor B Settings Only need one chart or the other but not both 2-4-123

2 2 Full Factorial Design Replication Example Run Factor A Factor B Yield 1 Yield 2 Average s 2 1 low low 55 56 55.5 0.5 2 high low 70 69 69.5 0.5 3 low high 65 71 68.0 18.0 4 high high 45 47 46.0 2.0 2-4-124

2 2 Full Factorial Factor Effects Factor A Effect Factor B Effect 63 62 61 60 59 58 57 56 61.75 57.75 63 62 61 60 59 58 57 56 62.5 57.0 Lo Settings Hi Lo Settings Hi 2-4-125

2 2 Full Factorial Test of Interaction Factor A Interaction 70.0 68.0 69.5 65.0 60.0 55.0 55.5 50.0 45.0 46.0 Lo Factor A Hi B Low B High Each of these charted lines represents conditions with B fixed 2-4-126

2 2 Full Factorial Exercise Your turn. An experiment was replicated three times. The data collected is shown below. Assume that a lower the output response is better. What is the optimum setting for each factor and what do the effects, and interaction look like? Calculate by hand. Run Factor A Factor B Yield 1 Yield 2 Yield 3 Average s 2 1 low low 22 32 28 2 high low 14 19 18 3 low high 30 24 26 4 high high 23 23 14 2-4-127

Answer to 2 2 Full Factorial Exercise Run Factor A Factor B Yield 1 Yield 2 Yield 3 Average s 2 1 low low 22 32 28 27.333 25.333 2 high low 14 19 18 17.000 7.000 3 low high 30 24 26 26.670 9.330 4 high high 23 23 14 20.000 27.000 2-4-128

Answer to 2 2 Full Factorial Exercise Factor Effects Factor A Effect Factor B Effect 27 24 27 27 24 23.333 21 21 22.167 18 18.5 Low Settings Hi 18 Low Settings Hi 2-4-129

Answer to 2 2 Full Factorial Exercise Test of Interaction 27 25 23 21 19 17 27.333 26.667 Factor A Interaction Low Settings High 20 B Low B High 17.000 2-4-130

2 2 Full Factorial Design 2 Factor 2 Level Coding A two level, two-factor experiment was performed. In this example the yield is the measured outcome of the experiment. For this example, the larger the yield is the better the results. Runs Factor A Factor B Yield Codes 1 low low 55-1 -1 2 high low 70 +1-1 3 low high 65-1 +1 4 high high 45 +1 +1 2-4-131

2 2 Full Factorial Design Interaction Added and Coding Runs A B AB Yield A B AB 1-1 -1 1 55-55 -55 55 2 1-1 -1 70 70-70 -70 3-1 1-1 65-65 65-65 4 1 1 1 45 45 45 45 Contrast 235-5 -15-35 Effect -2.5-7.5-17.5 Coeff. 58.75-1.25-3.75-8.75 2-4-132

Factorial Yield Prediction Equation Generalize d yˆ 0 1 x 1 2 x 2 From problem: yˆ Predictingyieldat FactorsA low (-1)and B low (-1) yˆ Form: 58.751.25( 1) 3 x 1 x 2 5875. -125. A-375. B-8. 75AB 3.75( 1) 8.5( 1) 55.0 2-4-133

2 2 Full Factorial Design With Replication Using DOEKISS or DOEPRO Run Factor Factor Yield 1 Yield 2 Average s2 A B 1 low low 55 56 55.5 0.5 2 high low 70 69 69.5 0.5 3 low high 65 71 68 18 4 high high 45 47 46 2 2-4-134

2 2 Full Factorial Design Replication, Interaction, Coding Run A B AB Y1 Y2 Mean s 2 A B AB 1-1 -1 1 55 56 55.5 0.5-55.5-55.5 55.5 2 1-1 -1 70 69 69.5 0.5 69.5-69.5-69.5 3-1 1-1 65 71 68 18.0-68 68-68 4 1 1 1 45 47 46 2.0 46 46 46 Contrast 239 21.0-8 -11-36 Effect -4-5.5-18 Coefficient 59.75-2 -2.75-9 2-4-135

2 2 Full Factorial Design Standard Error of Effects To test coefficients, calculate standard errors Pooled variance = SSE = s p 2 = (sum of the run variances) = 21 Standard error = s e = 2 s p 2 k = 2.291 se 2.291 Standard error of the coefficients = s = 0. 81 k 2 r *2 2*2 (Where k = # of factors = 2, r = # replications = 2) 2 2 2-4-136

2 2 Full Factorial Design Calculate Test Statistics In each case the test statistic is calculated using the relationship: t test = [ Coefficient/ s ] The three test statistics are calculated. t A = [-2.0/0.81] = -2.47 t B = [-2.75/0.81] = -3.40 t AB = [-9.0/0.81] = -11.11 The degrees of freedom are determined using the equation (r 1) * 2 k = (2 1) * 2 2 = 1 * 4 = 4 degrees of freedom. 2-4-137

1 tail 0.10 0.05 0.025 0.01 0.005 0.0005 2 tail 0.20 0.10 0.05 0.02 0.01 0.001 df 1 3.0777 6.3137 12.7062 31.8210 63.6559 636.5776 2 1.8856 2.9200 4.3027 6.9645 9.9250 31.5998 3 1.6377 2.3534 3.1824 4.5407 5.8408 12.9244 4 1.5332 2.1318 2.7765 3.7469 4.6041 8.6101 5 1.4759 2.0150 2.5706 3.3649 4.0321 6.8685 6 1.4398 1.9432 2.4469 3.1427 3.7074 5.9587 7 1.4149 1.8946 2.3646 2.9979 3.4995 5.4081 8 1.3968 1.8595 2.3060 2.8965 3.3554 5.0414 9 1.3830 1.8331 2.2622 2.8214 3.2498 4.7809 10 1.3722 1.8125 2.2281 2.7638 3.1693 4.5868 11 1.3634 1.7959 2.2010 2.7181 3.1058 4.4369 12 1.3562 1.7823 2.1788 2.6810 3.0545 4.3178 13 1.3502 1.7709 2.1604 2.6503 3.0123 4.2209 14 1.3450 1.7613 2.1448 2.6245 2.9768 4.1403 15 1.3406 1.7531 2.1315 2.6025 2.9467 4.0728 16 1.3368 1.7459 2.1199 2.5835 2.9208 4.0149 17 1.3334 1.7396 2.1098 2.5669 2.8982 3.9651 18 1.3304 1.7341 2.1009 2.5524 2.8784 3.9217 19 1.3277 1.7291 2.0930 2.5395 2.8609 3.8833 20 1.3253 1.7247 2.0860 2.5280 2.8453 3.8496 2-4-138

Answer for 2 2 Full Factorial Exercise Run A B AB Y1 Y2 Mean s2 A B AB 1-1 -1 1 55 56 55.500 0.500-55.500-55.500 55.500 2 1-1 -1 70 69 69.500 0.500 69.500-69.500-69.500 3-1 1-1 65 71 68.000 18.000-68.000 68.000-68.000 4 1 1 1 45 47 46.000 2.000 46.000 46.000 46.000 Contrast 239.000 21.000-8.000-11.000-36.000 s p = 2.291-4.000-5.500-18.000 s beta = 0.810-2.000-2.750-9.000 t(a)= -2.47 t(b)= -3.39 t(ab)= -11.11 tcrit= 2.776 2-4-139

2 2 Full Factorial Design Hypothesis Test of Coefficients Hypothesis: H 0 : 1 = 0 or 2 = 0 or 3 = 0 H 1 : 1 0 or 2 0 or 3 0 Test Statistic: t A = 2.47, t B = 3.40, t AB = 11.11 Critical Region: t crit = t (alpha,df) = t (.05,4) = 2.776 where df = (2 factors )x(replications 1) = 2 2 x (2-1) = 4 Interpretation: Do not reject H 0 for 1 Reject H 0 for 2 and 3 Conclusion: The coefficient for Factor A is equivalent to zero, while the coefficients for Factors B and AB are significant. 2-4-140

2 2 Full Factorial Design Reduced Prediction Equation Linear predicting relationship for yields is yˆ 59.75 2.0A 2.75B 9. 0AB Since A is not likely a significant influencing factor, the equation can be reduced to yˆ 59.75 2.75B 9. 0AB 2-4-141

2 3 Full Factorial Design 3 Factors Each With 2 Levels Factor Levels A 200 600 B Low High C 20 28 2-4-142

Balanced 2 3 Full Factorial Design/Standard Order Coding Run Factors A B C Y1 Y2 1 200 Low 20-1 -1-1 221 311 2 600 Low 20 1-1 -1 325 435 3 200 High 20-1 1-1 354 348 4 600 High 20 1 1-1 552 472 5 200 Low 28-1 -1 1 440 453 6 600 Low 28 1-1 1 406 377 7 200 High 28-1 1 1 605 500 8 600 High 28 1 1 1 392 419 2-4-143

2 3 Full Factorial Design Coded with Interactions Run Factors A B C AB AC BC ABC Y1 Y2 1 200 Low 20-1 -1-1 1 1 1-1 221 311 2 600 Low 20 1-1 -1-1 -1 1 1 325 435 3 200 High 20-1 1-1 -1 1-1 1 354 348 4 600 High 20 1 1-1 1-1 -1-1 552 472 5 200 Low 28-1 -1 1 1-1 -1 1 440 453 6 600 Low 28 1-1 1-1 1-1 -1 406 377 7 200 High 28-1 1 1-1 -1 1-1 605 500 8 600 High 28 1 1 1 1 1 1 1 392 419 2-4-144

2 3 Full Factorial Design - Std. Errors and Coefficient Test Values The same equations listed earlier apply and they are: Pooled variance = SSE = s p 2 = (sum of the run variances) = 19700 Standard error = s e = 2 s p 2 k = 49.6236 49.6236 e Standard error of the coefficients = s = 12. 4059 2 r *2 k 2*2 (Where k = #of factors = 2, r = # replications = 2) The Test Values for the t s are calculated from: s ttest Coefficient i e.g., B 1 Test Value is calculated as follows = 9.125/12.4059=0.73554 3 2 s 2-4-145

2 3 Full Factorial Design All Values Spreadsheet Calculated Run Average S2 A B C AB AC BC ABC 1 266 4050-266 -266-266 266 266 266-266 2 380 6050 380-380 -380-380 -380 380 380 3 351 18-351 351-351 -351 351-351 351 4 512 3200 512 512-512 512-512 -512-512 5 446.5 84.5-446.5-446.5 446.5 446.5-446.5-446.5 446.5 6 391.5 420.5 391.5-391.5 391.5-391.5 391.5-391.5-391.5 7 552.5 5512.5-552.5 552.5 552.5-552.5-552.5 552.5-552.5 8 405.5 364.5 405.5 405.5 405.5 405.5 405.5 405.5 405.5 3305 19700 Contrs 73 337 287-45 -477-97 -139 N/A s p =SSE Effects 18.25 84.25 71.75-11.25-119.25-24.25-34.75 Bo 413.125 N/A Coefs 9.125 42.125 35.875-5.625-59.625-12.125-17.375 TSS 134587.8 SSi 1332.25 28392.25 20592.25 506.25 56882.25 2352.25 4830.25 sigma=s e= 49.62358 t test 0.73554 3.39556 2.89177-0.45341-4.80618-0.97736-1.40054 S beta 12.4059 B1 B2 B3 B4 B5 B6 B7 2-4-146

2 3 Full Factorial Design Additional Thoughts Third order or higher interactions are rarely important (e.g., ABC, ABCD, and ABCDE) Second order interactions are often important (e.g., AB, AC, and BC) Number of trials increases significantly with more factors 2-4-147

Your Turn Exercise 1 Using DOE KISS or DOE PRO The following is the design of an experiment. The process under study was a printing process. Factor Levels A Press Speed 200 600 B Ink Viscosity Low High C Paper Weight 20 28 A complete factorial experiment was run. The results and coding for standard order are shown below. The higher the output yield the better. What is the optimum setting for this process? Use =.05. 2-4-148

Your Turn Exercise 1 Using DOE KISS or DOE PRO Factors Run PS IV PW Y1 Y2 1 200 Low 20 34 32 2 600 Low 20 16 18 3 200 High 20 26 24 4 600 High 20 21 23 5 200 Low 28 36 34 6 600 Low 28 26 28 7 200 High 28 39 37 8 600 High 28 31 33 2-4-149

Your Turn Exercise 1 Factors Codes Y1 Y2 Run PS IV PW 1 200 Low 20-1 -1-1 34 32 2 600 Low 20 1-1 -1 16 18 3 200 High 20-1 1-1 26 24 4 600 High 20 1 1-1 21 23 5 200 Low 28-1 -1 1 36 34 6 600 Low 28 1-1 1 26 28 7 200 High 28-1 1 1 39 37 8 600 High 28 1 1 1 31 33 2-4-150

Your Turn Exercise 1 Y-hat Model Factor Name Coeff P(2 Tail) Const 28.625 6.04E-13 A PS -4.12500 2.66E-06 B IV 0.62500 0.115077 C PW 4.37500 1.7E-06 AB 1.87500 0.000725 AC 0.62500 0.115077 BC 1.37500 0.004615 ABC -1.37500 0.004615 2-4-151

Your Turn Exercise 1 Y = 28.625-4.125 x PS + 4.375 x PW + 1.875 x PS x IV + 1.375 x IV x PW 1.375 x PS x IV x PW 28.625 1 28.625 A = Low A -4.125-1 4.125 B = High C 4.375 1 4.375 C = High AB 1.875-1 -1.875 BC 1.375 1 1.375 ABC -1.375-1 1.375 38 2-4-152

Your Turn Exercise 2 Using DOEPRO, set up the design and coding for the following experimental design. Factor Levels A Low High B Low High C Low High D Low High 2-4-153

Blank for 2 nd Exercise Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 2-4-154

Answer for 2 nd Exercise Run A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD 1-1 -1-1 -1 1 1 1 1 1 1-1 -1-1 -1 1 2 1-1 -1-1 -1-1 -1 1 1 1 1 1 1-1 -1 3-1 1-1 -1-1 1 1-1 -1 1 1 1-1 1-1 4 1 1-1 -1 1-1 -1-1 -1 1-1 -1 1 1 1 5-1 -1 1-1 1-1 1-1 1-1 1-1 1 1-1 6 1-1 1-1 -1 1-1 -1 1-1 -1 1-1 1 1 7-1 1 1-1 -1-1 1 1-1 -1-1 1 1-1 1 8 1 1 1-1 1 1-1 1-1 -1 1-1 -1-1 -1 9-1 -1-1 1 1 1-1 1-1 -1-1 1 1 1-1 10 1-1 -1 1-1 -1 1 1-1 -1 1-1 -1 1 1 11-1 1-1 1-1 1-1 -1 1-1 1-1 1-1 1 12 1 1-1 1 1-1 1-1 1-1 -1 1-1 -1-1 13-1 -1 1 1 1-1 -1-1 -1 1 1 1-1 -1 1 14 1-1 1 1-1 1 1-1 -1 1-1 -1 1-1 -1 15-1 1 1 1-1 -1-1 1 1 1-1 -1-1 1-1 16 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2-4-155

Response Surface Methodology -Uses a non-linear equation to fit the response surface. e.g., a polynomial equation with an interaction term. -Higher order terms are possible, e.g., -At least 3 levels are needed for each factor. -Available in DOEKISS and DOE PRO. x 2 1 2-4-156

Response Value Response Surface Surface Plot of O vs P Constants: 2300 2200 2100 2000 1900 1800 1700 1600 1500-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1 0.8 0.2 P -0.4 2200-2300 2100-2200 2000-2100 1900-2000 1800-1900 1700-1800 1600-1700 1500-1600 O -1 2-4-157

Making use of Fractional Factorials Screening designs Used to eliminate factors Follow with a full factorial on significant factors to identify interactions Separate lions from pussycats 2-4-158

Plackett-Burman Screening Designs Must have L levels where L is prime For screening designs usually 2 Number or runs is a multiple of L 2 (4) All main effects are estimated with the same precision All main factor effects are estimated independently of each other Can estimate main factor effects of up to k-1 factors where k is the number of tests run 2-4-159

Eight Run Plackett-Burman (2 7-4 ) Run A B C D E F G 1 - - - - - - - 2 - - - + + + + 3 - + + + + - - 4 - + + - - + + 5 + + - - + + - 6 + + - + - - + 7 + - + + - + - 8 + - + - + - + 2-4-160

Eight Run Plackett-Burman (2 7-4 ) Main Effects A B C D E F G Two-Factor -BC -AC -AB -AE -AD -AG -AF Interaction -DE -DF -DG -BF -BG -BD -BE Confounding -FG -EF -EF -CG -CF -CE -CD Three-Factor BDG ADG ADF ABG ABF ABE ABD Interaction BEF AEF BDE ACF BCD ACD DEF Confounding CDF CDE BFG BCE DFG DEG BCF CEG CFG AEG EFG ACG BCG ACE 2-4-161

Plackett-Burman Screening Designs For More Information: Understanding Industrial Experimentation by Donald J. Wheeler, SPC Press, Inc. Tables of Screening Designs by Donald J. Wheeler, SPC Press, Inc. 2-4-162

Making use of Fractional Factorials Full factorial analysis requires L K tests. For full factorial analysis of four factors, 2 4 = 16 tests are required. For various reasons, only 8 tests can be run. Cost Access Time Fractional factorial analysis required. 2-4-163

Issues with Fractional Factorials Leads to aliases Examples--C and ABD, D and ABC 4 th (D) factor in ABC column Confounding because of identical columns or exactly opposite columns A and BCD B and ACD C and ABD D and ABC AB and CD AC and BD BC and AD (I) and ABCD 2-4-164

2 3 Full Factorial Full Factorial Calculation Matrix Run (I) A B C AB AC BC ABC 1 + - - - + + + - 2 + + - - - - + + 3 + - + - - + - + 4 + + + - + - - - 5 + - - + + - - + 6 + + - + - + - - 7 + - + + - - + - 8 + + + + + + + + 2-4-165

2 k-p Fractional Factorials Fractional Factorial Calculation Matrix D Run (I) A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD 1 + - - - - + + + + + + - - - - + 2 + + - - + - - + + - - + - - + + 3 + - + - + - + - - + - + - + - + 4 + + + - - + - - - - + - - + + + 5 + - - + + + - - - - + + + - - + 6 + + - + - - + - - + - - + - + + 7 + - + + - - - + + - - - + + - + 8 + + + + + + + + + + + + + + + + ABC is an Alias for D in the 2 4-1 Fractional Factorial Experiment 2-4-166

2 k-p Fractional Factorials Fractional Factorial Calculation Matrix Run (I) A B C D AB AC AD BC BD CD ABC ABD ACD BCD ABCD 1 + - - - - + + + + + + - - - - + 2 + + - - + - - + + - - + - - + + 3 + - + - + - + - - + - + - + - + 4 + + + - - + - - - - + - - + + + 5 + - - + + + - - - - + + + - - + 6 + + - + - - + - - + - - + - + + 7 + - + + - - - + + - - - + + - + 8 + + + + + + + + + + + + + + + + AD and BC are Confounded in the 2 4-1 Fractional Factorial Experiment 2-4-167

DOE in Continuous Process Improvement Experiments done in phases to search for improvement/optimality. EVOP 2-4-168

Using a Catapult As a Model Conduct Physical review of catapult Identify major sources of variation Choose Factors to be studied 2-4 factors at two levels Set up Full Factorial Experiment with Replication Randomize Runs Run the experiment Develop linear model Test the model 2-4-169

Leading 6 Sigma Teams Through a DOE One Plan purpose and nature of experimentation Attempt to identify major sources of variation Chose most important controllable factors Select linear ranges of important factors 2-4-170

Leading 6 Sigma Teams Through a DOE Two Randomize/control factors not being studied Include replication to better understand variation and error Develop a set of hypotheses Select the DOE Design 2-4-171

Leading 6 Sigma Teams Through a DOE Three Plan the trials and data gathering Perform the analysis using DOE techniques Develop and test model parameters Verify model setting Use model to maximize yields 2-4-172

Where to Get More Information World Class Quality using Designed Experiments, Bhote, AMACOM 2000 Statistical Quality Design and Control, DeVor, Macmillian 1992 Analysis of Variance PDF in the readings section of the course web site 2-4-173