Design of Experiments (DOE) Instructor: Thomas Oesterle

Transcription

1 1 Design of Experiments (DOE) Instructor: Thomas Oesterle

2 2 Instructor Thomas Oesterle

3 3 Agenda Introduction Planning the Experiment Selecting a Design Matrix Analyzing the Data Modeling the Data, Optimizing Input Exercises

4 Minitab Analysis Significance 4

5 5 How to look at DOE data Big Picture (provides a general view of the data, to find the obvious, outliers, transcription errors etc) Histograms Scatter Plots DOE Output Main Effects & Interactions ANOVA P-Value and R-Sq. Residuals

6 Scatterplot 6

7 Scatterplot 7

8 8 ANOVA Analysis (R-Sq) R-Sq a measure of the strength of our model R-Sq.: 97.32% of the variation in our response is explained by the model R-Sq=f(sample size, model used) The closer to 100% the better Minimum R-Sq.: 80% Many processes desire 90% R-Sq(adj): takes terms vs sample size into consideration Launsby Consulting

9 9 ANOVA Analysis P(2tail) P: Probability that a term does not belong into the regression model P-value: probability that two means are identical H 0 : Mean 1 = Mean 2 If p is low, the Null must go! If p < α -> include into model Typically, α = 0.05 Launsby Consulting P < > Significant

10 10 Definition of Significance Layman s (or practical significance): Factor is a big deal relative to response in question Statistical significance: change made in response overwhelms the noise in experiment (signal is at least 2 or 3 times as large as the noise) Note: Practical significance and statistical significance are different

11 11 Significance of Standard Deviation Rule of Thumb: Use factor with steepest slope 1) Calculate standard deviation for low setting StDev(L) 2) Calculate standard deviation for high setting StDev(H) 3) If S(H) > 3 x S(L) -> good candidate to reduce variation

12 12 Practical versus Statistical Significance b u m p Main Effects If the difference is not greater than 4, it is not of practical importance to the engineer h t A(-) (A) B(+) Need both before you get very excited 1(-) 2(+) 3(-) 5(+) (B) (C) Factors 2(-) 4(+) (D) All are statistically significant Variable Coefficient Std Error 95% CI Tolerance T P(2 Tail) Constant ± tech(a):a ± tech(a):b ± (B) ± (C) ± (D) ± Not a big deal

13 13 Statistical Significance in Comparison Source DF SS MS F p-value A B A*B Residual Error Total 11 Statistically Significant A: 89% Practically Significant B: 5% Practically NOT significant Res. Error: 4% A*B: 2%

14 14 Practical and Statistical Significance Statistically Significant? no Statistically Significant? yes Practically significant? No Practically significant? yes Ignore in future Noisy response, investigate further Do cost/benefit analysis Excellent

15 15 One Factor Analysis (ANOVA) Problem Statement A circuit board is related to high field failure. After careful analysis the primary failure was attributed to low adhesion of the circuitry. The subject matter expert hypothesized that the problem is related to the type of solder paste used during production. Three types of paste are used; Type A, Type B, and Type C. The team decides to evaluate which paste is best. They randomly select 9 circuit boards and randomly assign them to the three paste types. After curing the samples they subjected the boards to a tensile test to quantify the adhesion of the different paste types.

16 16 Prepare Data for Analysis 1 2 3

17 17 Analysis 1 2 3

18 ANOVA Result 18

19 19 Residual Error (PCB ANOVA) Check Store Residuals

20 20 Examine Data Further Residuals

21 21 Are Residual randomly distributed? Fact: Residuals should be normally distributed around 0. Test: STAT->Basic Statistics-> Normality Test; Select C6. Evaluate Plot

22 Understanding R-Sq (Shaft DIA) 22

23 23 Understanding ANOVA, Residual Errors Residual Example Run Time Pressure L1 L2 L3 Average (L) Replicates: 3 Prediction Residual Base Runs: α: Levels (Time) Levels (Pressure) Effect Grand Mean Linear Model ANOVA Analysis Term Coefficient SS (Sum of Squares) DF MS (Mean Square) F p Constant Main Effects Pressure Pressure Time Time Interaction Interaction Error Total Time -1 Pressure -1 5 Interaction 1 Length (Predict)

24 24 Residual Analysis (Alternate Method) During Analysis Selection, check

25 25 Residual Analysis Normal Even Distribution No Trending

26 26 Alternative Method Select Fits and Residuals below Storage during Analyze Design Process

27 Fits and Residuals 27

28 Analyze Normality of Residuals 28

29 Normality Check 29

30 30

31 31 Avoiding MS Bias Error in an experiment is natural variation for replicates. Not sensitive enough? Too sensitive? Minitab ANOVA refers to this error as MSE MSE = Pure Error (replication) + Lack of Fit (Terms removed) Causes for MSE Bias (Pure Error) 1) (-) Repeats are taken and treated in Minitab as Replicates (measured variation smaller than actual) 2) (+) One replicate is run under different conditions and blocking feature is not used

32 32 Avoiding MS Bias Not sensitive enough? Causes for Lack of Fit 1) Terms are removed incorrectly from the model Too sensitive?

33 Indicators of MSE Bias 33

34 34 Remedies for MSE Bias Reference (last 4 Slides) Article from James A. Colton in Scientific Computing, Avoiding Mean Square Error Bias in Designed Experiments.

35 35

36 36 Confidence Levels Establish confidence level around mean Use Microsoft Excel Option 1: Standard Deviation of your process is known, use = confidence.norm(alpha, stddev, n) Option 2: Overall Standard Deviation is not known: Use = confidence.t(alpha, stddev, n) Alpha = 0.05 for 95% confidence level (Commands shown for Microsoft Excel 2010)

37 37 Confidence Levels Establish confidence level around mean: If the value as predicted by the model is within the mean +/ calculated confidence of the confirmation run, the model is good!

38 38 Formula If you do not have Microsoft Excel 2010, use the following equation: CI = mean +/- t * s/ n

39 39 T-Table for 95% Confidence Limit Degrees of Freedom t-value for 2-Sided 95% Confidence Limits on Mean t-value for 1-Sided 95% Confidence Limit On Mean df=n-1 t(n-1) t(n-1) Excerpted from Table II The t- Distribution, from Introductory Statistics, John A. Ingram, Cummings Publishing Company, Menlo Park, Ca (1974).

40 40 Example You completed a confirmation run with 20 samples. The sample mean is 40 cm. Standard deviation is 0.9 cm. 1)Calculate the confidence interval for alpha of 5% CI = Mean +/ x 0.9 / 20 = CI = 40 cm +/ cm

41 41 Why you may not confirm Data Transcription Error Experimental Deviation Unreliable / Incapable Measurement System Large Variation (MSE Bias) Missed Interactions Incorrect Interpretation of Interactions Inadequate Model Something along the way changed

42 42

43 43 Response Optimizer Example Multiple Response Example Suppose we are manufacturing bike tires. We have 3 responses we would like to optimize. They are tensile strength, hardness, and rolling resistance. You will optimize the factor levels using desirability function in Minitab. Factors are: (C) Mesh Size 40, 80 (B) % Ground Rubber: 3, 10 (A) Binder Amount: 5, 25 Specification: 1) Tensile Strength: less than 1390 unacceptable, 1390 to 1500 increasingly good, 1500 or greater perfect (Relative Importance: 10) 2) Hardness: 50 +/- 1 (Relative Importance: 10) 3) Rolling Resistance: Less than 20 is ideal; 20 to 30 is decreasingly acceptable; Greater than 30 is unacceptable (Relative Importance: 5)

44 44 Exercise 1) Generate a Full Factorial Design a. 2 Levels, 3 Factors 2) Open Worksheet Multiple Response Optimizer a. Analyze designs and generate response surface b. Generate

45 Tensile Strength 45

46 Hardness 46

47 Resistance 47

48 48 Response Optimizer Mintab Sequence

49 Minitab Set-Up 49

50 50 Response Optimizer Best fit when Composite Desirability is at 1

51 51 Response Optimizer Calculation % Tes. Tes. AVE DES AVE DES Ave DES Comp. Binder Ground Mesh Str. Str. Tensile Tensile Hardn. Hardn. Hardn. Hardn. Res. Res. Res. Res DES Refer to Engineering Today s Designed Experiments for equation on D

52 52

53 Airbag System 53

54 54 Airbag System Gas Weight Orifice diameter Propellant type and propellant weight Our task is to characterize a new engineering concept related to the deployment of an automotive air bag system. The subsystem consists of a steel canister filled with gas and charge. After the canister is filled, the orifice is welded closed with a steel sphere welded in place. The engineering team believes that the key variables are orifice diameter, gas weight, propellant weight, and type of charge.

55 55 Air Bag System Requirements Factors Orifice diameter Levels.086,.140 Gas Weight 12.8, 13.8 Prop. Type 23b, 35b Prop. Wt..9, 1.1 Response Pressure Max pressure First pre 2,3 T90 8,9,10 First Pressure Time Time to 90 % max Max pre 110,120,130 Bag fill and deflation is over in approximately.05 sec

56 Main Effects 56

57 57

58 58 References and Further Reading Reference 1: Keki R. Bhote (2001) The Ultimate Six Sigma, Beyond Quality Excellence to Total Business Excellence, AMACOM Reference 2: Robert Launsby, Jayme Lahey (2006) Engineering Today s Designed Experiments, Launsby Consulting Further Reading Stephen R. Schmidt, Robert Launsby, Understanding Industrial Designed Experiments