What is Experimental Design?

Size: px

Start display at page:

Download "What is Experimental Design?"

Hilary Quinn
6 years ago
Views:

1 One Factor ANOVA

2 What is Experimental Design? A designed experiment is a test in which purposeful changes are made to the input variables (x) so that we may observe and identify the reasons for change in the output response (y). Objectives: Which variables are most influential on the response y? Where to set the influential x s so y is near a desired value? Where to set the influential x s so variability in y is small?

3 Principals of design Replication: repetition of the basic experiment Allows the estimation experimental error Obtain a more precise estimate of the effect of a factor Randomization: both the allocation of the experimental material and the order in which the individual runs are performed are randomly determined Blocking: a block is a homogeneous portion of the experimental material Used to increase the precision of an experiment

4 Business Example A manufacturer is interested in maximizing the tensile strength of a new synthetic fiber that will be used to make cloth for men s shirt. From previous tests, the manufacturer knows: the strength is affected by the percentage of cotton in the fiber The range of the percentage is 10% to 40% Experiment with a Single Factor

5 Example continued Maximizing the tensile strength of a new synthetic fiber used to make shirts* Strength is affected by the % of cotton in the fiber Test five levels of cotton percentage 15%, 20%, 25%, 30%, 35% Test five specimens at each cotton % So, a = 5 levels of the factor, and n = 5 replicates All 25 runs are made in random order *DCM pp 50

6 Example continued Experimental Runs This randomized test sequence, known as a Completely Randomized Design (CRD), is necessary to prevent the effects of unknown nuisance variables from contaminating the results Test Sequences Run Numbers Percentage of Cotton

7 Data for the Cotton Percentage Example Cotton Percentage Observations Total Average

8 Tensile strength Tensile strength Tensile Strength Vs Cotton Percentage Cotton percentage Cotton percentage Box plots Scatter diagram

9 We strongly suspect that: Exploratory Results Cotton content affects tensile strength Around 30% cotton would result in maximum strength A more objective analysis: Should we perform a t-test on all possible pairs of means? No, results in a substantial increase in the type I error The appropriate procedure for testing the equality of several means is the analysis of variance (ANOVA)

10 ANOVA The name ANOVA is derived from a partitioning of total variability into its component parts: ANalysisOfVAriance The total sum of squares is a measure of overall variability in the data: a i=1 n j=1 SS total = (yij y.. ) 2 The total variability in the data can be partitioned into a sum of squares of the differences between the treatment averages and the grand average, PLUS a sum of squares of the differences of observations within treatments from the treatment average: SS total = SS treatment +SS within

11 ANOVA Degrees of freedom There are N = an total observations, so SS total has N -1 degrees of freedom. There are a levels of the factor, so SS treatment has a - 1 degrees of freedom. Thus, we have N a degrees of freedom for SS within

12 ANOVA The analysis of variance identity provides us with two estimates of σ 2 one based on the inherent variability within treatments and one based on the variability between treatments SS total = SS treatment +SS within If there are no differences in the treatment means, then these two estimates should be very similar. If they are not, we suspect that the observed differences must be caused by differences in treatment means.

13 ANOVA for the Cotton Percentage Data Source of Variation Sum of SS Squares treatment Degrees of Freedom Mean Square F 0 Test Statistics Cotton Percentage F 0 =14.76 Error SS within Total

14 ANOVA Additional Concepts A factor is fixed if the levels of a factor are predetermined and the experimenter is interested only in those particular levels (e.g. 250, 300, or 350 F). A factor is classified as random if the levels are selected at random from a population of levels (e.g. 254, 287, and 326 F). When there is only one factor, the classification does not have any effect on how the data are analyzed.

15 Fixed Effects Model y ij = μ + τ i + ε ij y ij : the ij th observation μ: grand mean, a parameter common to all treatments τ i : treatment effect, a parameter unique to the i th treatment, ε ij : random error

16 Least Squares Estimate The Sum of squares of error : a n L = (y ij μ τ i ) 2 i=1 j=1 Choose μ and τ that minimize L: L μ μ,τi = 0 L τ μ,τi = 0

17 Least Squares Estimate Adding a constraint: a τ i i=1 = 0 The least squares estimate for μ, τ i : μ = y.. τ i = y i. y.., i = 1, 2,, a

18 Cotton Percentage example -continued- Estimate of the overall mean: μ = y.. = = 15.04

19 Cotton Percentage example -continued- Estimate of treatment effects: τ 1 = y 1. y.. = = 5.24 τ 2 = y 2. y.. = = 0.36 τ 3 = y 3. y.. = = 2.56 τ 4 = y 4. y.. = = 6.56 τ 5 = y 5. y.. = = 4.24 *see slide 7 for the data

20 Confidence Interval (CI) 100(1 α) percent CI on the i th treatment mean: y i. ± tα 2,N a MS E n 100(1 α) percent CI on the difference of two treatments mean: y i. y j. ± tα 2,N a 2MS E n

21 Cotton Percentage example -continued- 95% CI on the mean of treatment 4: [21.60 ± ] y 4. t 0.025,20 MS E n

22 Temperature Example Determine the effects of temperature on process yields Case I: Two levels of temperature setting Case II: Three levels of temperature setting

23 Temperature Vs Process yields Week # 1 Temperature Day M Tu W Th F Week #3 Week # 2 M Tu W Th F Week # 4

24 Yield Exploratory analysis: Time Sequence plot T1 T2 Any difference? T 2 =350 F T 1 =250 F TGIF? M Tu W Th F M Tu W Th F Time

25 Analysis t-test General form of t Statistics for one population θ θ t = s θ θ: the parameter to be estimated θ: the sample statistic that is the estimate of θ s θ : the estimator of the standard deviation of θ

26 Temperature Data I t-test Our interest : the difference μ 1 μ 2 Let θ = X 1 X 2 σ X1 X 2 = σ 1 2 n 1 + σ 2 2 n 2 Generally, σ 1, σ 2 unknown, use estimates: o Two methods of estimating σ X1 X 2, yielding exact and approximate t-test.

27 Exact t-test Exact t-test: t = θ = X 1 X 2 X 1 X 2 d s p 1/n 1 + 1/n 2 zero or a constant Estimate of σ x1 X 2 in the form of pooled variance where n 1 and n 2 are sample sizes, s p2 = (n 1 1)s (n 2 1)s 2, n 1 +n 2 2 s a2 = 1 n a 1 n a x ai x 2 i=1 a., a = 1, 2

28 Temperature Data I exact t-test For the temperature data: T= = /10+1/10 P(t 0.893)=0.191 p-value = 0.191, at 5% significance level, fail to reject H 0 Conclusion -- two scenarios: μ 1 μ 2 OR Highly variable data in each sample

29 Exact t-test assumptions Normality of population Independence of samples Independence of observations Small (n<30) and equal or similar sample sizes(n 1 n 2 ) Equal variance σ 12 = σ 22 = σ 2

30 Alternative approach: Confidence Interval for difference One sided test, such as μ 1 < μ 2 (X 1 X 2 ) ± t α,d.f. s x1 x 2 Two Sided test, such as μ 1 μ 2 (X 1 X 2 ) ± tα 2,d.f.s x 1 x 2 Bounds can be calculated and the decision can be made based on interval information

31 ANOVA for Temperature Data I Source of Variations d.f. SS MS F Temperature a-1= SS treatment Within an-a= SS within Total an-1= Test Statistics Under H 0, F~ F(a-1,n-1), p= a SS treatment = n y i. y 2.. i=1 a i=1 n j=1 a i=1 n j=1 SS within = (yij y i. ) 2 SS total = (yij y.. ) 2 a: levels of treatment (temperature), a=2 n: replication of each treatment, n=10 Numerator of s p in the exact t test! ANOVA uses SAME assumptions as t test!

32 Temperature Data II Variations: Week to Week? Day to Day? Temperature Day M Tu W Th F M Tu W Th F

33 Process Yield (Tons) Exploratory Analysis: Temperature Data II- box plots H 0 : μ 1 = μ 2 = μ 3 H A : At least one of the mean is different Temperature (F) variability within each setting about the same

34 ANOVA for Temperature Data(3 levels) Source of Variations d.f. SS MS F Temperature Within Total p-value=.001 Reject H 0 SS temp = 3 i=1 3 i=1 10 j=1 n 10 j=1 y ij 2 3 i=1 3 i=1 10 j=1 an 10 j=1 y ij SS total = y ij2 ( y ij ) 2 /an 2 SS within =SS total -SS temp

35 Typical Data for a Single Factor Experiments Treatment (level) Observations Totals Averages 1 y 11 y 12 Y 1n y 1. y 1. 2 y 21 y 22 y 2n y 2. y 2. a i y i1 y i2 y in y i a y a1 y a2 y an y a. y a. n y.. y..

36 Clinical Trial Example A clinical trial is designed to estimate the efficacy of an experimental drug (D) compared to placebo (P) in congestive heart failure (CHF) Efficacy measure: the rate of change per week in distance walked after being administered therapy (D or P) Baseline measure of left ventricular ejection fraction (LVEF) --- the lower the LVEF, the more serious CHF 2 investigators Design and Analysis Tandem

37 Rate of Change in Distance Walked per Week by Drug, Investigator, and Baseline LVEF Investigator LVEF 15 15<LVEF 25 25<LVEF 30 30<LVEF D P D P D P D P

38 Analysis of CHF data Means, standard deviations and sample sizes: X D = 240.6, S D = 451.2, n D = 32, X P = 98.5, S P = 322.4, n p = 31, The two-sample t with d.f. = 61 is T = X D X P S D 2 /n D + S P 2 /n P = 3.43 corresponds to a p-value = : reject H 0, drug is significantly more efficacious than placebo overall

39 ANOVA for CHF data Original data Rank transform Factor d.f. F P F P Drug Investigator LVEF Inv. * drug LVEF*drug

40 Exploratory Data Analysis

41 Box plots by Investigator

42 Box plots Investigator effect pooled

43 Mean Distance by LVEF category

44 Mean Distance (D-P) by LVEF category

What If There Are More Than. Two Factor Levels?

What If There Are More Than. Two Factor Levels? What If There Are More Than Chapter 3 Two Factor Levels? Comparing more that two factor levels the analysis of variance ANOVA decomposition of total variability Statistical testing & analysis Checking