Vladimir Spokoiny Design of Experiments in Science and Industry

Transcription

1 W eierstraß-institut für Angew andte Analysis und Stochastik Vladimir Spokoiny Design of Experiments in Science and Industry Mohrenstr. 39, Berlin June, 23, 2009 DoE Vladimir Spokoiny 0-1

2 Outline 1 Experiments in Science and Industry 2 Simple comparative experiments, t-test: 3 Experiments with a single factor, ANOVA 4 Randomised blocks, Latin square, Graeco-Latin square: 5 Factorial and fractional factorial designs: DoE Vladimir Spokoiny 1-2

3 Experiments in Science and Industry The primary goal in scientific research is usually to show the statistical significance of an effect that a particular factor exerts on the dependent variable of interest. In industrial settings, the primary goal is usually to extract the maximum amount of unbiased information regarding the factors affecting a production process from as few (costly) observations as possible. DoE Vladimir Spokoiny 1-3

4 Computational Problems There are basically two general issues to which Experimental Design is addressed: 1 How to design an optimal experiment, and 2 How to analyze the results of an experiment. DoE Vladimir Spokoiny 1-4

5 Outline 1 Experiments in Science and Industry 2 Simple comparative experiments, t-test: 3 Experiments with a single factor, ANOVA 4 Randomised blocks, Latin square, Graeco-Latin square: 5 Factorial and fractional factorial designs: DoE Vladimir Spokoiny 2-5

6 Experimental data or Results Experimental data: Y 1,..., Y n Sample size n ; randomness and Experimental errors : the observations Y i are random (uncertain) and independent. Empirical mean: Y = (Y Y n )/n DoE Vladimir Spokoiny 2-6

7 Population and distribution Population Y : infinite or very large collection of data. Population mean IE(Y ) :) η = IE(Y ) = 1 N Y Population variance V (Y ) : σ 2 = IE(Y η) 2 = 1 N (Y η) 2. Population distribution: every Y i follows some reference distribution. Normal populations: every Y i η and variance σ 2. is normally distributed with mean Each variable σ 1 (Y i η) is standard normal. DoE Vladimir Spokoiny 2-7

8 Empirical (descriptive) measures of the Sample Sample Y 1,..., Y n. Sample mean: Y = (Y Y n )/n Sample variance (population mean known) ṡ 2 = 1 (Y i η) 2 n Sample variance (population mean unknown) s 2 = 1 n 1 i (Y i Y ) 2 i Student (t-) distribution: (Y η)/s follows the t-distribution with n 1 degrees of freedom. DoE Vladimir Spokoiny 2-8

9 Significance t-test Given two samples: n A observations with treatment A ; n B observations with treatment B. Sample means Y A, Y B ; population means η A, η B ; Sample variances: sa 2 = 1 (Y Y A ) 2 sb 2 n = 1 (Y Y B ) 2 A n B A B Pooled sample variance ( s 2 1 = (Y Y A ) 2 + n A + n B A B (Y Y B ) 2 ) DoE Vladimir Spokoiny 2-9

10 Significance t-test Hypothesis H 0 : η A = η B ; Alternative (two-sided) H 1 : η A η B ; Alternative (one-sided) H + 1 : η A > η B or η A < η B. Consider t 0 = (Y A Y B ) (η A η B ) s 1/n A + 1/n B If the individual errors are homogeneous between samples, then t 0 is t-distributed with n A + n B 2 degrees of freedom. Test statistic t = (Y A Y B ) s 1/n A + 1/n B Under the hypothesis, t = t 0. DoE Vladimir Spokoiny 2-10

11 Significance t-test. Cont Test level (size): given α > 0 IP(reject H 0 when correct) α. Quantiles of t 0 : given α < 1, define q α by IP(t 0 > q α ) = α. One-sided test: reject H 0 if t > q α. Two-sided test: reject H 0 if t > q α/2. Used that IP( t 0 > q α/2 ) = 2IP(t > q α/2 ) = α. DoE Vladimir Spokoiny 2-11

12 Outline 1 Experiments in Science and Industry 2 Simple comparative experiments, t-test: 3 Experiments with a single factor, ANOVA 4 Randomised blocks, Latin square, Graeco-Latin square: 5 Factorial and fractional factorial designs: DoE Vladimir Spokoiny 3-12

13 Comparing K treatment means Given K samples, K 2, each of size n k for k = 1,..., K. Sample means in k -treatment: Y k = 1 n k k -treatment sum of squares (SS): i Y ki S k = i (Y ki Y k ) 2 k -treatment sample variance s 2 k = S k/(n k 1) DoE Vladimir Spokoiny 3-13

14 Pooled measures Pooled sample size N = n n K. Pooled (grand) mean: Y = 1 Y ki N Within-treatment sum of squares: S R = S S K Between treatment sum of squares: S T = k k n k (Y k Y ) 2 Within- and between-treatment mean square: s 2 R = S R N k i s 2 T = S T K 1 DoE Vladimir Spokoiny 3-14

15 Analysis of Variance Table Overall variation S D = k (Y ki Y ) 2. i Then S D = k (Y ki Y k ) 2 + i k n k (Y k Y ) 2 = S T + S R. DoE Vladimir Spokoiny 3-15

16 Analysis of Variance Table source of sum of degree of mean variation squares freedom square average S A = NY 2 ν A = 1 s 2 A = S A/ν A between treatments S T = k n k(y k Y ) 2 ν T = K 1 s 2 T = S T /ν T within treatments total S R = k k i (Y ki Y ) 2 i (Y ki Y k ) 2 ν R = N K s 2 R = S R/ν R N DoE Vladimir Spokoiny 3-16

17 Treatment effect Basic model: Y ki = η k + ε ki, ε ki N(0, σ 2 ) i.i.d. where η k is the true treatment mean. Averaged treatment mean: η = 1 N nk η k. Treatment effect: τ k = η k η. Model: Y ki = η + τ k + ε ki Data analysis: Y ki = Y + (Y k Y ) + (Y ki Y k ) DoE Vladimir Spokoiny 3-17

18 Analysis of Variance Table. F-test No effect hypothesis: η 1 =... = η K η. Then all the values Y k and Y are of mean η. Under homogeneous normal errors, the ratio s 2 T /s2 R = ν 1 T ν 1 R k n k(y k Y ) 2 i (Y ki Y k ) 2 k has a fixed Fisher distribution F K 1,N K with (K 1, N K) degrees of freedom. If f α is the (1 α) -quantile of F K 1,N K Large values of S T /S R over f α indicate that the hypothesis of no effect is presumably wrong. DoE Vladimir Spokoiny 3-18

19 Analysis of Variance. Some issues Outlier check: ε ki > q α σ. Treatment dependent error variance: requires some correction of the test. Categorical data (binary, Poisson, multinomial, etc): apply Generalized Linear Models in place of linear: Y ki P η+τk where (P υ ) is a fixed exponential family (usually with canonical parameter). Interactions DoE Vladimir Spokoiny 3-19

20 Outline 1 Experiments in Science and Industry 2 Simple comparative experiments, t-test: 3 Experiments with a single factor, ANOVA 4 Randomised blocks, Latin square, Graeco-Latin square: 5 Factorial and fractional factorial designs: DoE Vladimir Spokoiny 4-20

21 Randomized blocks Y ki, observation with k th treatment applied to the i th block, k = 1,..., K and i = 1,..., n. Model: Y ki = η + β i + τ k + ε ki where η is general mean, β i, block effect, τ k, treatment effect. Data analysis: Y ki = Y + (Y i Y ) + (Y k Y ) + (Y ki Y i Y k + Y ) where Y is grand average, Y i, block average, Y k, treatment average, and Y ki Y i Y k + Y, residual. DoE Vladimir Spokoiny 4-21

22 Analysis of Variance Table source of variation sum of squares degree of freedom average (correction factor) S = nky 2 1 between blocks S B = K i (Y i Y ) 2 n 1 between treatments S T = n k (Y k Y ) 2 K 1 residuals S R = k i (Y ki Y k ) 2 (n 1)(K 1) total k i (Y ki Y i Y k + Y ) 2 DoE Vladimir Spokoiny 4-22 N = nk

23 Test of no effect The no effect hypothesis H 0 : η 1 =... = η K = η can be tested by the value of the ratio st 2 sr 2 = ν 1 T ν 1 R k n(y k Y ) 2 i (Y ki Y k ). 2 k Additivity Y ki = η + β i + τ k + ε ki implies that the block effect cancels, yielding the Fisher distribution F K 1,(n 1)(K 1). Gain of random block against completely random design: smaller variance of the test statistic. DoE Vladimir Spokoiny 4-23

24 Latin Square Design Latin square designs are used when the factors of interest have more than two levels there are no (or only negligible) interactions between factors. Feature: each row and each column receives each treatment exactly once. Example: Car Driver A B D C 2 D C A B 3 B D C A 4 C A B D DoE Vladimir Spokoiny 4-24

25 Latin Square Design: Additive model No interaction yields additive model Y ijk = η + β i + γ j + τ k + ε ijk where β i are row effects, γ j are column effects, τ k are treatment effects, i, j, k = 1,..., K. Requires only K 2 measurements (instead of K 3 ). Hypotheses tested: (i) all τ k are zeros (no treatment effect) (ii) all γ j are zero (no column effect) (iii) all β i are zero (no row effect) DoE Vladimir Spokoiny 4-25

26 Latin Square Design: Analysis of variance Data analysis: Y ijk = Ŷijk + ε ijk with Ŷ ijk = Y + (Y i Y ) + (Y j Y ) + (Y k Y ), where Y = K 2 Y ijk is Grand average, Y i is the row average, Y j is the column average, Y k is the treatment average. Sum of squares: S B = i (Y i Y ) 2, S C = j (Y j Y ) 2, S T = k (Y k Y ) 2, and S R = ijk (Y ijk Ŷijk) 2. DoE Vladimir Spokoiny 4-26

27 Latin Square Design: Analysis of variance Degrees of freedom: τ B = τ C = τ T = K 1, τ R = (K 1)(K 2) Mean squares: sb 2 = 1 (Y i Y ) 2, sc 2 τ = 1 (Y j Y ) 2, B τ C i st 2 = 1 (Y k Y ) 2, sr 2 τ = 1 (Y ijk T τ Ŷijk) 2 R k Ratio of mean squares as F-test: sb 2 /s2 R, row effect; sc 2 /s2 R, column effect ; st 2 /s2 R, treatment effect. DoE Vladimir Spokoiny 4-27 j ijk

28 Graeco-Latin Square Design Useful to eliminate more than two sources of variability. Allows to study of K treatments simultaneously with three different blocking variables. Constructed from two Latin squares by superimposing: A B C B C A C A B and α β χ χ α β β χ α results in Aα Bβ Cχ Bχ Cα Aβ Cβ Aχ Bα DoE Vladimir Spokoiny 4-28

29 Outline 1 Experiments in Science and Industry 2 Simple comparative experiments, t-test: 3 Experiments with a single factor, ANOVA 4 Randomised blocks, Latin square, Graeco-Latin square: 5 Factorial and fractional factorial designs: DoE Vladimir Spokoiny 5-29

30 Factorial design at two levels General factorial design: l k levels for the factor k, k = 1,..., K. Requires l 1... l K experimental runs. 2 K factorial design: K factor at two levels each. Requires 2 K runs. One can assume that each factor X k + and. takes the categorical values DoE Vladimir Spokoiny 5-30

31 Calculation of main effects main effect k = Y + k Y k Gain efficiency if variables act additively: 2 3 factorial is as precise as 24 one-fact.-at-a-time method. Permits to measure interaction effects. DoE Vladimir Spokoiny 5-31

32 Two-fact. interaction effects Consider any two factors, e.g. X 1, X 2. Compute average effects (AE) Define the interaction effect Alternatively Y 21 + = AE(X 2 X 1 = +), Y 21 = AE(X 2 X 1 = ). X 1 X 2 = 1 2[ Y 21 + Y 21 ]. X 1 X 2 = 2[ 1 ] Y 12 + Y 12 It suffices to check this formula for two factors only. DoE Vladimir Spokoiny 5-32

33 Three-fact. interaction effects Now consider any three factors, e.g. X 1, X 2, X 3. Define Y = 1 [ AE(X1 X 2 = +, X 3 = +) AE(X 1 X 2 =, X 3 = +) ] 2 Y 123 = 1 [ AE(X1 X 2 = +, X 3 = ) AE(X 1 X 2 =, X 3 = ) ] 2 and the interaction effect X 1 X 2 X 3 X 1 X 2 X 3 = 1 2( Y Y 123 ). DoE Vladimir Spokoiny 5-33

34 Signs for calculating effects from 2 3 mean DoE Vladimir Spokoiny 5-34

35 Fractional Factorial Design at Two Levels Problem: the number of runs of the full 2 K geometrically with K. factorial design grows Let K = 7. The number of effects: average main eff. 2-fact. 3-fact. 4-fact. 5-fact. 6-fact. 7-fact Redundancy: some of interactions are negligible. DoE Vladimir Spokoiny 5-35