Josh Engwer (TTU) 1-Factor ANOVA / 32

Transcription

1 1-Factor ANOVA Engineering Statistics II Section 10.1 Josh Engwer TTU 2018 Josh Engwer (TTU) 1-Factor ANOVA / 32

2 PART I PART I: Many-Sample Inference Experimental Design Terminology Josh Engwer (TTU) 1-Factor ANOVA / 32

3 Many-Sample Inference (Example) Suppose we wish to determine whether three light bulb brands all have similar lifetimes or not. A sample of 5 bulbs from each brand has their lifetimes measured (in years) and recorded in the below table: BULB BRAND: SAMPLE SIZE: LIFETIMES (in yrs): Brand 1 (x 1 ) , 9.07, 8.95, 8.98, 9.54 Brand 2 (x 2 ) , 8.88, 9.10, 8.71, 8.85 Brand 3 (x 3 ) , 8.99, 9.06, 8.93, 9.02 or expressed in terms of means and standard deviations: BULB BRAND: SAMPLE SIZE: MEAN LIFETIMES (in yrs): STD DEV: Brand 1 (x 1 ) 5 x 1 = s Brand 2 (x 2 ) 5 x 2 = s Brand 3 (x 3 ) 5 x 3 = s Now, the appropriate hypotheses are: H 0 : µ 1 = µ 2 = µ 3 H A : At least two of the µ s differ where µ i ( Population Mean of all Brand i light bulbs Josh Engwer (TTU) 1-Factor ANOVA / 32 )

4 Experimental Design Terminology It s crucial to have standard terminology for many-sample inference: Definition The collection of I samples to determine cause & effect is an experiment. Each data point of a sample is called an observation or measurement. The dependent variable to be measured is called the response. The person/object upon which the response is measured is a subject. The manner of sample collection & grouping is called experimental design. The main characteristic distinguishing all the samples is called the factor. The factor s particular values or settings are called its levels. Each sample corresponding to a level is called a group. Nuisance factors are uninteresting factors that influence the response. FACTOR A: GROUP SIZE: GROUPS: Level 1 J x 1 : x 11, x 12,, x 1J Level 2 J x 2 : x 21, x 22,, x 2J... Level I J x I : x I1, x I2,, x IJ Josh Engwer (TTU) 1-Factor ANOVA / 32

5 Experimental Design Terminology It s crucial to have standard terminology for many-sample inference: Definition The collection of I samples to determine cause & effect is an experiment. Each data point of a sample is called an observation or measurement. The dependent variable to be measured is called the response. The person/object upon which the response is measured is a subject. The manner of sample collection & grouping is called experimental design. The main characteristic distinguishing all the samples is called the factor. The factor s particular values or settings are called its levels. Each sample corresponding to a level is called a group. Nuisance factors are uninteresting factors that influence the response. FACTOR A: GROUP GROUP GROUP SIZE: MEAN: STD DEV: Level 1 (x 1 ) J x 1 s 1 Level 2 (x 2 ) J x 2 s Level I (x I ) J x I s I Josh Engwer (TTU) 1-Factor ANOVA / 32

6 Experimental Design Terminology (Example) FACTOR A: (BULB BRAND) GROUP SIZE: GROUPS: (BULB LIFETIMES in yrs) Level 1 (x 1 ) , 9.07, 8.95, 8.98, 9.54 Level 2 (x 2 ) , 8.88, 9.10, 8.71, 8.85 Level 3 (x 3 ) , 8.99, 9.06, 8.93, 9.02 or expressed in terms of means and standard deviations: FACTOR A: (BULB BRAND) GROUP SIZE: GROUP MEAN: GROUP STD DEV: Level 1 (x 1 ) 5 x 1 = s Level 2 (x 2 ) 5 x 2 = s Level 3 (x 3 ) 5 x 3 = s H 0 : µ 1 = µ 2 = µ 3 H A : At least two of the µ s differ where µ i ( Population Mean of all Brand i light bulbs ) REMARK: More about experimental design in later sections. Josh Engwer (TTU) 1-Factor ANOVA / 32

7 PART II PART II: The Problem with Many-Sample t-tests Josh Engwer (TTU) 1-Factor ANOVA / 32

8 The Problem with Many-Sample t-tests Suppose a designed experiment calls to test four independent samples: H 0 : µ 1 = µ 2 = µ 3 = µ 4 H A : At least two of the µ s differ One way to do this is perform ( 4 2) independent t-tests, each at signif. level α: H (1) 0 : µ 1 = µ 2 H H (1) A : µ 1 µ 2 (2) 0 : µ 1 = µ 3 H A : µ 1 µ 3 H (2) (3) 0 : µ 1 = µ 4 H (3) A : µ 1 µ 4 H (4) (5) (6) 0 : µ 2 = µ 3 H H (4) 0 : µ 2 = µ 4 H A : µ 2 µ 3 H (5) 0 : µ 3 = µ 4 A : µ 2 µ 4 H (6) A : µ 3 µ 4 Josh Engwer (TTU) 1-Factor ANOVA / 32

9 The Problem with Many-Sample t-tests Suppose a designed experiment calls to test four independent samples: H 0 : µ 1 = µ 2 = µ 3 = µ 4 H A : At least two of the µ s differ Alas, since each successive t-test is performed with the same dataset, the experiment-wise significance level, α exp, grows with each t-test: α exp := P(Committing a Type I Error in at least one t-test) = 1 P(Never Committing a Type I Error in any of the t-tests) ( 6 ) = 1 P i=1 (Not Committing a Type I Error in ith t-test) IND = 1 6 i=1 P(Not Committing a Type I Error in ith t-test) α = 1 6 i=1 (1 α) = 1 (1 α) 6 [ ( )] α := P Rejecting H (k) 0 H (k) 0 is True Josh Engwer (TTU) 1-Factor ANOVA / 32

10 The Problem with Many-Sample t-tests H 0 : µ 1 = µ 2 = µ 3 = µ 4 H A : At least two of the µ s differ N t-tests (# t-tests) = Alas, with successive t-tests, α exp grows (AKA α-inflation): Chosen α Resulting α exp = 1 (1 α) ( ) 4 = 6 2 One can determine which α achieves a desired α exp, but often it s not feasible: Required α = 1 (1 α exp ) 1/6 Desired α exp A loose (rough) upper bound for α exp is α (# t-tests): α exp αn t-tests Josh Engwer (TTU) 1-Factor ANOVA / 32

11 The Problem with Many-Sample t-tests To prevent α-inflation, all means should be simultaneously tested. Josh Engwer (TTU) 1-Factor ANOVA / 32

12 PART III PART III: 1-Factor Analysis of Variance (ANOVA) Josh Engwer (TTU) 1-Factor ANOVA / 32

13 1-Factor ANOVA Assumptions In order for the forthcoming ANOVA test to bear good statistical properties, certain assumptions regarding the samples & populations must be imposed (in a similar fashion to 2-sample t-tests & F-tests): Proposition (1-Factor ANOVA Assumptions 1FAA) The following assumptions are required in order to use 1-Factor ANOVA: (Randomization) All measurements are randomly selected. (Independence) All measurements are independent. (Normality) All populations are approximately normally distributed. (Same Spread) All populations have approximately same variance. Mnemonic Device: RINSS Josh Engwer (TTU) 1-Factor ANOVA / 32

14 1-Factor ANOVA Assumptions In order for the forthcoming ANOVA test to bear good statistical properties, certain assumptions regarding the samples & populations must be imposed (in a similar fashion to 2-sample t-tests & F-tests): Proposition (1-Factor ANOVA Assumptions 1FAA) Given an experiment with I > 2 groups, each of size J. Let random variables X ij j th measurement in the i th group. Then the following assumptions are required in order to use 1-Factor ANOVA: X ij iid Normal(µ i, σ 2 ) All groups are independent of one another. Josh Engwer (TTU) 1-Factor ANOVA / 32

15 1-Factor Analysis of Variance (Motivation) High variance between groups Low variance within groups s 2 between /s2 within 1 = Factor A clearly has a significant effect!! Josh Engwer (TTU) 1-Factor ANOVA / 32

16 1-Factor Analysis of Variance (Motivation) Low variance between groups High variance within groups s 2 between /s2 within 1 = Factor A clearly has no significant effect! Josh Engwer (TTU) 1-Factor ANOVA / 32

17 1-Factor Analysis of Variance (Motivation) Low variance between groups Low variance within groups s 2 between /s2 within 1 = Hard to tell if factor A has a significant effect... Josh Engwer (TTU) 1-Factor ANOVA / 32

18 1-Factor Analysis of Variance (Motivation) High variance between groups High variance within groups s 2 between /s2 within 1 = Hard to tell if factor A has a significant effect... Josh Engwer (TTU) 1-Factor ANOVA / 32

19 1-Factor ANOVA Test Statistic The preceding four slides suggest the natural statistic is the F-Test Statistic: Proposition (Best Test Statistic Value for 1-Factor ANOVA) Given an experiment with one factor and I > 2 groups. Moreover, suppose the ANOVA assumptions are all satisfied. Then, the F-test using the following test statistic value: f = s2 between s 2 within is the most-powerful test that prevents α-inflation for hypotheses: H 0 : H A : µ 1 = µ 2 = = µ I At least two of the µ s differ The fact that the fitting statistic involves variances is why this procedure is called ANalysis Of VAriance even though we want to compare means. Josh Engwer (TTU) 1-Factor ANOVA / 32

20 s 2 between in terms of a Mean Square & Sum of Squares Proposition Given a 1-factor experiment involving I groups, each of size J. Then the variance between groups, s 2 between, is the variance of sample consisting of the I group means x i scaled by common treatment size J: s 2 between MS A := SS A := J i (x i x ) 2 ν A I 1 where the grand mean, x, is the mean of the I group means, x i : x := 1 I i x i = 1 IJ i j x ij In essence, a large variance between groups indicates much of the observed variation is explained by the chosen Factor A hence, subscript A. SS A and ν A are used later for computing F-cutoffs/P-values and interpretation. Josh Engwer (TTU) 1-Factor ANOVA / 32

21 s 2 within in terms of a Mean Square & Sum of Squares Proposition Given a 1-factor experiment involving I groups, each of size J. Then the variance within groups, s 2 within, is the mean of the group variances: s 2 within MS err := 1 I i s2 i = (J 1) i s2 i I(J 1) := SS err ν err Effectively, a large variance within the groups indicates that much of the observed variation is not explained by the chosen Factor A. Therefore, the within variance is considered unexplained error in the experimental design. SS err and ν err are used later for finding F-cutoffs/P-values and interpretation. Josh Engwer (TTU) 1-Factor ANOVA / 32

22 Sums of Squares as a Partitioning of Variation Explanation SS total }{{} Total Variation in Experiment = SS A }{{} Variation due to Factor A + SS err }{{} Unexplained Variation ν }{{} Total dof s in Experiment = ν A }{{} Between Groups dof s + ν err }{{} Within Groups dof s ν = IJ 1 ν A = I 1 ν err = I(J 1) Josh Engwer (TTU) 1-Factor ANOVA / 32

23 F-Test Statistic Value in terms of Mean Squares We can now express the F-Test statistic value in terms of mean squares: Proposition (Test Statistic Value for 1-Factor ANOVA in terms of Mean Squares) f A = s2 between s 2 within = MS A MS err The test statistic value for 1-Factor ANOVA will be denoted f A instead of f. In terms of the F-test notation in section 9.5, f A is always f +. The following slides explain why this is always the case for ANOVA. Josh Engwer (TTU) 1-Factor ANOVA / 32

24 MS err & MS A as Point Estimators of σ 2 Proposition (Point Estimation of Mean Squares PEMS) Given I size-j random samples satisfying ANOVA assumptions. Then: (i) MS err is always an unbiased point estimator of the population variance: H 0 is indeed true OR H 0 is indeed false = E[MS err ] = σ 2 (ii) If the status quo prevails, MS A is an unbiased estimator of pop. variance: H 0 is indeed true = E[MS A ] = σ 2 (iii) If the status quo fails, MS A tends to overestimate population variance: H 0 is indeed false = E[MS A ] > σ 2 Josh Engwer (TTU) 1-Factor ANOVA / 32

25 MS err & MS A as Point Estimators of σ 2 (Proof) Proposition (Point Estimation of Mean Squares PEMS) Given I size-j random samples satisfying ANOVA assumptions. Then: (i) MS err is always an unbiased point estimator of the population variance: H 0 is indeed true OR H 0 is indeed false = E[MS err ] = σ 2 (ii) If the status quo prevails, MS A is an unbiased estimator of pop. variance: H 0 is indeed true = E[MS A ] = σ 2 (iii) If the status quo fails, MS A tends to overestimate population variance: PROOF: (i) E[MS err ] := E [ SSerr ν err ] = E H 0 is indeed false = E[MS A ] > σ 2 [ J 1 I(J 1) i S2 i = 1 I i σ2 = 1 I Iσ2 = σ 2 ] = E [ 1 I i ] S2 i = 1 I i E [ ] Si 2 Josh Engwer (TTU) 1-Factor ANOVA / 32

26 MS err & MS A as Point Estimators of σ 2 (Proof) Proposition (Point Estimation of Mean Squares PEMS) Given I size-j random samples satisfying ANOVA assumptions. Then: (i) MS err is always an unbiased point estimator of the population variance: H 0 is indeed true OR H 0 is indeed false = E[MS err ] = σ 2 (ii) If the status quo prevails, MS A is an unbiased estimator of pop. variance: H 0 is indeed true = E[MS A ] = σ 2 (iii) If the status quo fails, MS A tends to overestimate population variance: PROOF: H 0 is indeed false = E[MS A ] > σ 2 (ii) Suppose H 0 is true. Then, X i Normal(µ Xi = µ, σ 2 = σ 2 /J) [ ] [ X i SS E[MS A ] := E A J ν A = E I 1 ] ] i (X i X ) 2 := E [J S 2Xi = J σ2 J = σ 2 Josh Engwer (TTU) 1-Factor ANOVA / 32

27 MS err & MS A as Point Estimators of σ 2 (Proof) Proposition (Point Estimation of Mean Squares PEMS) Given I size-j random samples satisfying ANOVA assumptions. Then: (i) MS err is always an unbiased point estimator of the population variance: H 0 is indeed true OR H 0 is indeed false = E[MS err ] = σ 2 (ii) If the status quo prevails, MS A is an unbiased estimator of pop. variance: H 0 is indeed true = E[MS A ] = σ 2 (iii) If the status quo fails, MS A tends to overestimate population variance: PROOF: H 0 is indeed false = E[MS A ] > σ 2 (iii) (Deferred to section 10.3 when we introduce more flexible notation.) Josh Engwer (TTU) 1-Factor ANOVA / 32

28 1-Factor ANOVA (Given Cell Means x i & SD s s i ) 1 Determine df s: ν A = I 1, ν err = I(J 1) 2 Compute Grand Mean: x = 1 I i x i 3 Compute SS err = (J 1) i s2 i 4 Compute SS A = J i (x i x ) 2 5 Compute Mean Squares: MS err := SSerr ν err, MS A := SSA ν A 6 Compute Test Statistic Value: f A = MSA MS err 7 Compute F-cutoff/P-value: By hand, lookup fν A,ν err;α By SW, compute p A = 1 Φ F (f A ; ν A, ν err ) 8 Render Decision: If If f A fν A,ν err;α p A α, then reject H0 A; else accept HA 0., then reject H0 A; else accept HA 0. 1-Factor ANOVA Table (Significance Level α) Variation Sum of Mean F Stat df P-value Decision Source Squares Square Value Factor A ν A SS A MS A f A p A Acc/Rej H0 A Error ν err SS err MS err Total ν SS total Josh Engwer (TTU) 1-Factor ANOVA / 32

29 1-Factor ANOVA (Given Cell Means x i & ESE s σ xi ) 1 Determine df s: ν A = I 1, ν err = I(J 1) 2 Compute Grand Mean: x = 1 I i x i 3 Compute Group Std. Dev s: s i = J σ xi 4 Compute SS err = (J 1) i s2 i 5 Compute SS A = J i (x i x ) 2 6 Compute Mean Squares: MS err := SSerr ν err, MS A := SSA ν A 7 Compute Test Statistic Value: f A = MSA MS err 8 Compute F-cutoff/P-value: By hand, lookup fν A,ν err;α By SW, compute p A = 1 Φ F (f A ; ν A, ν err ) 9 Render Decision: If If f A fν A,ν err;α p A α, then reject H0 A; else accept HA 0., then reject H0 A; else accept HA 0. 1-Factor ANOVA Table (Significance Level α) Variation Sum of Mean F Stat df P-value Decision Source Squares Square Value Factor A ν A SS A MS A f A p A Acc/Rej H0 A Error ν err SS err MS err Total ν SS total Josh Engwer (TTU) 1-Factor ANOVA / 32

30 1-Factor ANOVA (Given Observations x ij ) 1 Determine df s: ν A = I 1, ν err = I(J 1) 2 Compute Cell Means: x i := 1 J j x ij for i = 1,, I 3 Compute Cell Variances: s 2 i := 1 J 1 j (x ij x i ) 2 4 Compute Grand Mean: x = 1 I i x i 5 Compute SS err = (J 1) i s2 i 6 Compute SS A = J i (x i x ) 2 7 Compute Mean Squares: MS err := SSerr ν err, MS A := SSA ν A 8 Compute Test Statistic Value: f A = MSA MS err 9 Compute F-cutoff/P-value: By hand, lookup fν A,ν err;α By SW, compute p A = 1 Φ F (f A ; ν A, ν err ) 10 Render Decision: If If f A fν A,ν err;α p A α, then reject H0 A; else accept HA 0., then reject H0 A; else accept HA 0. 1-Factor ANOVA Table (Significance Level α) Variation Sum of Mean F Stat df Source Squares Square Value P-value Decision Factor A ν A SS A MS A Error ν err SS err MS err f A p A Acc/Rej H0 A Total ν SS total Josh Engwer (TTU) 1-Factor ANOVA / 32

31 Textbook Logistics for Section 10.1 Difference(s) in Terminology: Difference(s) in Notation: TEXTBOOK TERMINOLOGY: Treatment/Cell SLIDES/OUTLINE TERMINOLOGY: Group CONCEPT TEXTBOOK SLIDES/OUTLINE NOTATION NOTATION Probability of Event P(E) P(E) Expected Value E(X) E[X] Variance V(X) V[X] Sum of Squares of Factor A SSTr SS A Mean Square of Factor A MSTr MS A Sum of Squares of Error SSE SS err Mean Square of Error MSE MS err Null Hypothesis for Factor A H 0 Alt. Hypothesis for Factor A H A H0 A HA A Josh Engwer (TTU) 1-Factor ANOVA / 32

32 Fin Fin. Josh Engwer (TTU) 1-Factor ANOVA / 32