1
What If There Are More Than Two Factor Levels? The t-test does not directly apply ppy There are lots of practical situations where there are either more than two levels of interest, or there are several lfactors of simultaneous interest t The analysis of variance (ANOVA) is the appropriate analysis engine for these types of experiments The ANOVA was developed by Fisher in the early 1920s, and initially applied to agricultural experiments Used extensively today for industrial experiments 2
An Example (See pg. 61) An engineer is interested in investigating the relationship between een the RF power setting and the etch rate for this tool. The objective of an experiment like this is to model the relationship between etch rate and RF power, and to specify the power setting that will give a desired target etch rate. The response variable is etch rate. She is interested in a particular gas (C2F6) and gap (0.80 cm), and wants to test four levels of RF power: 160W, 180W, 200W, and 220W. She decided to test five wafers at each level of RF power. The experimenter chooses 4 levels of RF power 160W, 180W, 200W, and 220W The experiment is replicated 5 times runs made in random order 3
4
An Example (See pg. 62) 5
Does changing the power change the mean etch rate? Is there an optimum level for power? We would like to have an objective way to answer these questions The t-test test really doesn t apply here more than two factor levels 6
The Analysis of Variance (Sec. 3.2, pg. 62) In general, there will be a levels of the factor, or a treatments, and n replicates of the experiment, run in random order a completely randomized design (CRD) N = an total runs We consider the fixed effects case the random effects case will be discussed later Objective is to test hypotheses about the equality of the a treatment means 7
The Analysis of Variance The name analysis of variance stems from a partitioning of the total variability in the response variable into components that are consistent with a model for the experiment The basic single-factor ANOVA model is i 1,2,...,,, a, j 1,2,..., n y ij i ij ij an overall mean, ith treatment effect, 2 experimental error, NID(0, ) i 8
Models for the Data There are several ways to write a model for the data: yij i ij is called the effects model Let, then i i y is called the means model ij i ij Regression models can also be employed 9
The Analysis of Variance Total variability is measured by the total sum of squares: a n SS ( y y ) T i 1 j 1 The basic ANOVA partitioning is: a n a n 2 2 ( yij y.. ) [( yi. y.. ) ( yij yi. )] i 1 j 1 i 1 j 1 ij.. 2 a a n 2 2 (...) i ( ij i. ) i 1 i 1 j 1 n y y y y SS SS SS T Treatments t E 10
The Analysis of Variance SS SS SS T Treatments E A large value of SS Treatments reflects large differences in treatment means A small value of SS Treatments likely indicates no differences in treatment t t means Formal statistical hypotheses are: H H 0 1 2 1 : a : At least one mean is different 11
The Analysis of Variance While sums of squares cannot be directly compared to test the hypothesis of equal means, mean squares can be compared. A mean square is a sum of squares divided by its degrees of freedom: MS dftotal dftreatments dferror an 1 a 1 a( n 1) SSTreatments SSE, MSE a 1 a( n 1) Treatments If the treatment means are equal, the treatment and error mean squares will be (theoretically) equal. If treatment means differ, the treatment mean square will be larger than the error mean square. 12
The Analysis of Variance is Summarized in a Table Computing see text, pp 69 The reference distribution ib ti for F 0 is the F a-1, a(n-1) distribution ib ti Reject the null hypothesis (equal treatment means) if F 0 F, a 1, a( n 1) 13
14
Example: Etch Rate Data (in Å/min) from the Plasma Etching Experiment RF Power Observed Etch Rate Totals Average (W) 1 2 3 4 5 yi. yi.bar 160 575 542 530 539 570 2756 551.2 180 565 593 590 579 610 2937 587.4 220 600 651 610 637 629 3127 625.4 220 725 700 715 685 710 3535 707 15
ANOVA Table Example 3-1 16
The Reference Distribution: P-value 17
ANOVA calculations may be made simpler by coding the observation. Observation may be coded by subtracting each data by 600. We get: Based on the data above, the sum of squares may be computed in the usual manner. The sum of squares are similar to that calculated previously. Therefore subtracting a constant from the original data does not change the sum of squares. DOX 6E Montgomery 18
Model Adequacy Checking in the ANOVA Text reference, Section 3.4, pg. 75 Checking assumptions is important t Normality Constant variance Independence Have we fit the right model? Later we will talk about what to do if some of these assumptions are violated 19
The normality assumption could be verified by constructing a normal probability plot of the residuals. If the underlying error distribution is normal, this plot will resemble a straight line. DOX 6E Montgomery 20
Model Adequacy Checking in the ANOVA Examination of residuals (see text, Sec. 3-4, pg. 75) e y yˆ ij ij ij y y ij i. Computer software generates the residuals Residual plots are very useful Normal probability plot of residuals 21
Plot of residuals in time sequence Plotting the residuals in time order of data collection is helpful in detecting correlation between the residuals. A tendency to have runs of positive and negative residuals indicates positive correlation. This would imply that the independence assumption on the errors has been violated. Important to prevent the problem and proper randomization of the experiment is an important step in obtaining independence. DOX 6E Montgomery 22
Skill of the experimenter may improve as the experiment progresses or the process may drift. This will often result in a change in the error variance over time depicted by more spread at one end than at the other. DOX 6E Montgomery 23
Plot of residuals versus fitted values If the model is correct and if the assumptions are satisfied, the residuals should be structureless. A simple check is to plot the residuals versus the fitted values. This plot should not reveal any obvious pattern as shown in Fig. 3.6. Nonconstant variance may show up on this plot in a form of an outward opening funnel or megaphone. One way in dealing with nonconstant variance is to apply a variance stabilizing transformation DOX 6E Montgomery 24
The Regression Model 25
Post-ANOVA Comparison of Means The analysis of variance tests the hypothesis of equal treatment means Assume that residual analysis is satisfactory If that hypothesis is rejected, we don t know which specific means are different Determining which specific means differ following an ANOVA is called the multiple comparisons problem There are lots of ways to do this see text, Section 3.5, pg. 84 We will use pairwise t-tests tests on means sometimes sometimes called Fisher s Least Significant Difference (or Fisher s LSD) Method 26
Design-Expert Output 27
Graphical Comparison of Means Text, pg. 88 28
ANOVA calculations are usually done via computer Text exhibits sample calculations from three very popular software packages, Design-Expert Expert, JMP and Minitab See pages 98-100 Text discusses some of the summary statistics provided by these packages 29
Figure 3.12 (p. 99) Design-Expert computer output for Example 3-1. R 2 SS Model SS Total 66870.55 72209.7575 0.9261 R 2 is loosely interpreted as the proportion of the variability in the data explained by the ANOVA model. Values closer to 1 is desirable. Adjusted R 2 reflects the number of factors in the model and useful for evaluating the impact of increasing or decreasing the number of model terms. Others see text pg. 98 DOX 6E Montgomery 30
Figure 3.13 (p. 100) Minitab computer output for Example 3-1. DOX 6E Montgomery 31
Why Does the ANOVA Work? We are sampling from normal populations, so SS SS if H is true, and Cochran's theorem gives the independence of Treatments 2 E 2 2 a 1 0 2 a ( n 1) these two chi-square random variables 2 SSTreatments /( a 1) a 1 /( 1) a So F0 F 2 a 1, a( n 1) SSE /[ a( n 1)] a( n 1) /[ a( n 1)] n 2 n i 2 i 1 2 EMSE Finally, EMS ( Treatments ) and ( ) a 1 Therefore an upper-tail F test is appropriate. 32
Sample Size Determination Text, Section 3.7, pg. 101 FAQ in designed experiments Answer depends on lots of things; including what type of experiment is being contemplated, how it will be conducted, resources, and desired sensitivity Sensitivity refers to the difference in means that the experimenter wishes to detect Generally, increasing the number of replications increases the sensitivity or it makes it easier to detect small differences in means 33
Sample Size Determination Fixed Effects Case Can choose the sample size to detect a specific difference in means and achieve desired values of type I and type II errors Type I error reject H 0 when it is true ( ) Type II error fail to reject H 0 when it is false ( ) Power =1 - Operating characteristic curves plot against a parameter where n a 2 i 1 2 a 2 i 34
Sample Size Determination Fixed Effects Case---use of OC Curves The OC curves for the fixed effects model are in the Appendix, Table V A very common way to use these charts is to define a difference in two means D of interest, then the minimum value of 2is 2 2 nd 2 2a Typically work in term of the ratio of D / and try values of n until the desired d power is achieved Most statistics software packages will perform power and sample size calculations see page 103 There are some other methods discussed d in the textt 35
Power and sample size calculations from Minitab (Page 103) 36
37
38
39
40
41
Question to be attempted No. 3.1, 3.3, 3.6, 3.9 (No. 37 3.7, 38 3.8, 311 3.11, 35) 3.5) DOX 6E Montgomery 42