Topics in Experimental Design

Transcription

1 Ronald Christensen Professor of Statistics Department of Mathematics and Statistics University of New Mexico Copyright c 2016 Topics in Experimental Design Springer

2

3

4 Preface An extremely useful concept in experimental design is the use of treatments that have factorial structure. For example, if you are interested in two things, say the effect of alcohol and the effect of sleeping pills, rather than performing separate experiments on each, you can incorporate both factors into the treatments of a single experiment. Briefly, suppose we are interested in two levels of alcohol, say, a 0 no alcohol and a 1 a standard dose of alcohol, and we are also interested in two levels of sleeping pills, say, s 0 no sleeping pills and s 1 a standard dose of sleeping pills. A factorial treatment structure involves forming 4 treatments, a 0 s 0 no alcohol, no sleeping pills, a 0 s 1 no alcohol, sleeping pills, a 1 s 0 alcohol, no sleeping pills, a 1 s 1 alcohol and sleeping pills. There are two benefits to doing this. First, if there is no interaction, i.e., if the effect of alcohol does not depend on whether or not the subject has taken sleeping pills, you can learn as much from running one experiment using factorial treatment structure on, say, 20 people as you can from two separate experiments, one for alcohol and one for sleeping pills, each involving 20 people. This is a 50% savings in the number of observations needed. Second, if interaction exist, i.e., if the effect of alcohol depends on the amount of sleeping pills a person has taken, you can study that interaction in an experiment with factorial treatment structure but you cannot study interaction in an experiment that was solely devoted to looking at the effects of alcohol. An experiment such as this involving two factors each at two levels is referred to as a 2 2 = 2 2 factorial treatment structure. Note that 4 is the number of treatments we end up with. If we had 3 levels of sleeping pills, it would be a 2 3 factorial, thus giving 6 treatments. If we had three levels of both alcohol and sleeping pills it would be a 3 3 = 3 2. If we had three factors, say, alcohol, sleeping pills, and benzedrine each at 2 levels, we would have a = 2 3 structure. If each factor were at 3 levels, say, none of the drug, a standard dose, and twice the standard dose, and we made up our treatments by taking every combination of the levels of each factor, we would get = 3 3 = 27 treatments. See Christensen (1996b, Chapter 11 or 2015, Chapter?) for more on factorial treatments. Mostly this monograph consists of things that got left out of other books. vii

5 viii Ronald Christensen Albuquerque, New Mexico November 12, 2016 BMDP Statistical Software is located at 1440 Sepulveda Boulevard, Los Angeles, CA 90025, telephone: (213) MINITAB is a registered trademark of Minitab, Inc., 3081 Enterprise Drive, State College, PA 16801, telephone: (814) , telex: MSUSTAT is marketed by the Research and Development Institute Inc., Montana State University, Bozeman, MT , Attn: R.E. Lund.

6 Contents Preface vii 1 Screening Designs Introduction Designs for Two Levels Blocking Hare s Full model Construction of Hadamard Matrices Designs for Three Levels Definitive Screening Designs Some Linear Model Theory Weighing out of conference Mixed Levels Experiment on Variability Notes Confounding and Fractional Replication: 2 f Factorial Systems Confounding Fractional replication Analysis of unreplicated experiments More on graphical analysis Augmenting designs for factors at two levels Exercises p f Factorial Treatment Structures f Factorials Column Space Considerations Confounding and Fractional Replication Confounding Fractional Replication Experiment on Variability ix

7 x Contents 3.4 Analysis of a Confounded The Expanded ANOVA Table Interaction Contrasts A DSD f Factorials Further Extensions Mixtures of Prime Powers Powers of Primes Response Surface Maximization Approximating Response Functions First-Order Models and Steepest Ascent Fitting Quadratic Models Interpreting Quadratic Response Functions Recovery of Interblock Information in BIB Designs Estimation Model Testing Contrasts Alternative Inferential Procedures Estimation of Variance Components References Index

8 Chapter 1 Screening Designs Three fundamental ideas in experimental design are replication, blocking and factorial treatment structure. Replication is required so that we have a measure of the variability of the observations. Blocking is used to reduce experimental variability. Blocking has inspired a number of standard designs including randomized complete blocks, Latin squares, balanced incomplete blocks, Youden squares, balanced lattice designs, and partially balanced incomplete blocks. The point of using factorial treatment structures is that they allow one to look very efficiently at the main effects of the factors but also that they allow examination of interactions between the factors. In this chapter we examine designs for looking efficiently at main effects without observing the entire set of factorial treatments. We will see that blocking can be accomplished by treating blocks as additional factors in the experiment. Replication tends to get short shrift in screening designs. It largely consists of pretending that interaction does not exist and using estimates of the nonexistent interaction to estimate variability. In Chapter 2 we will look at some methods for analyzing data without replications. In general, I contend that if interaction exists, it is the only thing worth looking at. In general, if interaction exists, main effects have no useful meaning. In particular, you could have an interaction between two factors, say alcohol and sleeping pills, where responses are low when you use both alcohol and sleeping pills or use neither alcohol nor sleeping pills but responses are high when you use either one but not the other. In such a case, neither the main effect for alcohol nor the main effect for sleeping pills will look important, because their average effects are unimportant, despite the importance of knowing the exact combination of alcohol and sleeping pill used. Screening designs are based on the hope that these sorts of interactions will not occur, cf. Christensen et al. (2010, Subsection 7.4.7). It can be expensive to collect enough data to explore all of the possible interactions. If you want to save money, you have to give something up. Obviously, you should give up the things that you think are unlikely to be important. In the case of screening designs, that means giving up the ability to look at any interactions. As designs get more sophisticated, such as those discussed in Chapters 2 and 3, one can look at main effects and some interactions with relatively few observations. 1

9 2 1 Screening Designs 1.1 Introduction When the treatments have factorial structure, screening designs can be set up to examine all of the factors main effects efficiently but without the cost and trouble of examining all of the factorial treatments. Often, especially in industrial experiments, there are so many factors that using a complete factorial structure becomes prohibitive because there are just too many treatments to consider. For example, suppose we have 8 factors each at two levels. The number of factor combinations (treatments) is 2 8 = 256. That is a lot of treatments, especially if you plan to perform replications in order to estimate error. If you only want to estimate the 8 factorial main effects, in theory you can do that with as few as 9 observations. Nine observations means 9 degrees of freedom, which can be allocated as one for each factor s main effect and one for fitting the grand mean (intercept). EXAMPLE Consider the experiment reported by Hare (1988) with five factors each at two levels for 2 5 = 32 factor combinations (treatments). The issue is excessive variability in the taste of a dry soup mix. The source of variability was identified as a particular component of the mix called the intermix containing flavorful ingredients such as salt and vegetable oil. Intermix is made in a large mixer. Factor A is the number of ports for adding vegetable oil to the mixer. This was set at either 1 (a 0 ) or 3 (a 1 ). Factor B is the temperature of the mixer. The mixer can be cooled by circulating water through the mixer jacket (b 0 ) or the mixer can be used at room temperature (b 1 ). Factor C is the mixing time, 60 seconds (c 0 ) or 80 seconds (c 1 ). Factor D is the size of the intermix batch, either 1500 pounds (d 0 ) or 2000 pounds (d 1 ). Factor E is the delay between making the intermix and using it in the final soup mix. The delay is either 1 day (e 0 ) or 7 days (e 1 ). Table 1.1 contains a list of the 16 treatments that were actually examined out of the 32 possible treatments. The order in which the treatments were run was randomized and they are listed in that order. Batch number 7 contains the standard operating conditions. Because the issue in Hare s experiment is variability, the data collection is complicated. For each of the 16 batches of intermix, the original data are groups of 5 samples taken every 15 minutes throughout a day of processing. Thus each batch yields data for a balanced one-way analysis of variance with N = 5. The data actually analyzed are derived from the ANOVAs on different batches. There are two sources of variability in the original observations, the variability within a group of 5 samples and variability that occurs between 15 minute intervals. From the analysis of variance data, the within group variability is estimated with the MSE and summarized as the estimated capability standard deviation s c = MSE. The process standard deviation is defined as the standard deviation of an individual observation. The standard deviation of an observation incorporates both the between group and the within group sources of variability. The estimated process standard deviation is taken as

10 1.1 Introduction 3 Table 1.1 Hare s intermix variability data. Batch Treatment s c s p 1 a 0 b 0 c 0 d 1 e 1 de a 1 b 0 c 1 d 1 e 1 acde a 1 b 1 c 0 d 0 e 0 ab a 1 b 0 c 1 d 0 e 0 ac a 0 b 1 c 0 d 0 e 1 be a 0 b 0 c 1 d 0 e 1 ce a 0 b 1 c 0 d 1 e 0 bd a 1 b 1 c 1 d 1 e 0 abcd a 0 b 1 c 1 d 1 e 1 bcde a 1 b 1 c 0 d 1 e 1 abde a 0 b 0 c 1 d 1 e 0 cd a 0 b 0 c 0 d 0 e 0 (1) a 1 b 0 c 0 d 0 e 1 ae a 1 b 1 c 1 d 0 e 1 abce a 1 b 0 c 0 d 1 e 0 ad a 0 b 1 c 1 d 0 e 0 bc s p = MSE + MSTrts MSE, 5 where the 5 is the number of samples taken at each time, cf. Christensen (1996, Subsection or 2016, Subsection ). These two statistics, s c and s p, are available from every batch of soup mix prepared and provide the data as reported in Table 1.1 for analyzing batches. As alluded to earlier, with 5 factors each at two levels, in theory we could get estimates of all the treatment main effects from as little as 6 observations. Hare s experiment uses 16 observations for reasons that will be examined in the next chapter. In practice, the smallest number of observations for examining 5 factors each at two levels that has nice properties is 8. We will return to this issue at the end of the continuation of the example. There are many methods in common use for identifying the treatment levels. For experiments like Hare s, with 5 factors each at two levels we have denoted the treatments using lower case letters and subscripts. The lower case letters really just provide an ordering for the factor subscripts so we know that, say, the third subscript corresponds to the third factor, C. Given the ordering, the subscripts contain all the information about treatments. In other words, a 16 5 matrix of 0s and 1s identifies the treatments. Another convenient method of subscripting is to replace the 0s with 1s, and that is sometimes reduced to reporting just plus and minus signs. Yet another way of identifying treatments is to write down only the treatment letters that have a subscript of 1 (and not write down the subscripts). This last method also appears in Table 1.1.

11 4 1 Screening Designs A screening design focuses on main effects. We can get the main effect information out of the data by fitting the model Y = Xβ + e where X is the matrix of subscript values together with an initial column of 1s. EXAMPLE CONTINUED. For Hare s experiment, Y is one of the last two columns of Table 1.1 and X has a first column of 1s and then the rest of X consists of the treatment subscripts from Table 1.1, i.e., X = An equivalent but alternative method of writing the model matrix replaces the subscript 0 with the subscript 1, X =, (1)

12 1.1 Introduction 5 in a model Y = Xγ + e. The matrix X has the useful mathematical property that X X = 16I 6, which makes the linear model easy to analyze. In particular, the estimate of γ is ˆγ = (1/16) X Y. For analyzing the s p data, fitting either the X or X model gives. Fitting the X model gives Analysis of Variance for s p Source df SS MS F P Regression Residual Error Total Table of Coefficients for s p Predictor ˆγ k SE( ˆγ k ) t P Constant A B C D E Fitting the X model gives estimates and standard errors, other than the intercept, that are twice as large but gives the same t statistics and P values. Specifically, for k = 1,...,5, ˆβ k = 2 ˆγ k and SE( ˆβ k ) = 2SE( ˆγ k ). For, say, factor E, the estimated change in going from the low treatment level e 0 to the high treatment level is 2( 0.235) = with a standard error of 2( ) = Again regardless of the model, we can divide the SSReg into one degree of freedom for each main effect. Source df SS A B C D E These sums of squares, divided by the MSE, are equal to the square of the t statistics from the Table of Coefficients. Because of the special structure of X, unlike standard regression problems, neither the sums of squares nor the t statistics change if you drop any other main effects out of the model. From this analysis, factor E, the time waited before using the intermix, has a much larger sum of squares and a much larger t statistic than any of the other factors, so it would seem to be the most important factor. The design Hare used allows examination of not only the main effects but also of all the two-factor interactions, and we will see in the next chapter that two-factor interactions are important in these data.

13 6 1 Screening Designs We mentioned earlier that, in theory, estimating all the main effects in a 2 5 factorial treatment structure requires only 6 observations and that a good practical design can be obtained using only 8. The last 5 columns of the following model matrix determines one good design for evaluating just the five main effects X =. (2) In the next section we will examine where such designs originate. 1.2 Designs for Two Levels For experiments having all factors at two levels, Plackett and Burman (1946) proposed using normalized Hadamard matrices to define screening designs, i.e., groups of factorial treatments that provide nice estimates of the main effects for all factors. A Hadamard matrix H is an n n square matrix that consists of the numbers ±1 for which 1 n H is an orthonormal (more often called orthogonal) matrix. In other words, H H = HH = ni. Recall that permuting either the rows or columns of an orthonormal matrix gives another orthonormal matrix. The number of rows n in a Hadamard matrix needs to be 1, 2, or a multiple of 4 and even then it is not clear that Hadamard matrices always exist. One Hadamard matrix of order 8 is H =. (1) One Hadamard matrix of order 12 is

14 1.2 Designs for Two Levels H = A normalized Hadamard matrix has the form H = [J,T ], where J is a column of 1s. The submatrix T, or any subset of its columns, can be used to define treatments (treatment subscripts) for analysis of variance problems involving many factors each at two levels in which our interest lies only in main effects. For example, if we have f factors each at two levels, an f + 1 dimensional normalized Hadamard matrix H, if it exists, determines a set of treatments whose observation allows us to estimate all f of the main effects. Randomly associate each of the f factors with one of the f columns in T. T provides the subscripts associated with each treatment to be observed. Indeed, the Hadamard matrix becomes X in the linear model for main effects, Y = Xγ + e. This is a smallest design that allows us to estimate the grand mean and all f of the factor main effects. But remember, Hadamard matrices do not exist for all values f + 1. Except for the trivial case of f = 1, Hadamard matrixes only exist when f + 1 is a multiple of 4. More often we have f factors and choose n > f + 1 as the size of the normalized Hadamard matrix H. Again, excluding the initial column of 1s, randomly associate each factor with one of the remaining n 1 columns of H. From the Hadamard matrix, extract the matrix X = [J,T ] that consists of the column of 1s followed by (in any convenient order) the columns associated with the factors. Because H H = ni n, we have X X = ni f +1 and the perpendicular projection operator onto C( X) is M = 1 n X X. T provides the subscripts associated with the treatments to be observed. Assuming no interactions, the model Y = Xγ + e involves n observations, provides n r( X) = n ( f + 1) = n f 1 degrees of freedom for estimating the error, as well as provides estimates of the f main effects and the intercept. If we take n >> f + 1, we should be able to do much more than merely examine main effects, e.g., examine at least some interactions. But in general, it is difficult to know what more we can do, i.e., what interactions we can look at. In the next chapter, we will examine in detail the special case where n is a power of 2. In the special case, we can keep careful track of which interactions can be estimated and which cannot.

15 8 1 Screening Designs EXAMPLE In Hare s example, the matrix X in equation (1.1.1) defined the model matrix for the main-effects linear model and its last five columns defined the treatments used. X consists of the first 6 columns of the following normalized Hadamard matrix: H = At the end of the previous section we mentioned that, if examining main effects was the only goal, Hare could have gotten by with examining only the 8 factor combinations defined by the last five columns of X in equation (1.1.2). The matrix X consists of the first six columns of the Hadamard matrix in equation (1.2.1). If Hare had 7 factors each at two levels, a smallest (orthogonal) design for obtaining all main effects takes X equal to the Hadamard matrix in (1.2.1) [or some other normalized Hadamard matrix of the same size]. Alternatively, Hare could have stuck with 16 treatments and used, say, columns 2 through 8 of H to define the factor combinations. That would be a perfectly good design for looking only at main effects. But Hare was also interested in two-factor interactions and the 7th and 8th columns of H happen to be associated with the AB and AC interactions. (More on this later.) Using the 7th and 8th columns to help define treatments for 7 factors would mean loosing the ability to estimate the AB and AC interactions: estimating these interactions is something that Hare could do with only 5 factors but something that typically is not a priority in a screening design. Permuting the rows of a Hadamard matrix gives another Hadamard matrix which is, for our purposes, equivalent to the first. The rows define the treatments we want, and permuting the rows does not change the collection of treatments. Also, permuting the rows does not change that X X = ni f +1. We just have to make sure that we apply the same permutation to the rows of Y. We could also permute the columns of either H or X, as long as we remember what factor is associated with each column.

16 1.2 Designs for Two Levels Blocking Returning to Hare s experiment with 5 factors and 16 observations (factor combinations), suppose Hare had wanted to run the experiment in four blocks of size 4. The first 6 columns of H define the intercept and treatments, any other two columns of H could be used to define the four blocks of size 4. Let s use the last two columns to define blocks. The last two columns define pairs of numbers (1,1), (1, 1), ( 1,1), ( 1, 1) that will define the blocks. Deleting the columns that we are not using and rearranging the rows of H so that the pairs of numbers from the last two columns are grouped, and introducing some extra space to focus on the five columns that define the treatments, gives We can read off the blocking structure from this matrix. Block one consists of the the treatments (with ±1 subscripts) corresponding to rows of H in which the last two columns are (1,1). The subscripts come from the first four rows of the previous matrix, , so changing the 1 subscripts back to 0s, the treatments are a 1 b 0 c 1 d 1 e 1 a 1 b 1 c 0 d 0 e 0. a 0 b 0 c 0 d 0 e 0 a 0 b 1 c 1 d 1 e 1

17 10 1 Screening Designs In the second block the treatment subscripts correspond to rows of H where the last two columns are (1, 1): The third block has ( 1,1), And the last block has ( 1, 1), The model matrix for the main effects with blocking model can be taken as X = Here the second through fourth columns account for blocks and the last five columns account for factors A, B, C, D, E. The order of listing the treatments has changed from Table 1.1, so the row order of listing the variability measures s c and s p would also need to change. The physical act of blocking would almost certainly change the data from that observed in Table 1.1, but if the data from the blocked experiment were the same as those reported, the estimates and sums of squares for the main

18 1.2 Designs for Two Levels 11 effects would also remain the same. Blocking should change the estimate of error. The whole point of blocking is to isolate substantial effects due to blocks and remove them from the error that would have applied without blocking. If we wanted two blocks of size 8, we would have used only one column (not previously used for treatments or the intercept) of the Hadamard matrix to define blocks. If we wanted 8 blocks of size 2, we would have used three (not previously used) columns to define blocks. Now let s examine blocking with f = 5 factors and n = 8 factor combinations. In this example, the last two columns of the Hadamard matrix in equation (1.2.1) are used to define 4 blocks of size 2. Rearranging the rows of (1.2.1) gives from which we gets the blocks [ ] , [ [ [ Unfortunately, using these blocks will lose us the ability to look at the main effect for factor D because D is at the same level in every block. There are 8 observations, so 8 degrees of freedom. There are 4 degrees of freedom for the blocks and the intercept, which leaves only 4 degrees of freedom for estimating effects, but we have 5 effects to estimate, so we must lose something. Again, a major virtue of the approach in Chapter 2 is that it allows us to keep track of such things. ], ], ] Hare s Full model The Hadamard matrix H was (implicitly) used by Hare to determine a group of 16 treatments to examine from a 2 5 factorial structure; treatments that provide a clean

19 12 1 Screening Designs analysis of main effects. From our discussion in this chapter, H could have been used to define a design and a main-effects model for up to 15 factors. The normalized Hadamard matrix H was actually constructed using the methods of Chapter 2, which allows us to identify each of the 10 columns of the matrix not used in the main-effects model with a particular two-factor interaction. Fitting the linear model Y = Hδ + e fits the data perfectly, leaving 0 degrees of freedom for error. The trick, in using this model, is identifying what effect each column represents. The first 6 columns correspond to the intercept and the factor main effects, the other 10 columns are two-factor interactions and obtained by multiplying two main-effects columns elementwise. In other words, column 7 is the AB interaction column because it is obtained from multiplying column 2 (factor A) times column 3 (factor B) elementwise. In the first row, columns 2 and 3 take the values 1 and 1, so column 7 is 1 1 = 1. In the second row, columns 2 and 3 take the values 1 and 1, so column 7 is 1 1 = 1. In the last row, columns 2 and 3 are 1 and 1, so column 7 is 1 1 = 1. There are 10 distinct pairs of main effects, hence 10 two-factor intereactions. Many ANOVA and regression programs have this method, or an equivalent process, automated. Incidentally, in the next chapter we will see that the AB interaction effect is indistinguishable from the CDE interaction effect. Note that column 7 (AB) is 1 times the elementwise product of columns 4, 5, and 6; the columns associated with C, D, and E. Few computer programs allow fitting models with 0 dfe, so we deleted the last column of H before fitting the model. The last column corresponds to the DE interaction. Analysis of Variance for s p Source df SS MS F P Regression Residual Error Total The sums of squares can be broken down into 15 individual terms associated with main effects and two-factor interactions. These numbers are just the elements of the last 15 terms of the vector H Y, squared, and divided by n = 16. (We have ignored the contribution from the intercept.) Again, the Error term is labeled as DE. Source df SS Source df SS Source df SS A AB BD B AC BE C AD CD D AE CE E BC DE In the next chapter we will explore these associations. In particular, we will see that all of these terms are indistinguishable from higher-order interaction terms. In the main-effects only model, we noted that E looked to be the most important effect. While that remains true in this more expansive model, the sum of squares for BE is of a similar size to that of E and the sum of squares for DE is not inconsiderable.

20 1.2 Designs for Two Levels Construction of Hadamard Matrices Hadmard matrices have very nice design properties but you have to be able to find them. In practice, you just look them up. But there are a variety of ways to construct Hadamards. For an n n Hadamard to exist, n has to be 1, 2, or a multiple of 4. For n = 1, H = ±1. For n = 2, some Hadamards are [ ], [ ], [ ], [ Given any two Hadamard matrices, it is easy to see that their kronecker product is Hadamard: [H 1 H 2 ][H 1 H 2 ] = [H 1 H 2 ][H 1 H 2] ]. = [H 1 H 1 H 1 H 2] = [n 1 I n1 n 2 I n2 ] = n 1 n 2 I n1 n 2. Paley s method of constructing Hadamards uses Jacobsthal matrices. A q q Jacobsthal matrix Q has 0s on the diagonal, ±1s elsewhere, and has the properties: (a) QQ = qi q Jq q and (b) QJ q = J qq = 0. From (a) q 1QQ is the ppo onto C(J q ), so the property J qq = 0 must be true. Clearly, if Q is Jacobsthal, Q is also. If q mod 4 = 3, Q will be skew symmetric, i.e. Q = Q. In that case, an n = q+1 dimensional Hadamard matrix can be obtained from [ ] 1 J H = q J q Q I q or, equivalently, [ ] 0 J H = I n + q J q Q [ ] 1 J = q. J q Q + I q If q mod 4 = 1, Q will be symmetric and Hadamards of dimension n = 2(q + 1) can be constructed by replacing elements of [ ] 0 J q J q Q by certain 2 dimensional Hadamards. Finally, you can construct Hadamards from conference matrices. (In the next section we will find a different use for conference matrices.) A q q conference matrix has 0 on the diagonal, ±1 elsewhere, and C C = (q 1)I. Examples of conference matrices are C 1 =, C = ,

21 14 1 Screening Designs and C 3 = C 2 is skew symmetric and C 3 is symmetric. If you think about what it takes for the columns of a conference matrix to be orthogonal, it is pretty easy to see that the dimension q of a conference matrix has to be an even number. Skew symmetric conference matrices have q a multiple of 4, with the other even numbers giving symmetric conference matrices if they exist. For example, conference matrices are not known to exist for q = 22,34. If C is skew symmetric, a Hadamard matrix is H = I +C. If C symmetric, [ ] C + I C I H = C I C I is Hadamard. We are not going to consider how one goes about constructing either Jacobsthal and conference matrices. 1.3 Designs for Three Levels Screening designs are used as a first step in identifying important factors. They rely on the hope that any interactions that exist will not mask the important main effects. Suppose we have f factors each at 3 levels. As before, we use capital letters to denote factors and small letters to denote levels. With three levels, we have two useful subscripting options for identifying levels. We can identify the levels of factor A as a 0, a 1, a 2 or as a 1, a 0, a 1. The first subscripting option is used in Chapter 3 (because if facilitates modular arithmetic). The second option is used here. Screening designs focus on main effects but now a main effect has 2 degrees of freedom. In this section, we will assume that the three levels are quantitative levels, which means that we can associate the 2 main effect degrees of freedom with one linear term and one quadratic term. Moreover, we assume that the levels are equally spaced, so that the actual levels might just as well be the subscripts 1,0,1. Looking at the linear effect involves comparing the treatments with the 1 subscript to treatments with the 1 subscript. Often the linear effect is considered more important than the quadratic effect. In a comparable two-factor screening design, the linear effects are the main effects. We will consider designs that focus on the linear effect but also retain the ability to estimate the quadratic effect. In Chapter 3 we will discuss fractional replications of 3 f factorial structures. It is often possible to create designs that allow independent estimation of all main

22 1.3 Designs for Three Levels 15 effects that involve observing 3 2 = 9, or 3 3 = 27, or 3 4 = 81 factor combinations. The designs in the next subsection offer more flexibility in terms of numbers of observations but lose the independence of quadratic effects Definitive Screening Designs As discussed in the previous section, a q q conference matrix C has 0 on the diagonal, ±1 elsewhere, and C C = (q 1)I. As such, every row of a conference matrix can be identified with a factor combination. Jones and Nachtsheim (2011) introduced definitive screening designs (DSDs) that allow one to examine both the linear and quadratic main effects efficiently. The treatments are defined by the matrix T = C, where 0 indicates a 1 q row vector of 0s. Again, the rows of T consist of the subscripts for the factor combinations to be observed. The number of observations is obviously n = 2q+1. A linear-main-effects-only model is Y = [J,T ]β +e. A model with all main effects is Y = [J,T,T 2 ]γ + e wherein, if T [t i j ] n q, then T 2 [t 2 i j ]. The designs involve relatively few treatment levels with the subscript 0, so the linear main effects are estimated with more precision than the quadratic main effects. The definitive screening design for q factors has n = 2q + 1 observations with n effects to estimate, i.e., an intercept, q linear main effects, and q quadratic main effects. So the main-effects model is a saturated model with 0 degrees of freedom for Error. As discussed earlier, conference matrices have to have an even numbered dimension, so the available sizes are n = 5,9,13,17,21,25,29,33,37,41,49,..., i.e., one more than a multiple of 4. You might expect n = 45 in this list but recall that there is no conference matrix (known) for q = 22. One way to avoid having a saturated main-effects model is to use a DSD based on larger order conference matrices. For f = 5 factors, the following matrix uses the DSD design based on the 6 dimensional conference matrix given earlier but deleting the fourth column. C 0

23 16 1 Screening Designs T = This design will provide 2 degrees of freedom for error, or for looking at interactions Some Linear Model Theory Let T denote the matrix of treatment subscripts 1,0,1 for a design with f factors at 3 levels. Write T in terms of its columns, rows and elements as T = [T 1,...,T f ] = t 1. t n = [t i j ]. Also define T 2 to be the matrix consisting of the squares of the elements in T, i.e., T 2 [t 2 i j]. Note that, because all entries are 1,0,1, any column of T 2, say Tk 2, has the property that J Tk 2 = T k T k and this equals n minus the number of 0s in T k. The main effects linear model associated with this design is Y = [J,T,T 2 ] δ 0 δ 1 + e, δ 2 where δ 0 is a scalar with δ 1 and δ 2 being f vectors. After centering T 2, we get the equivalent linear model [ Y = J,T, (I 1n ] )T JJ 2 β 0 β 1 + e. (1) β 2

24 1.3 Designs for Three Levels 17 To estimate all of the parameters we need the columns of the model matrix to be linearly independent. Linear independence requires n 2 f + 1 but can break down even if that condition is satisfied. In particular, to estimate all of the quadratic effects, we need T to satisfy the following properties. (i) There must be at least one 0 in every column of T. (ii) For every ordered pair ( j,k), j,k = 1,..., f, j k, there exists an i such that t i j = 0 and t ik 0. If (i) fails, Tk 2 = J for anyt k that does not contain a 0. If (ii) fails for both ( j,k) and (k, j), Tj 2 = Tk 2, so we cannot estimate both quadratic main effects. Similar to Jones and Nachtsheim, let the q f matrix W define a set of q f treatments and then define T as W T = W 0 so that n = 2q+1 2 f +1, which is the number of mean parameters to be estimated. The row of 0s in T ensures that property (i) is satisfied. These designs seem most appropriate as screening designs when q is close to f. The nice analysis properties of these designs follows from the fact that the submatrices of the model matrix in (1) are all orthogonal. In particular, [J,T,(I 1n ] JJ )T [J,T,(I 2 1n ] JJ )T 2 = n W W T 2 ( I 1 n JJ ) T 2 (2) To have a truly orthogonal design, we need the matrix in (2) to be diagonal, not just block diagonal. Nonetheless, the block diagonal structure facilitates finding the matrix inverse, which is really more important to the analysis. Clearly, J J = n. From the definition of T in terms of W, clearly J T = 0 1 f and T T = W 0 W W 0 W = 2 W W. It is also immediate that J (I 1 n JJ )T 2 = 0. The more difficult task is to show that T (I 1 n JJ )T 2 = T T 2 = 0. In fact, we show that the linear main effects are orthogonal, not only to the square terms, but also to linear by linear interaction terms. Let (T k T j ) denote the vector that results from multiplying the vectors T k and T j elementwise. In particular, (T k T k ) = Tk 2. To estimate the linear by linear interaction term between factors k and j, one simply includes the vector (T k T j ) as a column of the model matrix. Orthogonality follows from the fact that,.

25 18 1 Screening Designs T s (T k T j ) = n i=1 = t is t ik t i j = q i=1 2q i=1 t is t ik t i j + t is t ik t i j = q i=1 q i=1 t is t ik t i j + 2q i=q+1 q ( t is )( t ik )( t i j ) = i=1 t is t ik t i j t is t ik t i j q i=1 t is t ik t i j = 0. A similar argument shows that T s (Tk 2T j 2 ) = 0, so the linear main effect terms are also orthogonal to the quadratic by quadratic interaction terms. Without additional conditions on W, the linear main effects do not appear to be orthogonal to linear by quadratic or quadratic by linear interactions. Typically we place additional conditions on W. In particular, a q f dimensional weighing matrix W takes values 1,0,1 and has WW = wi for some w. Hadamard matrices and conference matrices are both weighing. Choose f columns from W to define a submatrix W q f. Any such submatrix has W W = wi f which, when substituted into equation (2), shows the linear main effects to be orthogonal. We do not need to identify the original weighting matrix W, as long as we have the submatrix W. If W is a conference matrix, condition (ii) for estimability of quadratic effects holds. For conference matrices ( T 2 I 1 [ )T n JJ 2 = 2 I + q 4 ] 2q + 1 J f f. Unless q = 4 so that n = 9, the quadratic effects will not be orthogonal. This seems to be the only (useful) case in which a DSD is also a 3 f fractional replication as discussed in Chapter 3. [ ] Since (1/ f )J f f is a ppo, 2 I + q 4 2q+1 J f f is easily inverted using Christensen (2011, Proposition ), i.e., for a ppo P, [ai + bp] 1 = 1 [ I b ] a a + b P. The actual numbers that result from doing this do not strike me as particularly interesting other than that the correlations are nonzero and identical among the estimates of the quadratic coefficients. Jones and Nachtsheim (2011) indicate that the common correlation is f 1 but I have not been able to reproduce that result. Choices for W other than conference matrices are less promising. If W is a Hadamard matrix, (ii) does not hold as is shown in the next subsection. In fact, with a Hadamard matrix we would get only 1 degree of freedom for estimating all f of the quadratic effects. In the next subsection, we also construct a weighting matrix W, with two 0s in each row and column, which aliases pairs of quadratic effects.

26 1.3 Designs for Three Levels Weighing out of conference For general weighting matrices, T 2 ( I 1 n JJ ) T 2 = T 2 T 2 (2w)2 2q + 1 J f f. The result for conference matrices was given earlier. Hadamard matrices are weighing but give [ ] T 2 J f = 2q, 0 (I 1n JJ ) T 2 = [ 1n J f 2q (n 1) n J f 1 both of which are rank 1 matrices, hence our earlier comment about only 1 degree of freedom for estimating quadratic effects, and ( T 2 I 1 ) n JJ T 2 = n 1 n J f f. Next we look at a weighing matrix for which w = q 2 and T 2 ( I 1 n JJ ) T 2 is again singular and incapable of estimating all of the quadratic main effects. Consider W 1 = This skew symmetric W 1 was constructed as [C 2 H 0 ] where C 2 was the skew symmetric conference matrix given near the end of Section 2 and [ ] 1 1 H 0 = 1 1 is a symmetric Hadamard matrix. The problem is, if we square the elements of W 1 to get W1 2 [w2 1i j ], unlike a conference matrix, the consecutive pairs of vectors become identical. Thus we could not be able to tell apart the quadratic terms for factors A and B, or for factors C and D, etc. In particular, using W 1 as W in our design T, causes a violation of condition (ii) for estimating quadratic effects, with T 2 not having full rank and T 2 ( I n 1JJ ) T 2 being singular. We could solve this problem by using every other column of W in W, but at that point we might just as well save ourselves some observations and just use C 2 to define the design. ],

27 20 1 Screening Designs The multiplier w associated with a weighing matrix needs to be q minus the common number of 0s in each row. (The jth diagonal element of WW is the number of nonzero elements in the jth row.) Mathematically, Hadamard matrices are weighing matrices with w = q and conference matrices are weighing matrices with w = q 1. Conference matrices C are required to have 0 s down the diagonal. Permuting the rows of C gives a weighting matrix with one 0 in each row and column. For design purposes, a conference matrix and a weighing matrix with one 0 per row are equivalent. What is relevant to the design is the number of 0s in each row (or column) of W. As with Hadamard and conference matrices, there are serious mathematical questions about when weighing matrices exist, cf. Koukouvinos and Seberry (1997). In particular, it would be interesting to know if any weighing matrices exist with w < (q 1) that satisfy property (ii) and allow estimation of all quadratic effects. While perhaps not terribly practical, it would be of theoretical interest to examine the relationship between a possible DSD with n = 81 = 2(20) + 1 and the n = 81 = fractional replication associated with Chapter Mixed Levels Probably the most famous mixed level design is Taguchi s L 1 8 for one factor at 2 levels and 3 factors at three levels T = I have no idea how this was constructed. I ve wondered if they just pulled 9 observations out of a 3 3 or somehow doubled a 3 2. There is a little discussion of mixed levels in Chapter 3.

28 1.4 Mixed Levels 21 Let T 1 define an design for factors a 3 levels and T 2 define an design for factors at 2 levels, then a mixed level design is T = [T 1 T 2 ]. This is what Taguchi did in the next example Experiment on Variability Byrne and Taguchi (1989) and Lucas (1994) considered an experiment on the force y, measured in pounds, needed to pull a tubing from a connector. Large values of y are good. The controllable factors in the experiment are as follows. A: Interference (Low, Medium, High) B: Wall Thickness (Thin, Medium, Thick) C: Ins. Depth (Shallow, Medium, Deep) D: Percent Adhesive (Low, Medium, High) The data are given in Table 1.2. The design is a fractional factorial involving four factors each at three levels. Specifically, this is a 1/9th rep of a design. With 3 4 = 81 factor combinations and only 9 observations, we must have a 1/9th rep. Fractional replications for 3 n factorials are discussed in detail in the Chapter 3. For now, it suffices to note that the design allows one to estimate all of the main effects. It also only allows estimation of the main effects and assumes the absence of any interactions. This is characteristic of the designs typically suggested by Taguchi. He was only interested in main effects involving the control factors. The treatment matrix is T 1 = Taguchi had his favorite choices of W, but I am not aware that they have better properties than other choices that are available from Chapter 3. There are also three factors at two levels that can only be controlled with great effort. E: Condition Time (24hr, 120hr) F: Condition Temp. (72 F, 150 F) G: Condition R. H. (25%, 75%)

29 22 1 Screening Designs The data with all of the factors identified are presented One approach to analyzing the data in Table 1.2 is to ignore the information about the noise factors and treat the data associated with the noise factors as replications. We make use of our knowledge of the levels of the noise factors. Table 1.2 Taguchi design Controlled Noise Factors Run a b c d e 0 f 0 g 0 e 0 f 0 g 1 e 0 f 1 g 0 e 0 f 1 g 1 e 1 f 0 g 0 e 1 f 0 g 1 e 1 f 1 g 0 e 1 f 1 g The 2 3 design being used corresponds to three columns from an 8 dimensional Hadamard, T 2 = This design T 2 is not a screening design, it contains all the factor combinations. The overall design is T = {[J 8 T 1 ],[T 2 J 9 ]}. 1.5 Notes I spent a long time unsuccessfully trying to figure out, on my own, why Plackett- Burman designs were reasonable. It was only after Chris Nachtsheim give an excellent talk at UNM in 2017 on definitive screening designs that I was inspired to explore the basis of these designs and write this chapter. I found some of the course notes on Bill Cherowitzo s webpage useful (math.ucdenver.edu/ wcherowi) and a useful book is Stinson (2003). wcherowi/courses/m6406/m6406f.html wcherowi/courses/m6023/m6023f.html