Analysis of Designed Experiments

Transcription

1 8 Analysis f Designed Experiments 8.1 Intrductin In this chapter we discuss experiments whse main aim is t study and cmpare the effects f treatments (diets, varieties, dses) by measuring respnse (yield, weight gain) n plts r units (pints, subects, patients). In general the units are ften gruped int blcks (grups, sets, strata) f similar plts r units. We assume we have a respnse y which is cntinuus (fr example, a nrmal randm variable). We als have pssible explanatry variables which are discrete r qualitative, called factrs. Here we cnsider treatment factrs whse values are assigned by the experimenter. The values r categries f a factr are called levels, s that the levels f a treatment factr are labels fr the different treatments. Fr example, in cmparing engineering cmpnents frm several different manufacturers, we culd define indicatr variables t dente the manufacturer. If there are k different manufacturers, ne way t d it is t define { 1 if cmpnent i cmes frm manufacturer, x i = 0 therwise, (8.1) fr = 1, 2,..., k 1. We d nt need t define x ik because if x i1 =... = x i(k 1) = 0 then it autmatically fllws that cmpnent i is frm manufacturer k. It is pssible t use the regressr variables defined by 8.1, either n their wn r in cnunctin with ther cntinuus regressrs, t fit a linear regressin mdel using standard sftware. If that were all there was t it, we wuld nt need a separate thery fr the analysis f variance. Hwever, the specific issues raised by these mdels are such as t ustify and require a detailed treatment in their wn right. 8.2 Cmpletely Randmized Design We allcate r treatments randmly t the n sample units such that the ith treatment is allcated t n i units (i = 1, 2,..., r) (i.e. each treatment has n i replicatins). Nte there is n blcking here: very ften in experimental

2 Chapter 8. Analysis f Designed Experiments 389 design, the units are blcked int grups f similar units befre being allcated t treatments, but we are nt assuming that kind f experiment yet (see Sectin 7.3) all n = i n i units are regarded as similar and there is ne treatment factr with r levels. If the n i are all equal, then the experiment is called balanced. Nte that a cmpletely randmized design with r = 2 gives an experiment which is the same set-up as fr a tw-sample t- test (Appendix A). An experiment with tw treatments arranged in blcks, s that each blck cntains ne member assigned t the first treatment and ne member assigned t the secnd, crrespnds t a matched pairs t-test. Fr the cmpletely randmized design we use the ne-way mdel y i = µ + α i + ɛ i where = 1, 2,..., n i, i = 1, 2,..., r and r i=1 n i = n. In this mdel y i is the respnse f the th unit receiving the ith treatment, µ is an verall mean effect, α i is the effect due t the ith treatment and ɛ i is randm errr. Anther natural way t write this mdel is y i = µ i + ɛ i where = 1, 2,..., n i, i = 1, 2,..., r and r i=1 n i = n. The nly difference between these tw mdels is that µ i is used t dente the ttal respnse t treatment i. The first specificatin has the advantage that α i has a specific interpretatin as a treatment effect assciated with treatment i. We have the usual least squares assumptins that the ɛ i s have zer mean and cnstant variance σ 2 and are uncrrelated. In additin we usually make the nrmality assumptin that the ɛ i s are independent nrmal randm variables. Tgether these mean that and are independent. We may nte the fllwing: y i N(µ + α i, σ 2 ) 1. If treatment acts multiplicatively n the respnse, where y i = µα i ɛ i (where in this kind f mdel ɛ i > 0 fr all i and ), then the variance f y i is nn-cnstant, but by taking lgarithms f the respnse, the ne way mdel may then be valid. (The mdel assumes the variance is the same fr each treatment grup.) 2. Units, even in the same treatment grup, are assumed t be uncrrelated. 3. The riginal respnse may need t be transfrmed t achieve nrmality. Within this framewrk, sme f the questins that we might want t answer include (a) estimating the µ i s, r equivalently µ and the individual α i s,

3 390 Chapter 8. Analysis f Designed Experiments (b) estimating linear cmbinatins f the µ i s r α i s, especially cntrasts: linear cmbinatins f the frm i c iα i where i c i = 0, (c) testing equality f all the µ i s, r equivalently the hypthesis H 0 : α 1 =... = α r = 0. Theretical treatments f the analysis f variance ften fcus unduly n bective (c), thugh it is imprtant t remember that this is rarely an interesting bective in its wn right. It may well be a necessary preliminary t sme mre meaningful questin such as determining which f several drugs r agricultural varieties is the best, r whether any f a number f engineering manufacturers is differing unacceptably frm a predefined standard. We can write the mdel in matrix frm y = Xβ + ɛ, y 11 y 12.. y 1n1 y 21 y 22. y rnr µ. α = α α r ɛ 11 ɛ 12.. ɛ 1n1 ɛ 21 ɛ 22.. ɛ rnr. We can estimate the unknwn parameters β = (µ, α 1, α 2,..., α r ) by least squares r equivalently by maximum likelihd under the nrmality assumptin by slving the nrmal equatins (X T X) ˆβ = X T y. These are (r + 1) equatins in (r + 1) unknwns. We have n n 1 n 2... n r n 1 n X T X = n 2 0 n , n r n r i y i G y 1 X T y = y T 2 1 = T 2..., y r T r where G is the grand ttal and T i is the ttal fr the ith treatment.

4 Chapter 8. Analysis f Designed Experiments 391 The nrmal equatins are therefre r nˆµ + n i ˆα i = i=1 i n i ˆµ + n i ˆα i = y i, (8.2) y i i = 1, 2,..., r. (8.3) The equatins are nt independent as the sum ver i in (8.3) equals the ttal in (8.2), s there are nly r equatins fr r + 1 unknwns and there are infinitely many slutins. Hwever, this des nt prevent the mdel being estimated, because the fitted values are always the same whichever slutin is chsen and hence the residuals and residual sum f squares are the same. In rder t slve the equatins we have t add anther equatin r cnstraint n the estimates. Cnsider three pssible cnstraints: 1. Applying the cnstraint i n iα i = 0 imples that µ equals the verall mean, µ = ȳ = G n = yi. n Hence α i is estimated by ˆα i = ȳ i ȳ where ȳ i = y i/n i and ȳ = i y i/n. 2. A secnd methd is t set µ = 0, s that ˆα i = T i /n i = ȳ i 3. Take α 1 = 0. This is the slutin adpted by sme statistical packages e.g. GLIM. It fllws that µ = ȳ 1. and hence that α i = ȳ i ȳ 1. fr i = 2, 3,..., n. Fr each f the three slutins the fitted values (estimated means) are ˆµ + ˆα i = ȳ i fr a unit in the ith treatment grup. S the fitted values are identical, as are the residuals, e i = y i ȳ i, the residual sum f squares (SSE r deviance) and hence s 2. The residual sum f squares is given by SSE = (y i ȳ i ) 2. i

5 392 Chapter 8. Analysis f Designed Experiments The ANOVA Table As in Chapters 2 and 3, a cnvenient way t represent the results f an analysis f variance is thrugh a table knwn as the ANOVA table. Althugh this is mathematically ust a special case f the general develpment in Sectin 3.5, it is useful t re-derive the results directly fr this mdel. The ttal (crrected) sum f squares is SST O = i (y i ȳ ) 2, which is als knwn as the deviance after fitting the null mdel y i = µ+ɛ i, i.e. a mdel with n treatment effects, α 1 = α 2 =... = α r = 0. The ttal sum f squares SST O can be decmpsed: since (y i ȳ ) 2 i = i = i = i Hence [(y i ȳ i ) + (ȳ i ȳ )] 2 (y i ȳ i ) 2 + i (y i ȳ i ) 2 + i n i (ȳ i ȳ ) i n i (ȳ i ȳ ) 2 n i =1 (y i ȳ i ) = 0. SST O = SSE + SST R where SSE is the deviance under the full mdel y i = µ + α i + ɛ i. (ȳ i ȳ ) (y i ȳ i ) The SST R is the sum f squares due t treatments, analgus t SSR in an ANOVA table fr regressin. It is the difference in deviances between the null mdel and the full mdel, in ther wrds, the extra sum f squares due t the treatment effects r the increase in deviance when we assume H 0 : α 1 = = α r = 0 t be true. This is the extra sum f squares due t H 0. In rder t test H 0, nte that E{SSE} = E{ i (y i ȳ i ) 2 }

6 Chapter 8. Analysis f Designed Experiments 393 = i E{ (y i ȳ i ) 2 } = i (n i 1)σ 2 = (n r)σ 2 and hence s 2 = SSE n r is an unbiased estimate f σ 2. Nte that the divisr is ttal number f bservatins minus number f independent parameters, which is als the ttal number f degrees f freedm when the treatment means are uncnstrained. Under the null mdel y i = µ + ɛ i, all treatments have the same effect. If H 0 is true E{SST O} = (n 1)σ 2 and E{extra SS due t treatments} = (n 1)σ 2 (n r)σ 2 = (r 1)σ 2 Hence if H 0 is true SST R r 1 is an unbiased estimate f σ 2. Under the nrmality assumptin it can be shwn that SST R σ 2 χ 2 r 1 and SSE σ 2 χ 2 n r and that they are independent. Hence F = SST R/(r 1) SSE/(n r) = Treatment Mean Square s 2 F r 1,n r (8.4) under H 0. S we reect H 0 (and hence cnclude that there are significant differences amng the treatments) if F > F r 1,n r;1 α. The whle infrmatin may be summarized in the frm f an analysis f variance (ANOVA) table as in Table 8.1.

7 394 Chapter 8. Analysis f Designed Experiments Surce DF SS MS F i Treatments r 1 G2 n Residual n r by subtractin s 2 Ttal n 1 SST O = i T 2 i n i y2 i G2 n SS r 1 Table 8.1 ANOVA table fr ne-way analysis f variance. SS/(r 1) s Testing equality f variances The preceding analysis has assumed that the variances are the same in all samples. If the variances are allwed t be cmpletely arbitrary, then there is n exact prcedure t test equality f the means in the case r = 2, this is ust the Behrens-Fisher prblem discussed in Appendix A. Hwever, it is advisable t test fr the equality f variances. We describe here hw t cnstruct a likelihd rati test fr this. Define S i = (y i ȳ i. ) 2 fr i = 1,..., r. Assume that y i N(µ i, σi 2) independently fr all i and, and cnsider testing H 0 : σ1 2 =... = σr 2 against the alternative H 1 that impses n cnstraints n the {σi 2 }. The likelihd L, maximized with respect t {µ i }, is given by L { r ( 1 i=1 σ 2 i ) ni /2 ( exp S ) } i 2σi 2. This is maximized with respect t σi 2 by setting σ i 2 = S i /n i. Hence the maximized likelihd under H 1 is given, up t a cnstant f prprtinality, by { r ( ) } ni /2 1 L 1 = e n i/2. i=1 σ 2 i Under H 0, we replace σ 2 i by a cmmn σ2 in L, and maximize with respect t σ 2. This leads t σ 2 = i S i/n, and the resulting likelihd given by L 0 = ( ) N/2 1 σ 2 e N/2. Cnsequently the lg likelihd rati statistic is T = 2 lg ( L1 L 0 ) = r ( ) σ 2 n i lg. (8.5) i=1 Under H 0, the apprximate distributin f this statistic is T χ 2 r 1. σ 2 i

8 Chapter 8. Analysis f Designed Experiments Bartlett s mdificatin Bartlett (1937), in the earliest example f what is nw knwn as the Bartlett crrectin, prpsed a mdificatin f this test t imprve the χ 2 apprximatin. The steps in this mdificatin are: 1. Replace n i by n i 1 everywhere, s that σ 2 i = S i/(n i 1), σ 2 = i S i/(n r) and replace n i by n i 1 in (8.5), 2. With the definitin f T thus mdified, the distributinal apprximatin is { (r 1) r i=1 ( 1 n i 1 1 ) } 1 T χ 2 N r r 1. Hence reect H 0 at level α if T > χ 2 r 1;1 α, the upper-α quantile f the χ 2 r 1 distributin Tw examples. (i) The PEMA data. Our first example is based n Streete et al. (1986). Table 8.2 gives the serum levels f 2-ethyl-2-phenylmalnamide (PEMA) in patients receiving anticnvulsant medicatin. The aim is t measure PEMA levels in patients wh are prescribed primidne either singly r in cmbinatin with ther anticnvulsants. In this example we shall nly cnsider patients wh had similar dse levels f each f the fur drug cmbinatins listed belw. The first grup were given primidne alne, the secnd primidne and phenbarbitne, the third primidne and phenytin and the last a cmbinatin f all three, primidne, phenbarbitne and phenytin. We shall fit a ne-way mdel and lk fr evidence t indicate real differences amng the fur different drug cmbinatins. DRUG Table 8.2 PEMA data.

9 396 Chapter 8. Analysis f Designed Experiments Quick inspectin f the data suggests a highly skewed distributin in sme f the grups, s a lgarithmic transfrmatin was taken t imprve the fit t a nrmal distributin. The ANOVA table fr these transfrmed data is in Table 8.3. Surce DF SS MS F Treatments Residual Ttal Table 8.3 ANOVA table fr the lgged PEMA data. Frm tables we find F 24,3;0.99 = and therefre there is very strng evidence fr differences between the treatments which in this case are the drugs (the p-value is.007). Table 8.4 gives the fitted values and residuals. DRUGS Table 8.4 Fitted values and residuals fr the PEMA data. Fr example, fr drug 1 the average f lg PEMA values is and the residuals are lg(9.9) 1.659=0.634, lg(8.6) 1.659=0.493, etc. Nte that frm the ANOVA table s 2 = s that s = The largest crude residual in mdulus is 1.477, fr a t statistic f Based n the nminal t 24 distributin, the tw-sided p-value assciated with that (in ther wrds, 2 Pr{T > 2.623} when T t 24 ) is.0149, nt especially surprising cnsidering that this is the largest f 28 values. We culd plt the residuals (r the standardized residuals) against bth the fitted values and the treatment grup number t check fr hmgeneity f variance. Using Bartlett s mdified test prduces the values in Table 8.5. Here n = 28 and t = 4. We find that T = With a null distributin f χ 2 3, this has a p-value f.084, s there is sme slight evidence against H 0.

10 Chapter 8. Analysis f Designed Experiments 397 ȳ i s 2 i n i Table 8.5 Bartlett s test fr the PEMA data (ii) Rund-rbin data. Rund-rbin tests are tests perfrmed under suppsedly the same cnditins in a series f labratries. They play an imprtant part in the standardizatin f measurement prcedures in physics and engineering. Hwever, the tests are ften expensive t perfrm, s the results f such studies are ften small unbalanced sets f data. Table 8.6 is based n an actual example f such a study int the measurement f the rate f creep rupture. Samples f a material were sent t 11 labratries which were asked t perfrm repeat measurements. Tabulated are n i, the number f tests perfrmed at labratry i, and the mean and standard deviatin (S.D.) f the measurements made at labratry i. Als cmputed in clumn 5 is S i = (n i 1)(S.D.) 2. The last tw clumns will be explained a little further n. Labratry i n i Mean S.D. S i α i S.E Table 8.6 Rund-rbin test data It can be seen immediately that the S.D. fr labratry 1 is much larger than the thers, and an applicatin f (8.5) immediately cnfirms that this is significant: the likelihd rati statistic is with a nminal χ 2 10 distributin. The % pint f χ 2 10 is In fact, the raw data fr labratry 1 shwed ne extreme utlier which was nt deleted frm the

11 398 Chapter 8. Analysis f Designed Experiments reprted data, almst certainly due t a recrding errr. If we ignre the apparent discrepancy in variances and cmpute the F statistics (8.4) fr equality f means, the F value is 3.86, which is highly significant (with a nminal distributin f F 10,48, the p-value is.0007). Hwever, in this case the discrepancy in variances is t sme extent masking differences in the means, and t avid such a bias in ther cmparisns, we delete labratry 1 fr the rest f the discussin. T prceed with the analysis, deleting labratry 1 and repeating the tests fr the ther 10 labratries, (8.5) yields the test statistic 22.6, nminally χ 2 9, which is still high (significant at 1% actual p-value 0.007) but nt nearly as bad as befre. In this case, the F statistic in (8.4) is 20.2, very highly significant against the nminal F 9,44 distributin (theretical p-value abut ). Further inspectin f the data indicates that labratry 8 has a much lwer mean than the thers, and als the lwest standard deviatin. Omissin f labratry 8 (as well as labratry 1) leads t the test statistics 13.2 in (8.5), 3.67 in (8.4). The value 13.2 is nt significant against χ 2 8 at the 10% level, s seems acceptable fr the equality f variances. If the likelihd rati statistics (8.5) are mdified by the Bartlett crrectin, they becme 77.9 fr the full mdel with all 11 labratries, 16.9 fr the mdel with labratry 1 mitted, and 11.4 fr the mdel with labratries 1 and 8 bth mitted. The mdificatin in the first case is substantial but irrelevant. The value 16.9 is almst exactly at the 5% pint fr χ 2 9, s the effect f the mdificatin here is t cnclude that the assumptin f equality f variances might ust be acceptable in this case (with labratry 1 mitted), thugh since it is clear that labratry 8 is discrepant in its mean value, this des nt affect ur verall cnclusins. The crrectin frm 13.2 t 11.4 in the third mdel strengthens the cnclusin that the variances can be assumed equal in this case. Thus the Bartlett crrectin des nt change any f ur cnclusins, but it des turn ut t be numerically nt negligible. In general it can be recmmended that, in experiments f this nature when the individual n i are small, it is wrthwhile t apply the Bartlett crrectin. Fr the final mdel, with labratries 1 and 8 bth mitted, an ANOVA table is shwn in Table 8.7. The final F statistic fr the hypthesis f n treatment effect is 3.66, fr a p-value f.003. Thus, we cnclude that even when labratries 1 and 8 are mitted, there is still a significant difference amng the remaining 9 labratries. The last tw clumns f Table 8.6 shw the estimates α i and their standard errrs fr all labratries except 1 and 8. T cmpute the standard errrs, nte that

12 Chapter 8. Analysis f Designed Experiments 399 Surce DF SS MS F Treatments Residual Ttal Table 8.7 One way ANOVA table fr the rund-rbin test data with labratries 1 and 8 mitted. and cnsequently α i = ȳ i. ȳ.. = Var( α i ) = ( 1 1 ) y i 1 n i N N i i y i ( 1 1 ) 2 ( ) 2 1 σ 2 n i + σ 2 (N n i ) n i N N = σ 2 N n i Nn i s the standard errr f α i may be cmputed as s (N n i )/(Nn i ) where s 2 is the usual unbiased estimate f σ 2. Frm Table 8.7 this is calculated as s 2 = and hence s = 6.84, and this is used t calculate the standard errrs in Table 8.6. In the light f these calculatins we can nw see that the treatment effects are significantly negative in labratries 4 and 7, and significantly psitive in labratry 3, while the thers are nt significantly different frm 0. The verall cnclusin therefre is that labratries 1 and 8 are grss utliers, while labratries 3, 4 and 7 are als prducing results significantly different frm the remainder. Fllw-up studies wuld be expected t identify the reasns fr this fr example, it is pssible that sme f the labratries were using a different measurement technique frm the thers, and the cnclusin in that case wuld be that the different techniques culd nt be regarded as interchangeable Multiple Cmparisns If we have nt pre-planned any specific cmparisns between treatments and there is evidence fr differences we may still wish t investigate where these differences lie. We shall describe three methds (i) Least significant differences (LSD) Suppse we have a balanced experiment s that n = rt. A general prcedure fr paired cmparisns is t cmpute a least significant difference

13 400 Chapter 8. Analysis f Designed Experiments 2 LSD = t ν;0.025 s t, 2 where ν = n r and s t is the estimated standard errr f the difference between tw treatment means. This gives a lwer bund fr a significant difference between pairs, s that if a preplanned pair f means differs by mre than LSD then there is evidence t cntradict the hypthesis that the tw treatment effects are the same. Nte that there are r(r 1)/2 pairs f treatments, s even if there are n differences in treatment effects, apprximately 5% f differences between pairs will exceed LSD. We shuld use LSD sparingly! (ii) Scheffé s methd Suppse we have an arbitrary set f treatments fr which the treatment sum f squares is significant (i.e. we reect a null hypthesis that the treatment means are all equal), but there are n specific cmparisns (r cntrasts) fr which we planned in advance t test. In this cntext, it is natural t try t find simultaneus 100(1 α)% cnfidence intervals fr all pssible cntrasts. This raises issues similar t the discussin f simultaneus cnfidence intervals in Sectins 2.8 and 3.6. A cntrast r i=1 a iα i = a T α is estimated by a T ˆα r r i=1 a iȳ i with variance r i=1 a2 i σ2 n i. Hence in the case f a preplanned cmparisn a 100(1 α)% cnfidence interval fr this cntrast wuld be r a i ȳ i ± t n r; 1 2 α s r a 2 i n i i=1 and if this cnfidence interval includes the value zer, then this cntrast is nt significant, i.e. there is n evidence against the hypthesis i=1 H 0 : r a i α i = 0. i=1 An adaptatin f Scheffé s methd (Sectin 3.8) t this scenari shws that the set f all cnfidence intervals f the frm r a i ȳ i ± s r a 2 i (r 1)F r 1,n r;α n i i=1 i=1 cntain the true values ( r i=1 a iα i ) with int prbability 1 α (s are simultaneus cnfidence intervals. A useful prperty f Scheffé s methd is that at least ne cntrast will be significant (in ther wrds, the cnfidence interval will exclude zer) if

14 Chapter 8. Analysis f Designed Experiments 401 the treatment sum fr squares f the standard F test is significant at the same level f significance α. (iii) Tukey s methd This methd gives simultaneus 100(1 α)% cnfidence intervals fr all cntrasts r r a i a i ȳ i ± s 2 q(α, r, ν) n i i=1 i=1 where q(α, r, ν) represents the percentage pints f the distributin f the studentized range statistic 1 { } s 2 max(ȳ i ) min(ȳ i ) i i fr which several published tables are available (e.g. Harter 1960, Rhlf and Skal 1995). Here ν = n r the degrees f freedm in this case. Fr paired cmparisns with balanced design the three methds give us the fllwing intervals: Scheffé : Tukey : LSD : 2 ȳ i ȳ ± s t (r 1)F r 1,n r;α, ȳ i ȳ ± s q(α, r, n r), t 2 ȳ i ȳ ± s t t n r,α/2. A methd f illustrating the cmparisns is t write the means in rder n a scaled line and underline pairs nt significantly different frm each ther. Fr the PEMA data, the Scheffé interval width is ±0.904, Tukey is ±0.970 and LSD is ± The fur drug means are 1.659, 2.760, and fr drugs 1 t 4 respectively. The nly tw drugs which have a difference larger than these values is btained when cmparing drugs 1 and 2 where the difference is Hence using the methds f multiple cmparisns suggests that nly treatments 1 and 2 are significantly different. 8.3 The Tw-Way Layut In the ne-way layut, there is nly ne factr that varies between the grups. A mre cmplicated and typical situatin, hwever, is that there is mre than ne factr. Fr example, in the rund-rbin example discussed in sectin 8.2.4, it is pssible that the different labratries, instead f simply repeating the same experiment a number f times, were in fact asked t perfrm a fixed sequence f experiments, the bective being t determine whether there was any verall difference amng the labratries. Anther

15 402 Chapter 8. Analysis f Designed Experiments example is in clinical trials. Suppse a trial is set up t cmpare tw drugs. If the patients are simply assigned at randm t the tw drugs, it is likely that the final distributin f ages, sexes and ther prgnstic variables such as disease cnditin, will be quite different between the tw grups. This culd seriusly bias the results. A better way is first t stratify, i.e. divide the patients int subgrups f rughly cmparable individuals, and then assign the tw drugs randmly within each subgrup (r stratum). In general, there culd be many different factrs that have t be taken int accunt, and a cmplex arrangement culd be necessary t d that. The tw-way layut refers t a set-up in which there are ust tw kinds f effect, a treatment effect which is the main variable f interest, and a blck effect which is nt f direct interest but which wuld bias the results if it were nt taken int accunt. In the rund-rbin example, assuming that it is still the differences amng labratries that are f interest, the treatment effect is the labratry and the blck effect is an individual experiment within a labratry. Fr a clinical trial, the treatment effect is usually the treatment itself, e.g. the type f drug, while the blck effect refers t the factrs such as the age, sex and clinical cnditin f the patients, which determine the subdivisin int strata. We cnsider nly a balanced experiment, in which the same number t f bservatins are cllected fr each treatment-blck cmbinatin, knwn as a randmized blck design. Initially, we nly cnsider the case t = 1. Assume there are r treatments and c blcks, and let y i dente the bservatin fr treatment i and blck. A plausible mdel fr this situatin is y i = µ + α i + β + ɛ i, 1 i r, 1 c, (8.6) in which µ is an verall mean and α i and β are respectively a treatment effect and a blck effect. The cnstraints n these are α i = 0, β = 0, (8.7) i and we make the usual assumptin that the {ɛ i } are independent N(0, σ 2 ). Typically the {β } are thught f as nuisance parameters and the real interest is still in the {α i }. The hypthesis f n treatment effect is still frmulated as H 0 : α 1 = α 2 =... = α r = 0 (8.8) but nw we want t test this withut making any implicit r explicit assumptin that the {β } are als 0. We can als test the hypthesis f n blck effect which is frmulated as H 0 : β 1 = β 2 =... = β c = 0. (8.9)

16 Chapter 8. Analysis f Designed Experiments Estimates f the treatment and blck effects If we define ȳ i. = y i/c t be the i th treatment mean, ȳ. = i y i/r t be the th blck mean, and ȳ.. = i y i/(rc) the verall mean, then the apprpriate estimates are btained by minimizing S = r i=1 =1 c (y i µ α i β ) 2, and this is achieved by differentiating with respect t α i, i = 1,..., r and β, = 1,..., c. There are r+c+1 equatins, but nly r+c 1 independent equatins. S there are an infinite number f slutins. One pssible slutin is t assume that r i=1 ˆα i = 0 and c ˆβ =1 = 0 and hence µ = ȳ.., α i = ȳ i. ȳ.., β = ȳ. ȳ... The fitted values (estimated means) are ˆµ + ˆα i + ˆβ = ȳ i. + ȳ. y.. Fr all slutins the residual sum f squares S min = = r i=1 =1 r i=1 =1 c (y i µ α i β ) 2 c (y i ȳ i ȳ + ȳ ) 2 As in the ne-way layut (with N = rc, n i = c), we may calculate the variance f α i t be (r 1)σ 2 /(rc), and similarly variance f β is (c 1)σ 2 /(rc). Fr a cntrast f the frm i c iα i ( i c i = 0) we have the estimatr i c i α i = i c iȳ i., which has variance σ 2 i c2 i /c. Orthgnal cntrasts can be used as befre fr preplanned cmparisns f bth treatments and blcks when either f these tw tests shw evidence fr differences. Otherwise the methds f multiple cmparisns (e.g. Scheffé) can be used The ANOVA table Cnsider the decmpsitin (y i ȳ.. ) 2 i

17 404 Chapter 8. Analysis f Designed Experiments = i = i = i {(y i ȳ i. ȳ. + ȳ.. ) + (ȳ i. ȳ.. ) + (ȳ. ȳ.. )} 2 (y i ȳ i. ȳ. + ȳ.. ) 2 + c (ȳ i. ȳ.. ) 2 + r (ȳ. ȳ.. ) 2 i (y i µ α i β ) 2 + c α i 2 + r β 2 i where as in earlier ANOVA calculatins, the crss-prduct terms all turn ut t be 0. The final result may als be written in the frm SST O = SSE + SST R + SSB as a decmpsitin f the ttal sum f squares SST O int a sum f squares due t errr (r residual) SSE, a sum f squares due t treatment SST R and a sum f squares due t blcks SSB. The crrespnding partitin f the degrees f freedm is rc 1 = (r 1)(c 1) + (r 1) + (c 1). Hence we derive the ANOVA table in Table 8.8. SOURCE SUM OF SQUARES D.F. MEAN SQUARE Treatments SST R r 1 SST R/(r 1) Blcks SSB c 1 SSB/(c 1) Residual SSE (r 1)(c 1) SSE/{(r 1)(c 1)} Ttal SST O rc 1 Table 8.8 Tw-way ANOVA table. T test the hypthesis (8.8) we use the F statistic SST R/(r 1) (c 1)SST R = F r 1,(r 1)(c 1) under H 0. SSE/{(r 1)(c 1)} SSE Similarly t test the hypthesis (8.9) we use the F statistic SSB/(c 1) (r 1)SSB = F c 1,(r 1)(c 1) under H SSE/{(r 1)(c 1)} SSE 0. Example - Tensile prperties f an ally. A quantity f an ally was prepared under careful prcessing cnditins t attain precisin and hmgeneity f its prperties. The ally was rlled

18 Chapter 8. Analysis f Designed Experiments 405 Bar Labratries Mean Mean Table 8.9 Tensile prperties f fur ally bars measured at six labratries int bars and lengths cut frm fur f these bars were sent t six labratries. The analysis cncerned a particular tensile prperty. The data given in Crwder (1992), frm a balanced layut in which each labratry cnducts a test n nly ne length frm each f the fur bars, and are given in Table T calculate the ANOVA table by hand, we need SST O = i = i (y i ȳ.. ) 2 yi 2 crȳ.. 2 = = 3.819, SST = c i (ȳ i. ȳ.. ) 2 = c i ȳ 2 i. crȳ 2.. = = 1.605, SSB = r (ȳ. ȳ.. ) 2 = r ȳ 2. crȳ 2.. = = 0.981, SSE = SST O SST SSB = The cmplete ANOVA table is therefre given in Table The respective F -tests that bth the labratries and bars are significant with resultant p-values f and respectively. Residual plts can be dne t check the mdel assumptins.

19 406 Chapter 8. Analysis f Designed Experiments SOURCE SUM OF SQUARES D.F. MEAN SQUARE F Labratries Bars Residual Ttal Table 8.10 ANOVA table fr tensile prperty data. 8.4 The tw-way layut with interactin The analysis in sectin 8.3 relies n the assumptin that (8.6) is an apprpriate mdel. This is knwn as an additivity assumptin, since it implies that ne can deduce the cmbined effect f treatment and blck simply by adding up the separate effects fr the tw. One cnsequence f this assumptin, fr example, is that it implies that differences between treatments are the same fr all blcks: if drug 1 is better than drug 2 n ne grup f patients, then it is als better (by the same amunt) n any ther grup f patients. Such an assumptin is ften gd enugh as a first apprximatin but may nt be valid in general. Indeed ne may even argue that detecting interactins is smetimes the main purpse f the analysis. An example f an interactin wuld be a statement that a certain drug is particularly effective in cmparisn with anther drug with ne grup f patients, but nt necessarily with ther grups. This implies that we might want t cnsider a mre general mdel y ik = µ + α i + β + γ i + ɛ ik, 1 i r, 1 c, 1 k t, (8.10) subect t the cnstraints (8.7) plus γ i = 0 fr each i, γ i = 0 fr each. i In this mdel we are assuming that there are t > 1 bservatins fr each treatment-blck cmbinatin. The ttal number f parameters fr the treatment-blck means is 1 fr µ, r 1 fr the treatment effects (i.e. r treatment effects subect t 1 cnstraint), c 1 fr the blck effects and (r 1)(c 1) fr the interactins, a ttal f rc. This means that any cmbinatin f the rc treatment-blck means may be handled within the framewrk f (8.10). Hwever, if t = 1 then there are n degrees f freedm left ver t estimate σ 2, hence ur need t assume t > 1. As in previus ntatin we use a dt t indicate averaging ver the dtted parameter - thus ȳ i.. is the mean f all bservatins with treatment

20 Chapter 8. Analysis f Designed Experiments 407 i, ȳ i. is the mean f all bservatins with the i th treatment and the th blck, and s n. In this ntatin we have µ = ȳ..., α i = ȳ i.. ȳ..., β = ȳ.. ȳ..., γ i = ȳ i. ȳ.. ȳ i.. + ȳ.... The ANOVA decmpsitin becmes (y ik ȳ... ) 2 i = i k (y ik ȳ i. ) 2 + t i which may als be written as k γ i 2 + ct i α 2 i + rt β 2 with degrees f freedm SST O = SSE + SSI + SST R + SSB rct 1 = rc(t 1) + (r 1)(c 1) + (r 1) + (c 1). Here SSI stands fr sum f squares due t interactin. The ANOVA table is Table SOURCE SUM OF SQUARES D.F. MEAN SQUARE Treatments SST R r 1 SST R/(r 1) Blcks SSB c 1 SSB/(c 1) Interactin SSI (r 1)(c 1) SSI/{(r 1)(c 1)} Residual SSE rc(t 1) SSE/{rc(t 1)} Ttal SST O rct 1 Table 8.11 Tw-way ANOVA table with interactins. Fr example, ne culd test the hypthesis f n treatment effect with SST R/(r 1) SSE/{rc(t 1)} F r 1,rc(t 1) under H 0 : α i = 0 fr all i. Hwever, it might als be f interest t test whether there is any interactin, in which case the apprpriate F -test is based n

21 408 Chapter 8. Analysis f Designed Experiments SSI/{(r 1)(c 1)} SSE/{rc(t 1)} Tukey s 1-DF test fr additivity F (r 1)(c 1),rc(t 1) under H 0 : γ i = 0 fr all i,. Suppse we nly have ne bservatin fr each treatment-blck cmbinatin, but we still want t test fr interactins. As previusly explained, we cannt fit the full mdel in which all the interactins are uncnstrained. Tukey (1949) prpsed a slutin t this prblem. The idea is t assume the mdel y i = µ + α i + β + θα i β + ɛ i, 1 i r, 1 c, (8.11) in which the interactin γ i takes the specific frm θα i β. This is unlikely t be the crrect mdel but a test f θ = 0 shuld be a useful indicatin f whether the mdel is additive r nt. T define a test fr this, let us first define and cnsider the mdel z i = y i ȳ i. ȳ. + ȳ.., z i = θa i b + e i (8.12) in which the cnstants a i and b are knwn and subect t i a i = b = 0, and the e i are errrs. In (8.12), the usual least squares estimatin f θ (treating the errrs as independent) yields i θ = z ia i b i = y ia i b, (8.13) i a2 i b2 i a2 i b2 the tw expressins being identical because i a i = b = 0. Frm the secnd expressin, it can be seen that the variance f θ is i a2 i Hence under H 0 : θ = 0 we have b2 σ 2 b2 θ 2 i a2 i σ 2 = ( σ 2 i a2 i. i y ia i b ) 2 b2 χ 2 1. Cnsider the ANOVA decmpsitin zi 2 = (z i θa i b ) 2 + θ 2 i i i a 2 i b 2

22 Chapter 8. Analysis f Designed Experiments 409 which we als write as SSI = SSIE + SSG with degrees f freedm (r 1)(c 1) = (rc r c) + 1. Let us calculate Cv(z i, θ) fr given i,. Frm (8.13) we have that Hwever Cv(z i, θ) 1 = a 2 i b 2 a i b Cv(z i, y i ). i Cv(z i, y i ) = Cv(y ( i ȳ i. ȳ. + ȳ.., y i ) = σ 2 δ ii δ 1 c δ ii 1 r δ + 1 ) rc where δ is the Krnecker delta (δ i = 1 if i =, 0 therwise). Hence Cv(z i, θ) σ 2 = a 2 i b 2 a i b i = a ib σ 2 a 2 i b 2 = a i b Var( θ). ( δ ii δ 1 c δ ii 1 r δ + 1 ) rc Thus z i a i b θ is uncrrelated with θ fr each i,. Since everything is intly nrmal, uncrrelated implies independent and hence the tw sums f squares, SSIE and SSG, are themselves independent. Frm this we deduce the F test, that under H 0 : θ = 0 we have SSG SSIE/(rc r c) F 1,rc r c. (8.14) Nw cmes the key step: we can d all this with the cnstants a i, b replaced by α i, β respectively, and the final result (8.14) is still valid. The reasn is simply that all the {z i } are independent f the { α i } and { β } s that nne f the distributinal arguments leading t (8.14) are affected by the fact that the {α i } and {β } are estimated rather than knwn. T summarize, the prcedure is as fllws. First calculate the { α i }, { β }

23 410 Chapter 8. Analysis f Designed Experiments and {z i }, then cmpute SSI = i z 2 i, SSG = ( i z i α i β ) 2 i α2 β i 2 = ( i y i α i β ) 2 i α2 β, i 2 and let SSIE = SSI SSG. Then 8.14 gives an exact F test f the additivity f the mdel Example - Fisher data. The data in Table 8.12 are taken frm Fisher (1971), page 68, and give the yields f five varieties f barley in six lcatins in each f tw years, 1931 and The interest here is presumably in the cmparisn amng different varieties f barley (s treatment is variety), and ne pssible strategy is t treat each place-year cmbinatin as a separate blck. This suffers frm the disadvantage that it may nt be pssible t detect interactins. A secnd strategy is t treat the places as blcks and the tw years f data as separate replicatins f the whle experiment, prducing a 5 6 table with tw bservatins per cell. The disadvantage f this is that the residuals may be crrelated with year. We shall try bth strategies, starting with the secnd ne. (A third strategy might be t cnsider variety, place and year all as separate factrs and t d a three-way analysis, but we shall nt cnsider that.) Place year Manchuria Svansta Velvet Trebi Peatland Mean Clumn Mean Table 8.12 Fisher s data n barley varieties Table 8.13 shws the analysis f variance table fr this mdel. The F statistics are /458.9=2.89 fr varieties, /458.9=9.25 fr places,

24 Chapter 8. Analysis f Designed Experiments 411 SOURCE SUM OF SQUARES D.F. MEAN SQUARE SST SSB SSI SSE Ttal Table 8.13 ANOVA Table fr 2-way mdel with interactins Table 8.14 Residuals frm 2-way mdel 221.7/458.9=0.48 fr interactins. These are barely significant fr a variety effect (the 5% pint f F 4,30 is 2.69), highly significant fr a place effect, and nt at all significant fr interactins. Hwever, Table 8.14, which gives the residuals frm this mdel, shws that there is indeed a crrelatin with year. In nearly every case the yield in 1931 was higher than 1932, except fr place 3 where the pattern is curiusly reversed. The (linear) crrelatin cefficient between residual and year is 0.525, bviusly a significant effect. In view f this we try a secnd analysis, in which each place-year cmbinatin is treated as a separate blck (12 blcks in all), and Tukey s test is used t test fr interactins. The analysis f variance table is given in Table The F ratis are 8.18 fr variety, 17.9 fr the blck effects, and 3.27 fr the interactin effect. Hwever the latter is nt significant, the 5% pint fr the F 1,43 distributin being abut 4.1.

25 412 Chapter 8. Analysis f Designed Experiments SOURCE SUM OF SQUARES D.F. MEAN SQUARE SST SSB SSG SSIE Ttal Table 8.15 ANOVA Table fr Tukey s 1-DF test Residual Predicted Value Figure 8.1 Plt f residuals vs. fitted values fr barley data, 2-way mdel withut interactins Finally, the residuals frm this mdel have been examined with nthing suspicius being bserved. As an example, Figure 8.1 shws a plt f the residuals against fitted values. Frm this it appears that the secnd analysis is satisfactry. The prblem with the first analysis was that, by failing t treat year as a separate blck effect, the estimated variance was inflated because f what was evidently a significant difference between the tw years. 8.5 Implementatin in SAS and S-PLUS ANOVA in SAS The SAS prcedure ANOVA may be used t fit any versin f ne-way r tw-day analysis f variance, and many mre different types f mdels.

26 Chapter 8. Analysis f Designed Experiments 413 Here is a simple example, applied t the PEMA data frm Sectin First, we create a data file pema.dat, as fllws: Here, the data are arranged in tw clumns with the drug in the first clumn and the serum level in the secnd. A SAS prgram t analyze these data is as fllws: ptins ls=64 ps=58; data pema; infile pema.dat input drug serum; lserum=lg(serum); run; ; prc anva; class drug; mdel lserum=drug; means drug /lsd sheffe tukey; run; ; In the call t prc anva, the statement class drug means that drug is being treated as a classified variable (r factr), rather than as a numerical variable. The mdel statement then fits the ne-way ANOVA mdel, with lg serum as the respnse variable. The means statement asks fr cmparisns amng the mean respnses fr each drug, with differences evaluated accrding t each f the three criteria discussed in Sectin The utput frm this prcedure is as fllws: The ANOVA Prcedure Class Level Infrmatin

27 414 Chapter 8. Analysis f Designed Experiments Dependent Variable: lserum Class Levels Values drug Number f bservatins 28 Sum f Surce DF Squares Mean Square Mdel Errr Crrected Ttal Surce F Value Pr > F Mdel Errr Crrected Ttal R-Square Ceff Var Rt MSE lserum Mean Surce DF Anva SS Mean Square drug Surce F Value Pr > F drug t Tests (LSD) fr lserum NOTE: This test cntrls the Type I cmparisnwise errr rate, nt the experimentwise errr rate. Alpha 0.05 Errr Degrees f Freedm 24

28 Chapter 8. Analysis f Designed Experiments 415 Errr Mean Square Critical Value f t Least Significant Difference Means with the same letter are nt significantly different. t Gruping Mean N drug A A B A B B B B Tukey s Studentized Range (HSD) Test fr lserum NOTE: This test cntrls the Type I experimentwise errr rate, but it generally has a higher Type II errr rate than REGWQ. Alpha 0.05 Errr Degrees f Freedm 24 Errr Mean Square Critical Value f Studentized Range Minimum Significant Difference Means with the same letter are nt significantly different. Tukey Gruping Mean N drug A A B A B B B B Scheffe s Test fr lserum

29 416 Chapter 8. Analysis f Designed Experiments NOTE: This test cntrls the Type I experimentwise errr rate. Alpha 0.05 Errr Degrees f Freedm 24 Errr Mean Square Critical Value f F Minimum Significant Difference Means with the same letter are nt significantly different. Scheffe Gruping Mean N drug A A B A B B B B The first part f the utput reprduces the analysis f variance table given earlier. The three sets f results in respnse t the means statement all cme t the same cnclusin: the rder f the drugs in terms f decreasing serum levels is 2, 4, 3, 1; drugs 2 and 4 frm a cmmn grup in the sense that they are n significantly different when udged by any f the three tests; similarly, drugs 4, 3 and 1 frm a cmmn grup. The actual critical values f t r F which determine these grupings are, hwever, different. Nw let us cnsider a tw-way ANOVA withut interactin, using the allys data set f Sectin The data are again arranged in clumn frmat, in a file ally.dat : A sample SAS cde is nw as fllws: ptins ls=64 ps=58; data ally;

30 Chapter 8. Analysis f Designed Experiments 417 infile ally.dat ; input bar lab strength; run; ; prc anva; class bar lab; mdel strength=bar lab; means bar /sheffe; means lab / sheffe; run; ; The nly difference frm the previus example is that the are nw tw class variables, bar and lab, and the mdel includes bth f them. The means statement asks fr cmparisns f bth, using the Scheffé test criterin. Here is part f the utput (edited): Dependent Variable: strength Sum f Surce DF Squares Mean Square Mdel Errr Crrected Ttal Surce DF Anva SS Mean Square bar lab Surce F Value Pr > F bar lab Scheffe s Test fr strength

31 418 Chapter 8. Analysis f Designed Experiments... Scheffe Gruping Mean N bar A A B A B A B A B B Scheffe Gruping Mean N lab A A B A B A B A B A B A B A B A B B The results f the ANOVA table are the same as given earlier, resulting in the cnclusin that bth bar and lab are significant factrs. The means cmparisn shws that bars 3, 1 and 2 frm a cmmn grup (nt separated by the Scheffé test); likewise, bars 1, 2 and 4 frm a cmmn grup, but there are significant differences amng all fur grups. The crrespnding cmparisns fr labratries shw that labratries 4, 1, 3, 2, 5 frm a cmmn grup, as d 1, 3, 2, 5, 6, but again, we reect the hypthesis that all six labratries are the same. Nw let us cnsider the Fisher barley yields example f Sectin 8.4. The data set is again arranged in clumns, with year in clumn 1, place in clumn 2, variety (cded as 1 5) in clumns 3, and yield in clumn 4:

32 Chapter 8. Analysis f Designed Experiments A pssible SAS prgram is as fllws: ptins ls=64 ps=58; data fisher; infile fisher.dat ; input year place variety yield; run; ; prc anva; class year place variety; mdel yield=place variety place*variety; run; ; prc anva; class year place variety; mdel yield=year place variety; run; ; The first analysis ignres year, but treats the experiment as a twway ANOVA with interactin the variable place*variety creates this interactin. The secnd analysis is slightly different frm the ne given earlier in effect a three-day analysis f variance withut interactins, treating each f year, place and variety as a categrical variable. The ANOVA table frm the first analysis gives: Sum f Surce DF Squares Mean Square Mdel Errr Crrected Ttal Surce DF Anva SS Mean Square place variety place*variety Surce F Value Pr > F place 9.25 <.0001

33 420 Chapter 8. Analysis f Designed Experiments variety place*variety This analysis cnfirms that, analyzed as a tw-way ANOVA, the interactin term is nwhere near significant. Hwever, the three-way analysis including year yields the fllwing: Sum f Surce DF Squares Mean Square Mdel Errr Crrected Ttal Surce DF Anva SS Mean Square year place variety Surce F Value Pr > F year place <.0001 variety This cnfirms that the year variable is highly significant, with a p-value f.0008, and therefre speaks strngly twards including it in the mdel ANOVA in R In R, it is pssible t d an analysis f variance using the lm cmmand that we usually use fr linear regressin, after first using factr t redefine the treatment r blck variable as a factr variable. Fr example, t fit the PEMA data using lm, we culd try dat1<-matrix(scan( D:/r/b/s105/dat1/pema.dat ),ncl=2,byrw=t) drug<-factr(dat1[,1]) lpema<-lg(dat1[,2]) lm1<-lm(lpema~drug) The cmmand summary(lm1) then prduces the utput Call: lm(frmula = lpema ~ drug) Residuals:

34 Chapter 8. Analysis f Designed Experiments 421 Min 1Q Median 3Q Max Cefficients: Estimate Std. Errr t value Pr(> t ) (Intercept) e-08 *** drug ** drug drug Signif. cdes: 0 *** ** 0.01 * Residual standard errr: n 24 degrees f freedm Multiple R-Squared: , Adusted R-squared: F-statistic: n 3 and 24 DF, p-value: Frm this we see, in particular, that the F statistic fr n treatment effect has a p-value f abut.007, implying we shuld reect that null hypthesis. In this analysis, the treatment effect fr drug 1 is by default assumed t be 0, and the analysis shws that the estimated treatment effects fr drugs 2, 3, 4 are respectively 1.10, 0.19 and 0.51 abut that fr treatment 1. Alternatively may replace the lm cmmand with av1<-av(lpema~drug) The cmmand summary(av1) then yields > summary(av1) Df Sum Sq Mean Sq F value Pr(>F) drug ** Residuals Signif. cdes: 0 *** ** 0.01 * At first sight, this is nt mre infrmative. Hwever, the av bect ffers mre. Fr example, > TukeyHSD(av1) Tukey multiple cmparisns f means 95% family-wise cnfidence level Fit: av(frmula = lpema ~ drug) $drug diff lwr upr p ad

35 422 Chapter 8. Analysis f Designed Experiments This cmputes the mean and a 95% cnfidence limit fr all the pairwise treatment differences, using the Tukey pairwise cmparisns prcedure t crrect fr multiple cmparisns. The result shws that drug 2 yields a higher mean respnse than either drug 1 r drug 3; hwever, nne f the ther pairwise differences is statistically significant. Anther useful cmmand is plt(av1). This prduces a number f diagnstic plts, illustrated in Figure 8.2. The next analysis perfrms a similar b fr the allys data set. x<-matrix(scan(file= ally.dat ),ncl=3,byrw=t) strength<-x[,3] bar<-factr(x[,1]) lab<-factr(x[,2]) av2<-av(strength~bar+lab) This perfrms a tw-way analysis withut interactins, treating bth bar and lab as factr variables. The cmmand summary(av2) prduces Df Sum Sq Mean Sq F value Pr(>F) bar * lab * Residuals Signif. cdes: 0 *** ** 0.01 * cnfirming that bth the lab effect and the bar effect are statistically significant. If we which t knw which pairwise differences are significant, we can use TukeyHSD(av2) yielding Tukey multiple cmparisns f means 95% family-wise cnfidence level Fit: av(frmula = strength ~ bar + lab) $bar diff lwr upr p ad