MULTIPLE COMPARISON PROCEDURES

Transcription

1 MULTIPLE COMPARISON PROCEDURES RAJENDER PARSAD Indan Agrcultural Statstcs Research Insttute Lbrary Avenue, New Delh Introducton Analyss of varance s used to test for the real treatment dfferences. When the null hypothess that all the treatment means are equal s not reected, t may seem that no further questons need to be asked. However, n some expermental stuatons, t may be an oversmplfcaton of the problem. For example, consder an experment n rce weed control wth 15 treatments vz. 4 wth hand weedng, 10 wth herbcdes and 1 wth no weedng (control). The probable questons that may be rased and the specfc mean comparsons that can provde ther answers may be: ) Is any treatment effectve n controllng the weeds? Ths queston may be answered smply by comparng the mean of the control treatment wth the mean of each of the 14 weed-control treatments. ) Is there any dfference between the group of hand-weedng treatments and the group of herbcde treatments? The comparson of the combned mean of the four hand weedng treatment effects wth the combned mean of the 10-herbcde treatment effects may be able to answer the above queston. ) Are there dfferences between the 4 hand weedng treatments? To answer ths queston one should test the sgnfcant dfferences among the 4 hand weedng treatments. Smlar queston can be rased about the 10-herbcde treatments and can be answered n the above fashon. Ths llustrates the dversty n the types of treatment effects comparsons. Broadly speakng, these comparsons can be classfed ether as Par Comparson or Group Comparson. In par comparsons, we compare the treatment effects parwse whereas ngroup comparsons, the comparsons could be between group comparsons, wthn group comparsons, trend comparsons, and factoral comparsons. In the above example, queston () s the example of the par comparsons and queston () llustrates the between group comparson and queston () s wthn group comparson. Through trend comparsons, we can test the functonal relatonshp (lnear, quadratc, cubc, etc.) between treatment means and treatment levels usng orthogonal polynomals. Factoral comparsons are related to the testng of means of levels of a factor averaged over levels of all other factors or average of treatment combnatons of some factors averaged over all levels of other factors. For parwse treatment comparsons there are many test procedures, however, for the group comparsons, the most commonly used test procedure s to partton the treatment sum of squares nto meanngful comparsons. Ths can be done through contrast analyss ether usng sngle degrees of freedom contrasts or contrasts wth multple degrees of freedom. Further, the comparsons can be dvded nto two categores vz. planned comparsons and unplanned comparsons or data snoopng. These have the followng meanngs. Before the experment commences, the expermenter wll have wrtten out a checklst, hghlghtng the comparsons or contrasts that are of specal nterest, and desgned the experment n

2 Multple Comparson Procedures such a way as to ensure that these are estmable wth as small varances as possble. These are the planned comparsons. After the data have been collected, the expermenter usually looks carefully at the data to see whether anythng unexpected has occurred. One or more unplanned contrasts may turn out to be the most nterestng, and the conclusons of the experment may not be antcpated. Allowng the data to suggest addtonal nterestng contrasts s called data snoopng. The most useful analyss of expermental data nvolves the calculaton of a number of dfferent confdence ntervals, one for each of several contrasts or treatment means. The confdence level for a sngle confdence nterval s based on the probablty, that the random nterval wll be correct (meanng that the random nterval wll contan the true value of the contrast or functon). It s shown below that when several confdence ntervals are calculated, the probablty that they are all smultaneously correct can be alarmngly small. Smlarly, when several hypotheses are tested, the probablty that at least one hypothess s ncorrectly reected can be uncomfortably hgh. Much research has been done over the years to fnd ways around these problems. The resultng technques are known as methods of multple comparson, the ntervals are called smultaneous confdence ntervals, and the tests are called smultaneous hypothess test. Suppose an expermenter wshes to calculate m confdence ntervals, each havng a 100( 1 α *)% confdence level. Then each nterval wll be ndvdually correct wth probablty 1 α *. Let S be the event that the th confdence nterval wll be correct and = 1,...,m. Then, usng the standard rules for probabltes of unons and ntersectons of events, t follows that S the event that t wll be ncorrect ( ) P ( S S... S ) = 1 P( S S... ). 1 2 m 1 2 Sm Ths says that the probablty that all of the ntervals wll be correct s equal to one mnus the probablty that at least one wll be ncorrect. If 2 m=, P ( S1 S2) = P( S1) + P( S2) P( S1 S2) P( S ) + P( S ) 1 2 A smlar result, whch can be proved by mathematcal nducton, holds for any number m of events, that s, ( S S... S ) P( ) P, 1 2 m S wth equalty f the events P S are mutually exclusve. Consequently, 1, S2,..., Sm ( S S... S ) 1 P( S ) = 1 m * (1.1) 1 2 m α 2

3 Multple Comparson Procedures that s, the probablty that the m ntervals wll smultaneously be correct s at least 1 mα *. The probablty mα * s called the overall sgnfcance level or experment wse error rate or famly error rate. A typcal value for α * for a sngle confdence nterval s 0.05, so the probablty that sx confdence ntervals each calculated at a 95% ndvdual confdence level wll smultaneously be correct s at least 0.7. Although at least means bgger than or equal to, t s not known n practce how much bgger than 0.7 the probablty mght actually be. Ths s because the degree of overlap between the events S 1, S2,..., Sm s generally unknown. The probablty at least 0.7 translates nto an overall confdence level of at least 70% when the responses are observed. Smlarly, f an expermenter calculates ten confdence ntervals each havng ndvdual confdence level 95%, then the smultaneous confdence level for the ten ntervals s at least 50%, whch s not very nformatve. As m becomes larger the problem becomes worse, and when m 20, the overall confdence level s at least 0%, clearly a useless asserton! Smlar comments apply to the hypothess-testng stuaton. If m hypotheses are to be tested, each at sgnfcance level α *, then the probablty that at least one hypothess s ncorrectly reected s at most mα *. Varous methods have been developed to ensure that the overall confdence level s not too small and the overall sgnfcance level s not too hgh. Some methods are completely general, that s, they can be used for any set of estmable functons, whle others have been developed for very specalzed purposes such as comparng each treatment wth a control. Whch method s best depends on whch contrasts are of nterest and the number of contrasts to be nvestgated. Some of these methods can also be used for dentfyng the homogeneous subsets of treatment effects. Such procedures are called as multple range tests. Several methods are dscussed n the sequel some of them control the overall confdence level and overall sgnfcance level. Followng the lecture notes on Fundamentals of Desgn of Experments, let l t denote a treatment contrast, l 0 where t s the effect of treatment. The (BLUE) and the = confdence nterval for the above contrast can be obtaned as per procedure gven n the aforementoned lecture notes. However, besdes obtanng confdence ntervals one may be nterested n hypothess testng. The outcome of a hypothess test can be deduced from the correspondng confdence nterval n the followng way. The null hypothess H : l t = wll be reected at sgnfcance level α n favour of the two-sded 0 h alternatve hypothess fals to contan h. H : l t 1 h f the correspondng confdence nterval for In the followng secton, we dscuss the confdence ntervals and hypothess tests based on several methods of multple comparsons. A shorter confdence nterval corresponds to a more powerful hypothess test. l t 3

4 Multple Comparson Procedures 2. Multple Comparson Procedures The termnology a set of smultaneous 100( 1 α *)% confdence ntervals wll always refer to the fact that the overall confdence level for a set of contrasts or treatments means s (at least) 100( 1 α *)%. Each of the methods dscussed gves confdence ntervals of the form l t ltˆ ± w Vâr( ltˆ )...(2.1) where w,whch we call the crtcal coeffcent, depends on the method, the number of treatments v, on the number of confdence ntervals calculated, and on the number of error degrees of freedom. The term ( l tˆ ) msd = w Vâr (2.2) whch s added and subtracted from the least square estmate n (2.1) s called the mnmum sgnfcant dfference, because f the estmate s larger than msd, the confdence nterval excludes zero, and the contrast s sgnfcantly dfferent from zero. 2.1 The Least Sgnfcant Dfference (LSD) Method Suppose that, followng an analyss of varance F test where the null hypothess s reected, we wsh to test H : l t = aganst the alternatve hypothess H : l t. 0 0 For makng parwse comparsons, consder the contrasts of the type expermenters are often nterested, are obtanable from 1 0 t t n whch l t by puttng l = 1, l = 1 and zero for the other l 's. The 100 (1-α)% confdence nterval for ths contrast s ( ltˆ ± tedf, α / 2 Vâr( ltˆ ) ) l t (2.3) where edf denotes the error degrees of freedom. As we know that the outcome of a hypothess test can be deduced from the correspondng confdence nterval n the followng way. The null hypothess wll be reected at sgnfcance level α n favour of the two-sded alternatve hypothess f the correspondng confdence nterval for to contan 0. The nterval fals to contan 0 f the absolute value of l tˆ t Vâr l tˆ edf, α / 2 ( ) l t fals s bgger than. The crtcal dfference or the least sgnfcant dfference for testng the sgnfcance of the dfference of two treatment effects, say ( l tˆ ) t t s lsd = tedf, α / 2 Vâr, where t edf, α / 2 s the value of Student's t at the level of sgnfcance α and error degree of freedom. If the dfference of any two-treatment means s greater than the lsd value, the correspondng treatment effects are sgnfcantly dfferent. 4

5 Multple Comparson Procedures The above formula s qute general and partcular cases can be obtaned for dfferent expermental desgns. For example, the least sgnfcant dfference between two treatment effects for a randomzed complete block (RCB) desgn, wth v treatments and r replcatons s t ( v 1)(r 1), α / 2 2MSE / r, where t (v 1)(r 1), α / 2 s the value of Student's t at the level of sgnfcance α and degree of freedom (v - 1)(r - 1). For a completely randomzed desgn wth v treatments such that th treatment s replcated r tmes and v r = n, the total number of expermental unts, the least sgnfcant dfference between = two treatment effects s t ( n v), / 2 MSE. r r α + It may be worthwhle mentonng here that the least sgnfcant dfference method s sutable only for planned par comparsons. Ths test s based on ndvdual error rate. However, for those who wsh to use t for all possble parwse comparsons, should apply only after the F test n the analyss of varance s sgnfcant at desred level of sgnfcance. Ths procedure s often referred as Fsher's protected lsd. Snce F calls for us to accept or reect a hypothess smultaneously nvolvng means. If we arrange the treatment means n ascendng or descendng order of ther magntude and keep the, means n one group for whch the dfference between the smallest and largest mean s less than the lsd, we can dentfy the homogeneous subsets of treatments. For example, consder an experment that was conducted n completely randomzed desgn to compare the fve treatments and each treatment was replcated 5 tmes. F-test reects the null hypothess regardng the equalty of treatment means. The mean square error (MSE) s The means of fve treatments tred n experment are 9.8, 15.4, 17.6, 21.6 and 10.8 respectvely. The lsd for the above comparsons s 3.75, then the homogeneous subsets of treatments are Group1: Treatment 1 and 5, group 2: treatment 2 and 3 and group 3: treatment 4. Treatments wthn the same homogeneous subset are dentfed wth the same alphabet n the output from SAS. 2.2 Duncan's Multple Range Test A wdely used procedure for comparng all pars of means s the multple range test developed by Duncan (1955). The applcaton of Duncan's multple range test (DMRT) s smlar to that of lsd test. DMRT nvolves the computaton of numercal boundares that allow for the classfcaton of the dfference between any two treatment means as sgnfcant or non-sgnfcant. DMRT requres computaton of a seres of values each correspondng to a specfc set of par comparsons unlke a sngle value for all parwse comparsons n case of lsd. It prmarly depends on the standard error of the mean dfference as n case of lsd. Ths can easly be worked out usng the estmate of varance of an estmated elementary treatment contrast through the desgn. For applcaton of the DMRT rank all the treatment means n decreasng or ncreasng order based on the preference of the character under study. For example for the yeld data, the rank 1 s gven to the treatment wth hghest yeld and for the pest ncdence the treatment wth the least nfestaton should get the rank as 1. Consder the same example as n case of lsd. The ranks of the treatments are gven below: 5

6 Multple Comparson Procedures Treatments T1 T5 T2 T3 T4 Treatment Means Rank Compute the standard error of the dfference of means (SE d ) that s same as that of square root of the estmate of the varance of the estmated elementary contrast through the desgn. In the present example ths s gven by 2 (8.06) / 5 = Now obtan the value rα (p, edf ) *SEd of the least sgnfcant range R p =, where α s the desred sgnfcance 2 level, edf s the error degrees of freedom and p = 2,, v s one more than the dstance n rank between the pars of the treatment means to be compared. If the two treatment means have consecutve rankngs, then p = 2 and for the hghest and lowest means t s v. The r α p,edf can be obtaned from Duncan's table of sgnfcant ranges. values of ( ) For the above example the values of r α ( p,edf) at 20 degrees of freedom and 5% level of sgnfcance are r ( 2,20) = 2. 95, r ( 3,20) = 3. 10, r. 05 ( 4,20) ( 5,20) R are r =. Now the least sgnfcant ranges p R 2 R 3 R 4 R = and Then, the observed dfferences between means are tested, begnnng wth largest versus smallest, whch would be compared wth the least sgnfcant range R v. Next, the dfference of the largest and the second smallest s computed and compared wth the least sgnfcant range R v 1. These comparsons are contnued untl all means have been compared wth the largest mean. Fnally, the dfference of the second largest mean and the smallest s computed and compared aganst the least sgnfcant range R v 1. Ths process s contnued untl the dfferences of all possble v(v 1) 2 pars of means have been consdered. If an observed dfference s greater than the correspondng least sgnfcant range, then we conclude that the par of means n queston s sgnfcantly dfferent. To prevent contradctons, no dfferences between a par of means are consdered sgnfcant f the two means nvolved fall between two other means that do not dffer sgnfcantly. For our case the comparsons wll yeld 4 vs 1: = 11.8 > 4.13( R 5 ); 4 vs 5: = 10.8 > 4.04( R 4 ); 4 vs 2: = 6.2 > 3.94( R 3 ); 4 vs 3: = 4.0 > 3.75( R 2 ); 3 vs 1: = 7.8 > 4.04( R 4 ); 3 vs 5: = 6.8 > 3.94( R 3 ); 3 vs 2: = 2.2 < 3.75( R 2 ); 2 vs 1: = 5.6 > 3.94( R 3 ); 6

7 Multple Comparson Procedures 2 vs 5: = 4.6 > 3.75( R 2 ); 4 vs 1: = 1.0 < 3.75( R 2 ); We see that there are sgnfcant dfferences between all pars of treatments except T3 and T2 and T5 and T1. A graph underlnng those means that are not sgnfcantly dfferent s shown below: T1 T5 T2 T3 T It can easly be seen that the confdence ntervals of the desred parwse comparsons followng (2.1) s ( p, edf) rα l t l ± tˆ Vâr ltˆ (2.4) 2 and least sgnfcant range n general s ( p,edf) rα lsr = Vâr l 2 tˆ. The methods of multple comparson gven n Sectons 2.1 and 2.2 uses ndvdual error rates (probablty that a gven confdence nterval wll not contan the true dfference n level means). Ths may be msleadng as s clear from nequalty (1.1),,e., f m smultaneous confdence ntervals are calculated for preplanned contrasts, and f each confdence nterval has confdence level 100( 1 α *)% then the overall confdence level s greater than or equal to 100( 1 mα *)%. Therefore, the methods of multple comparsons that utlze experment wse error rate or famly error rate (Maxmum probablty of obtanng one or more confdence ntervals that do not contan the true dfference between level means) may be qute useful. In the sequel, we descrbe some methods of multple comparsons that are based on famly error rates. 2.3 Bonferron Method for Preplanned Comparsons In ths method the overall confdence level of 100( 1 α *)% for m smultaneous confdence ntervals can be ensured by settng α * = α / m. Replacng α by α / m n the formula (2.3) for an ndvdual confdence nterval, we obtan a formula for a set of smultaneous 100( 1 α *)% confdence ntervals for m preplanned contrasts l t s l t l ± tˆ tedf, α / 2m Vâr ltˆ (2.5) Therefore, f the contrast estmate s greater than correspondng contrast s sgnfcantly dfferent from zero. t edf, α / 2m Vâr ltˆ the It can easly be seen that ths method s same as that of least sgnfcant dfference wth α n least sgnfcant dfference to be replaced by / m α / 2m s lkely to be a α. Snce ( ) 7

8 Multple Comparson Procedures typcal value, the percentles tedf, α /( 2m) may need to be obtaned by use of a computer package. When m s very large, α /( 2m) s very small, possbly resultng n extremely wde smultaneous confdence ntervals. In ths case the Scheffe or Tukey methods descrbed n the sequel would be preferred. Note that ths method can be used only for preplanned contrasts or any m preplanned estmable contrasts or functons of the parameters. It gves shorter confdence ntervals than the other methods lsted here f m s small. It can be used for any desgn. However, t cannot be used for data snoopng. An expermenter who looks at the data and then proceeds to calculate smultaneous confdence ntervals for the few contrasts that look nterestng has effectvely calculated a very large number of ntervals. Ths s because the nterestng contrasts are usually those that seem to be sgnfcantly dfferent from zero, and a rough mental calculaton of the estmates of a large number of contrasts has to be done to dentfy these nterestng contrasts. Scheffe s method should be used for contrasts that were selected after the data were examned. 2.4 Scheffe Method of Multple Comparsons In the Bonferron method of multple comparsons, the maor problem s that the m contrasts to be examned must be preplanned and the confdence ntervals can become very wde f m s large. Scheffe's method, on the other hand, provdes a set of smultaneous 100( 1 α *)% confdence ntervals whose wdths are determned only by the number of treatments and the number of observatons n the experment. It s not dependent on the number of contrasts are of nterest. It utlzes the fact that every possble contrast l t can be wrtten as a lnear combnaton of the set of ( v 1) treatment - versus - control contrasts, t2 t1,t3 t1,...,tv t1. Once the expermental data have been collected, t s possble to fnd a 100( 1 α *)% confdence regon for these v 1 treatment - versus - control contrasts. The confdence regon not only determnes confdence bounds for each treatment - versus - control contrasts, t determnes bounds for every possble contrast l t and, n fact, for any number of contrasts, whle the overall confdence level remans fxed. For mathematcal detals, one my refer to Scheffe (1959) and Dean and l can be Voss (1999). Smultaneous confdence ntervals for all the contrasts obtaned from the general formula (2.1) by replacng the crtcal coeffcent w by where w wth a as the dmenson of the space of lnear estmable functons s = af a,edf, α beng consdered, or equvalently, a s the number of degrees of freedom assocated wth the lnear estmable functons beng consdered. The Scheffe's method apples to any m estmable contrasts or functons of the parameters. It gves shorter ntervals than Bonferron method when m s large and allows data snoopng. It can be used for any desgn. 2.5 Tukey Method for All Parwse Comparsons Tukey (1953) proposed a method for makng all possble parwse treatment comparsons. The test compares the dfference between each par of treatment effects wth approprate adustment for multple testng. Ths test s also known as Tukey s honestly sgnfcant dfference test or Tukey s HSD. The confdence ntervals obtaned usng ths method are shorter than those obtaned from Bonferron and Scheffe methods. Followng the formula t w s 8

9 Multple Comparson Procedures (2.1), one can obtan the smultaneous confdence ntervals for all the contrasts of the type l t by replacng the crtcal coeffcent w by w t = qv, edf, α / 2 where v s the number of treatments and edf s the error degree of freedom and values can be seen as the percentle correspondng to a probablty level α n the rght hand tal of the studentzed range dstrbuton tables. For the completely randomzed desgn or the one-way analyss of varance model, Vâr(tˆ tˆ ) = 1 1 MSE +, where r denotes the replcaton number of treatment r r ( = 1,2,, v ). Then Tukey's smultaneous confdence ntervals for all parwse comparsons t t, wth overall confdence level at least 100( 1 α *)% s obtaned by takng w t = qv, n-v, α / 2 and Vâr(tˆ tˆ ) = 1 1 MSE +. The values of q v,n v, α r r can be seen the studentzed range dstrbuton tables. When the sample szes are equal ( r r;= 1,...,v) =, the overall confdence level s exactly 100( 1 α *)%. When the sample szes are unequal, the confdence level s at least 100( 1 α *)%. It may be mentoned here that Tukey's method s the best for all parwse treatment comparsons. It can be used for completely randomzed desgns, randomzed complete block desgns and balanced ncomplete block desgns. It s beleved to be applcable (conservatve, true α level lower than stated) for other ncomplete block desgns as well, but ths has not yet been proven. It can be extended to nclude all contrasts but Scheffe's method s generally better for these types of contrasts. 2.6 Dunnett Method for Treatment-Versus-Control Comparsons Dunnett (1955) developed a method of multple comparsons for obtanng a set of smultaneous confdence ntervals for preplanned treatment-versus-control contrasts t t1( = 2,...,v) where level 1 corresponds to the control treatment. The ntervals are shorter than those gven by the Scheffe, Tukey and Bonferron methods, but the method should not be used for any other type of contrasts. For detals on ths method, a reference may be made to Dunnett (1955, 1964) and Hochberg and Tamhane (1987). In general ths procedure s, therefore, best for all treatment-versus-control comparsons. It can be used for completely randomzed desgns, randomzed complete block desgns. It can also be used for balanced ncomplete block desgns but not n other ncomplete block desgns wthout modfcatons to the correspondng multvarate t-dstrbuton tables gven n Hochberg and Tamhane (1987). However, not much lterature s avalable for multple comparson procedures for makng smultaneous confdence statement about several test treatments wth several control treatments comparsons. A partal soluton to the above problem has been gven by Hoover (1991). 9

10 Multple Comparson Procedures 2.7 Hsu Method for Multple Comparsons wth the Best Treatment Multple comparsons wth the best treatment s smlar to multple comparsons wth a control, except that snce t s unknown pror to the experment whch treatment s the best, a control treatment has not been desgnated. Hsu (1984) developed a method n whch each treatment sample mean s compared wth the best of the others, allowng some treatments to be elmnated as worse than best, and allowng one treatment to be dentfed as best f all others are elmnated. Hsu calls ths method RSMCB, whch stands for Rankng, Selecton and Multple Comparsons wth the Best treatment. Suppose, frst, that the best treatment s the treatment that gves the largest response on th t max t denote the effect of the treatment mnus the effect of the average. Let ( ) best of the other v 1 treatments. When the be the effect of the second-best treatment. So, ( ) s the best, zero f the s worse than best. th treatment s the best, ( t )( ) max wll t max t wll be postve f treatment th treatment s ted for beng the best, or negatve f the treatment If the best treatment factor level s the level that gves the smallest response rather than the t mn t n place of largest, then Hsu s procedure has to be modfed by takng ( ) t max( t ). To summarze, Hsu's method for multple comparsons selects the best treatment and dentfes those treatments that are sgnfcantly worse than the best. It can be used for completely randomzed desgns, randomzed block desgns and balanced ncomplete block desgns. For usng t n other ncomplete block desgns, modfcatons of the tables s requred. 3. Multple Comparson Procedures usng SAS/ SPSS The MEANS statement n PROC GLM or PROC ANOVA can be used to generate the observed means of each level of a treatment factor. The TUKEY, BON, SCHEFFE, LSD, DUNCAN, etc. optons under MEANS statement causes the SAS to use Tukey, Bonferron, Scheffe's, least sgnfcant dfference, Duncan's Multple Range Test methods to compare the effects of each par of levels. The opton CLDIFF asks the results of above methods be presented n the form of confdence ntervals. The opton DUNNETT ('1') requests Dunnett's 2-sded method of comparng all treatments wth a control, specfyng level '1' as the control treatment. Smlarly the optons DUNNETTL ('1') and DUNNETTU ('1') can be used for rght hand and left hand method of comparng all treatments wth a control. To Specfy Post Hoc Tests for GLM Procedures n SPSS: From the menus choose: Analyze General Lnear Model From the menu, choose Unvarate, Multvarate, or Repeated Measures In the dalog box, clck Post Hoc Select the factors to analyze and move them to the Post Hoc Tests For lst Select the desred tests. Please note that Post hoc tests are not avalable when covarates have been specfed n the model. GLM 10

11 Multple Comparson Procedures Multvarate and GLM Repeated Measures are avalable only f you have the Advanced Models opton nstalled. 4. Conclusons Each of the methods of multple comparsons at subsectons 2.3 to 2.7 allows the expermenter to control the overall confdence level, and the same methods can be used to control the experment wse error rate when multple hypotheses are to be tested. There exst other multple comparson procedures that are more powerful (.e. that more easly detect a nonzero contrast) but do not control the overall confdence level nor the experment wse error rate. Whle some of these are used qute commonly, however, we don't advocate ther use. The selecton of the approprate multple comparson method depends on the desred nference. As dscussed n Secton 3 that for makng all possble parwse treatment comparsons, the Tukey s method s not conservatve and gves smaller confdence ntervals as compared to Bonferron, Sdak and Scheffe s methods. Therefore, one may choose Tukey s method for makng all possble parwse comparsons. For more detals on methods of multple comparsons, one may refer to Steel and Torre (1981), Gomez and Gomez (1984) and Montgomery (1991), Hsu (1996), Dean and Voss (1999). References and Suggested Readng Dean, A. and Voss, D.(1999). Desgn and Analyss of Experments. Sprnger Texts n Statstcs, Sprnger, New York Duncan, D.B. (1955). Multple range and multple F-Tests. Bometrcs, 11, Dunnett, C.W.(1955). A multple comparsons procedure for comparng several treatments wth a control. J. Am. Statst. Assoc., 50, Dunnett, C.W.(1964). New tables for multple comparsons wth a control. Bometrcs, 20, Gomez, K.A. and Gomez, A.A. (1984). Statstcal Procedures for Agrcultural Research, 2 nd Edton. John Wley and Sons, New York. Hayter, A.J. (1984). A proof of the conecture that the Tukey-Cramer multple comparson procedure s conservatve. Ann. Statst., 12, Hochberg, Y. and Tamhane, A.C.(1987). Multple Comparson Procedures. John Wley and Sons, New York. Hoover, D.R.(1991). Smultaneous comparsons of multple treatments to two (or more) controls. Bom. J., 33, Hsu, J.C. (1984). Rankng and Selecton and Multple Comparsons wth the Best. Desgn of Experments: Rankng and Selecton (Essays n Honour of Robert E.Bechhofer). Edtors: T.J.Santner and A.C.Tamhane , Marcel Dekker, New York. Hsu, J.C. (1996). Multple Comparsons: Theory and Methods. Chapman & Hall. London. Montgomery, D.C.(1991). Desgn and Analyss of Experments, 3 rd edton. John Wley & Sons. New York Peser, A.M. (1943). Asymptotc formulas for sgnfcance levels of certan dstrbutons. Ann. Math. Statst., 14, (Correcton 1949, Ann.Math. Statst., 20, ). Scheffe, H.(1959). The Analyss of Varance. John Wley & Sons. New York. Steel, R.G.D. and Torre,J.H.(1981). Prncples and Procedures of Statstcs: A Bometrcal Approach. McGraw-Hll Book Company, Sngapore, Tukey, J.W.(1953). The Problem of Multple Comparsons. Dttoed Manuscrpt of 396 Pages, Department of Statstcs, Prnceton Unversty. 11