Permutation Tests for More Than Two Samples

Transcription

1 Permutato Tests for ore Tha Two Samples Ferry Butar Butar, Ph.D. Abstract A F statstc s a classcal test for the aalyss of varace where the uderlyg dstrbuto s a ormal. For uspecfed dstrbutos, the permutato test s easy to utlze both for the orgal observatos ad ras. If oe ows a pror the drecto of the alteratve hypothess the oe ca easly use a Resamplg Stats to obta a p-value. Ths test s easy to mplemet ad the sgfcace level s exact whe calculatg all possble permutatos. The approxmate sgfcace level ca be used whe the umbers of permutatos are very large. Itroducto There are may felds do emprcal ad extesve use of expermetato, for example, the research educato, agrcultural, medce, egeerg, dustry ad psychology. Statstcal methods ca crease the effcecy of these expermets ad stregthes the coclusos so obtaed. I order to the expermeter eeds to use the smple statstcal techques, he eeds to desg the data as smple as possble. Oe-way aalyss of varace s oe of the techques to use f oe wats to test that there s o dfferece betwee treatmet, treatmet, up to treatmet. If the uderlyg dstrbuto of the observatos s ormal, the oe usually uses a F statstc. ost practtoers hardly chec whether ther data come from a ormal dstrbuto, but they stll force to use a F statstc. I order to use a F statstc, there are some assumptos eed to satsfy such as the radom sample for treatmets are mutually depedet, a radom sample comes from a ormal dstrbuto, ad the varace for each of the treatmets are equal ad costat. Note that we observed may graduate studets (or doctoral studets) from educato major here at our uversty use a F statstc eve though ther data are qualtatve. The permutato test does ot eed all of the assumptos above. Uder the ull hypothess, there s o dfferece amog treatmet effects. Thus ay observatos from oe treatmet ca be permuted to ay other treatmets. Ths s called the data are exchageable. For example, suppose there are two classes statstcs. Class A cossts of 0 studets ad class B cossts of 8 studets. The uder ull hypothess that there s o dfferece betwee these two classes, ay studets class A ca st class B or vce versa. I permutato test you ca relax to choose your ow statstc ad use t to perform the aalyss. You ca use your orgal data set as a bass for permutato test or you ca use the ras of the data. See (Good, 994; Hggs, 004; Hollader ad Wolf, 999). Joural of athematcal Sceces & athematcs Educato 0

2 Testg Hypothess ad Data Layout Suppose observatos are radomly selected from populato wth cumulatve dstrbuto fucto (cdf) F (x), F (x),...,f (x). The ull hypothess to be tested s equalty of dstrbuto, that s H 0 : F (x) = F (x) =...= F (x) ad the alteratve hypothess s gve by H a : F (x) F j (x) or F (x) F j (x) wth strct equalty hold for at least oe x. The alteratve of hypothess ca be thought as the shft locatos of parameters μ, μ,...,μ ot all equal so that H a : F (x) = F(x μ ). Ths shft ca be modeled as the aalyss of varace as j = μ + ε j, where j s the j th respose varable j=,..., for the th treatmet =,..., ad μ are meas of treatmet, ad ε j s are depedet ad detcally dstrbuted radom varable wth dstrbuto F(ε). The followg table (I) s the layout of the observato. Table I Oe-Way Data Layout Treatmets Observatos Sampleszes eas Varaces,,,..., j,...,,..., j,..., S S,,..., j,..., S,,..., j,..., S Test Statstc The t-test statstc s a classcal test for comparg two populatos uder ormalty assumptos. Ad a classcal test for comparg several populatos (treatmets) s a F test statstc. Based o Aalyss of Varace (ANOVA) Joural of athematcal Sceces & athematcs Educato

3 techques, we ca partto the total sum of squares terms of sum squares betwee treatmets ad sum squares wth treatmets, that s (see Kuehl, 994) ( j ) = ( j + ) = j= = j= = = ( j ) + ( ) = j= = j= ( ) S + ( ) = = = SSE + SST, where = N, = j = j N = = The mea squares for treatmet s = ST = SST /( ) = ( ) ad the mea squares for error s SE = SSE /( N ) = ( N ) = Joural of athematcal Sceces & athematcs Educato, = j = ( ) ( ) S. A test statstc s gve by F = ST/SE. If the dstrbuto of the observato s ormal ad the varace s costat, the test statstc F has a F dstrbuto wth degrees of freedom - for the umerator ad N- for the deomator. Usg ths dstrbuto oe ca calculate ts p-value. I practce, we may ot ow or ot assume that the observatos come from a ormal dstrbuto. Sce observatos are depedet ad detcally dstrbuted, hece they are exchageable, so we ca employ the permutato dstrbuto of F. Permutato Tests R. A Fsher (935) was the frst to troduce the dea of permutg data amog treatmets as a way of statstcal ferece. Sce the a cosderable theoretcas have developed ths dea. I the cotext of -sample problems (>), the permutato tests s that all samples were radomly selected from the same populato ad radomly assged to label,,... Calculate the test statstc for the orgal data set, called t TS obs. The exame all possble permutato of the N observatos amog treatmets where there are observatos treatmet, =,,...,. All N! possble permutatos are. Calculate the statstc for each!!...! permutatos. Now obta the p-value as the fracto of statstc from all permutatos calculated above that are greater tha or equal to TS obs. Ths p- value s exact proporto sce we calculate from all possble permutatos ad j.

4 usually called as sgfcace of the test. (See Butar Butar ad Jae-Wa Par, 007). If t s ot possble to cosder all possble permutatos, the we radomly select sample of R permutatos. I practce a radom sample of s usually suffcet to approxmate a p-value. Ths procedure does ot requre aalytcal dervato of test statstc uder the ull hypothess. There s a relaxato choosg the test statstc. Wth ths relaxato, ths permutato test has advatageous over the parametrc test (Hollader ad Wolfe, 999. The proposed Formula We would le to fd a smple test statstc, easy to mplemet but powerful result. Recall that the total sum of squares (TSS) s TSS = ( ) = j = j. Note that TSS = SST + SSE. For every permutato, total sum of squares remas the same value, let's call ths as a costat, C. Thus the F statstc ca be wrtte as SST /( ) F =. ( C SST ) /( N ) Sce F s a creasg fucto of SST, hece the test statstc ca reduce based o SST ad s equvalet to a test based o F statstc. Note that sum of squares treatmet ca be smplfed to ( ) = N, = = SST = whch shorte the calculatos. Also, sce has the same value for every permutato of the observatos, we ca use our test statstc whch reduces to = = SS Ths test s aga easy to mplemet o the Resamplg Stats software (000). Example Data were collected o studet teachers relatve to ther use of certa teachg strateges that had bee preseted to them preservce educato. I 978 there were 6 teachers who dd ot lear to use the strateges ad they were used as a cotrol group. There were 9 979, 9 980, ad 0 98 studets teachers who had leared to use strateges. The vestgator recorded the average umber of strateges used per wee by each of the studet teachers durg studet teachg assgmets. The hypothess s the umber of strateges used by the studets were dfferet amog the years. Joural of athematcal Sceces & athematcs Educato 3,

5 Table II Average Number of Dfferet Strateges Used The permutato F ad SS test are mplemeted Resamplg Stats. The orgal F observed data s.335. Based o 0000 radomly selected permutatos, the p-value are ad , respectvely. The 80 th, 85 th, 90 th, 95 th, 97.5 th ad 99 th percetles of the permutatos were foud to be.5978,.848,.947,.8445, , 4.469, respectvely. The statstc s ot sgfcat for ay level. The Krusal-Walls Test Statstc The Krusal-Walls test use ras stead of the orgal observatos. Smlar to the table, let R j be the ra of observato j. Let N= To determe the ras of all the observatos we combe the sample szes treatmets. If all observatos are dstct, let be the ra of the smallest observato, be the ra of the secod smallest observato, ad N be the ra of the largest observato. The sum of the ra s N (N+)/. The Krusal- Walls test statstc s smlar to the F test statstc ad s gve by where S R R = N KW = ( S R ) = ( R R ), () s the sample varace of the combed ras. Note that N R = = j = j, ad + Joural of athematcal Sceces & athematcs Educato 4

6 S R = ( N ) = ( N ) N( N + ) = ( N ) N( N + ) = = j= ( R R ) N( N + ) = R j= j 4 N( N + )(N + ) N( N + ) = 6( N ) 4( N ) ( (N + ) 3( N + ) ) j () Thus, usg (), equato () becomes KW = N( N + ) = N + R (3) We ca fd a p-value for the KW statstc from the permutato dstrbuto of ths statstc. Ths ca be doe whch s smlar to the F statstc, except that we use ra stead of the orgal observatos. Note that for large sample szes, N + R ca be approxmated by χ dstrbuto wth - degrees = of freedom. We ca also N + smplfy R becomes = = proposed test statstc s gve by R N( N + ) 4 Joural of athematcal Sceces & athematcs Educato 5 KW = = R. Thus, the If two or more observatos have same values or tes, the we use the average ras of tes ad assged the average ra to each of the te observatos. For example, the orgal observatos are: Sce two values has the same umber 3, assg ad rag to them ad average those ras ad assg to each of 3, thus fal rags are: 3,.5,.5, 4. Comparso betwee F ad Permutato Dstrbutos Cosder four treatmets geerated from ormal dstrbuto. Observatos for treatmet,, 3, ad 4 were geerated from ormal dstrbuto wth mea

7 0, 0, 30 ad 40 wth the same stadard devato of 0. The followg table s the observatos geerated from the above statemet. Table III Samples from Normal populato, μ =0, μ =0, μ 3 = 30, μ 4 =40, σ=0 Treatmet Treatmet Treatmet Treatmet Usg the Resamplg Stats software, we radomly selected 0000 permutatos ad obta the permutato dstrbuto. The followg table compares the percetles of the F dstrbuto wth 3 degrees of freedom for umerator ad 6 degrees of freedom for deomator to the permutato dstrbutos. Table IV Comparso of Permutato Percetles to F-dstrbuto Percetles Percetle Permutato Dstrbuto F-dstrbuto From the table whch has oly fve observatos per treatmet, we ca coclude that there s ot much dfferet betwee the values of F dstrbutos ad the permutato dstrbutos. Ordered Alteratves Hypothess It s possble researchers have pror belef that the observatos from treatmet ted to be smaller tha observatos from treatmet, ad so o. For example qualty or quatty of materals, severty of dsease, drug dosage levels, ad temperature. The Krusal-Walls statstc does ot employ ay such pror formato regardg a alteratve hypothess. The Jocheere-Terpstra statstc s more powerful tha the Krusal-Walls statstc whe the treatmets ca be ordered a pror partcular drecto. Let θ be the populato meda for the th treatmet. The ull hypothess ca be wrtte as H 0 : θ = θ =... = θ Joural of athematcal Sceces & athematcs Educato 6

8 versus the alteratve hypothess H : θ θ... θ, where at least oe of the alteratves s strct equalty. Note that the orderg of the hypothess must be specfed before the observatos are collected. I terms of the cdf's, let F (x) be the cdf of treatmet, the the alteratve hypothess ca be wrtte as H F ( x) F ( x)... F ( ). : x Let the observatos from treatmet be deoted,,..., ad the observatos from treatmet be deoted,,...,. If the data have o tes, the ay gve observatos are ether strctly less tha or strctly greater tha ay other observato. The a-whtey statstc, deoted U j s defed as U = umber of pars ) for whch <. j The Jocheere-Terpstra statstc, JT, s gve by JT = < j (, U j = U j. = j= + To compute a p-value from the permutatos test, frst obta the JT obs from the observed data. Secod, f possble exame all possble permutatos of the data amog the treatmets. If ths ot feasble exame a radomly selected subset of the permutatos. I practce o more tha 000 s eough for a smulato. Record a f a statstc you just foud greater or equal to the JT obs. Last, the upper-tal p-value s the total umber of 's dvded by umber of permutatos. Note that oe ca also use the Wlcoxo ra-sum statstc for calculatg U j. Example The followg data from the Natoal Trasportato Safety Admstrato are left femur loads o drver-sde crash dummes for automobles varous weght classes. We wat to test whether the left femur loads o drver-sde are creasg order of vehcle weght classfcato from the weght of 700 lb to weght of 3700 lb. Data source: The permutato test based o Wlcoxo ra sum test was appled to the JT statstc to the above table. The value of the JT statstc based o the orgal data was Based o 5000 radomly selected permutatos, the 80 th, 85 th, 90 th, 97.5 th, ad 99 th percetles of the permutato dstrbuto foud to be 35.5, 364.5, 38.5, 407.5, 43.5, 45.3, respectvely. Therefore, the statstc s sgfcat oly at the 0% level. A approxmate permutato p-value s Joural of athematcal Sceces & athematcs Educato 7

9 Table V Vehcle Weght Classfcato 700 lb 300 lb 800 lb 300 lb 3700 lb Cocluso The percetle for a permutato dstrbuto s comparable to the F dstrbuto oe-way aalyss of varace. Ths permutato test ca be used as a approxmate to a F test especally whe the sample sze s large. Oe ca smplfy the formula to reduce the calculato tme. The Krusal-Walls s the test whch s smlar to a F test wth the ras used stead of the orgal observatos. If oe ows the order of alteratve before collectg the data, the the more powerful test s the Jocheere-Terpstra test. The Resamplg Stats ca be use to perform all of the methods above. Ferry Butar Butar, Ph.D., Sam Housto State Uversty, USA Refereces Butar Butar, F, ad Jea-Wa Par (007). Permutato tests for comparg two populatos, Techcal report, Sam Housto State Uversty. Coover, W. J. (999). Practcal Noparametrc Statstcs, thrd ed., Joh Wley & Sos, New Yor. Good, P. (994). Permutato Tests: A practcal gude to resamplg methods for testg hypothess, Sprger, New Yor. Hcs, C.R., Turer, K.V. (999). Fudametal Cocepts the Desg of Expermets, Ffth ed., Oxford Uversty Press, New Yor. Hggs, J.J. (004). Itroducto to oder Noparametrc Statstcs, Duxbury. Hollader,., ad Wolfe, D. A. (999). Noparametrc Statstcal ethods, Secod ed., Wley, New Yor. Joural of athematcal Sceces & athematcs Educato 8

10 Johso, R. A., ad Wcher, D.W. (000). Appled ultvarate Statstcal Aalyss, fourth ed., Pretce Hall, New Jersey. Kuehl, R. O. (994). Statstcal Prcples of Research Desg ad Aalyss, Duxbury. Resamplg Stats User's Gude. (000). Resamplg Stats, Ic., Vol 5.0., Arlgto, VA. Segel, S. ad Castella, N.J. (998). Noparametrc Statstcs for the Behavoral Sceces, secod ed. cgraw Hll, Bosto. Joural of athematcal Sceces & athematcs Educato 9