Power Comparison of Some Goodness-of-fit Tests

Transcription

1 Florida Iteratioal Uiversity FIU Digital Commos FIU Electroic Theses ad Dissertatios Uiversity Graduate School Power Compariso of Some Goodess-of-fit Tests Tiayi Liu DOI: /etd.FIDC Follow this ad additioal works at: Part of the Statistical Methodology Commos Recommeded Citatio Liu, Tiayi, "Power Compariso of Some Goodess-of-fit Tests" (2016). FIU Electroic Theses ad Dissertatios This work is brought to you for free ad ope access by the Uiversity Graduate School at FIU Digital Commos. It has bee accepted for iclusio i FIU Electroic Theses ad Dissertatios by a authorized admiistrator of FIU Digital Commos. For more iformatio, please cotact dcc@fiu.edu.

2 FLORIDA INTERNATIONAL UNIVERSITY Miami, Florida POWER COMPARISON OF SOME GOODNESS-OF-FIT TESTS A thesis submitted i partial fulfillmet of the requiremets for the degree of MASTER OF SCIENCE i STATISTICS by Tiayi Liu 2016

3 To: Dea Michael R. Heithaus College of Arts, Scieces ad Educatio This thesis, writte by Tiayi Liu, ad etitled Power Compariso of Some Goodess-of-fit Tests, havig bee approved i respect to style ad itellectual cotet, is referred to you for judgmet. We have read this thesis ad recommed that it be approved. Gauri Ghai Florece George Zhemi Che, Major Professor Date of Defese: July 6, 2016 The thesis of Tiayi Liu is approved. Dea Michael R. Heithaus College of Arts, Scieces ad Educatio Adrés G. Gil Vice Presidet for Research ad Ecoomic Developmet ad Dea of the Uiversity Graduate School Florida Iteratioal Uiversity, 2016 ii

4 ABSTRACT OF THE THESIS POWER COMPARISON OF SOME GOODNESS-OF-FIT TESTS by Tiayi Liu Florida Iteratioal Uiversity, 2016 Miami, Florida Zhemi Che, Major Professor There are some existig commoly used goodess-of-fit tests, such as the Kolmogorov-Smirov test, the Cramer-Vo Mises test, ad the Aderso-Darlig test. I additio, a ew goodess-of-fit test amed G test was proposed by Che ad Ye (2009). The purpose of this thesis is to compare the performace of some goodess-offit tests by comparig their power. A goodess-of-fit test is usually used whe judgig whether or ot the uderlyig populatio distributio differs from a specific distributio. This research focus o testig whether the uderlyig populatio distributio is a expoetial distributio. To coduct statistical simulatio, SAS/IML is used i this research. Some alterative distributios such as the triagle distributio, V-shaped triagle distributio are used. By applyig Mote Carlo simulatio, it ca be cocluded that the performace of the Kolmogorov-Smirov test is better tha the G test i may cases, while the G test performs well i some cases. iii

5 TABLE OF CONTENTS CHAPTER PAGE I. INTRODUCTION Itroductio Basic Ideas... 3 II. METHODOLOGY Fidig Critical Values Decisio Rules... 8 III. POWER COMPARISON Selected Alterative Distributios Summary of The Results IV. CONCLUSION AND DISCUSSION REFERENCES iv

6 LIST OF TABLES TABLES PAGE 1. Critical Values of G Test Statistic Critical Values of K-S Test Statistic Power Compariso: Triagle (h=0.25) Power Compariso: Triagle (h=0.5) Power Compariso: Triagle (h=0.75) Power Compariso: V-shaped Triagle (h=0.25) Power Compariso: V-shaped Triagle (h=0.5) Power Compariso: V-shaped Triagle (h=0.75)...20 v

7 LIST OF FIGURES FIGURES PAGE 1. Alterative Distributio 1: Triagle (h=0.25) Alterative Distributio 2: Triagle (h=0.5) Alterative Distributio 3: Triagle (h=0.75) Alterative Distributio 4: V-shaped Triagle (h=0.25) Alterative Distributio 5: V-shaped Triagle (h=0.5) Alterative Distributio 6: V-shaped Triagle (h=0.75) 22 vi

8 CHAPTER I INTRODUCTION 1.1 Itroductio The goodess-of-fit test is usually used whe judgig whether or ot the uderlyig populatio distributio, from which a sample is draw, differs from a specific distributio. The method ca be used for testig ay specified distributios. I the preset thesis, the problem of testig whether a populatio distributio is a expoetial distributio is discussed. Goodess-of-fit tests typically summarize the differece betwee observed values ad expected values i the give model. Various test methods have bee published i the literature. There are some commoly use goodess-of-fit tests icludig the Chi-squared test (Pearso, 1900), the Kolmogorov- Smirov test (Kolmogorov, 1933 ad Smirov, 1939), the Cramer-Vo Mises test (Cramer ad vo Mises, 1928), ad the Aderso-Darlig test (Aderso ad Darlig, 1952). To determie which test should be applied while testig for differet distributios, power compariso plays a importat role. It has bee show that oe of the existig statistical tests ca be cosidered the best test. I the recet years, some ew statistical tests have bee developed to raise the power of goodess-of-fit test. Che ad Ye (2009) proposed a ew method for testig whether the populatio distributio is a uiform distributio. The proposed test is origially for testig uiformity. However, by applyig the well-kow probability itegral trasformatio, the proposed test ca be used to test for ay specified distributio. 1

9 The curret research will discuss the performace of some existig goodessof-fit tests whe they are used to check whether or ot the uderlyig probability distributio is a expoetial distributio. Mote Carlo simulatio will be used to compare the power of those give tests. The Chi-square test, also kow as the Pearso s Chi-square test is a wellkow oparametric goodess-of-fit test. Chi-square test is widely used i may cases due to the cetral limit theorem. However, whe the sample size is small, the performace of Chi-square test is ot satisfactory. The Kolmogorov-Smirov (K-S) test is the most popular oparametric goodess-of-fit test. The test was proposed by Kolmogorov ad Smirov (1933 ad 1939). The K-S test statistic D measures the distace betwee the empirical distributio fuctio (EDF) usig the observed data ad the hypothesized distributio fuctio F(x). * The test statistic of the K-S test ca be writte as D sup F ( x) F( x). Past research showed that K-S test may be preferred over the Chi-square test if the sample size is small. The Cramer-vo Mises test is a alterative of K-S test. The test was developed by Harald Cramer ad Richard Edler vo Mises ( ). The test has bee show to be more powerful compared to the K-S test for some alterative x hypotheses. The origial test statistic, W 2, is defied as * 2 * [ ( ) ( )] ( ) F x F x df x, where F is a give EDF of the observed data ad F * is a CDF of the hypothesized distributio. 2

10 Aderso ad Darlig (1952) further adapted the Cramer-vo Mises test, ad itroduced a ew test statistic A 2, calculated by [ F ( x) F ( x)] * 2 * df x * * 2 F ( x) ( F ( x)) ( ). It has bee show that the Aderso-Darlig test ca be more powerful tha K-S test uder some situatios. For istace, whe testig the ormality of the observed data, Aderso-Darlig test provides oe of the most powerful statistic for detectig a ormal distributio adequacy. 1.2 Basic Ideas The test proposed by Che ad Ye uses a differet method to test uiformity. A power study has show that this test ca provide quite decet power for testig uiformity. As metioed above, the test ca be used for testig ay specified distributio after the probability itegral trasformatio is used. I the preset research, the expoetial case is cosidered. Let X1, X2,,X be observatios of a radom sample from a populatio distributio with support set [0,1]. Suppose X(1), X(2),,X() are the correspodig order statistics. The hypotheses will be: H0: The populatio distributio is uiform distributio o [0,1]. H1: The populatio distributio is ot uiform distributio o [0,1]. The test statistic is defied as ( 1) ( X ( i ) X i( 1) ) i1 1 G( X1, X 2,..., X ). (1) Here X(0)=0 ad X(+1)=1. It ca be show that the value of G(X1,X2,,X) is always 3

11 1 betwee 0 ad 1. It ca also be show that E( X ( i) X ( i1) ) for i=1,2,,+1. 1 Whe H0 is true, the value of G(X1,X2,,X) should be small. O the other had, if the value of G(X1,X2,,X) is too large, it could be a idicatio that H0 should be rejected. For ay give 0<α<1, defie Gα such that P(G(X1,X2,,X)< Gα)=α. The H0 will be rejected at α level of sigificace if G(X1,X2,,X)>G1-α. To use this G statistic to test whether the uderlyig distributio is a expoetial distributio, the well-kow probability itegral trasformatio eeds to be used. Let F(x) be the CDF of expoetial distributio, the 0, x 0 Fx ( ) x 1 e, x 0. (2) Let Y F( X ) 1 e X. The Y has a U[0,1] distributio o [0,1]. The G statistic is origially proposed for testig whether the data are from a uiform distributio. Usig the above trasformatio, the G test ca ow be used to test whether the populatio distributio is a expoetial distributio. Here the test is valid oly whe the parameter λ is kow. However, the parameter λ i the expoetial distributio is usually ukow. The Lilliefor s method will be itroduced to solve this problem. The basic idea is to estimate the parameter by calculatig the sample mea. For the expoetial distributio, ˆ 1. x Assume X is from a expoetial distributio with parameter λ. The the CDF will be: F( x) 1 e x (x>0). 4

12 Let Y 1 e X. The Y is uiformly distributed o [0,1]. This is because F( y) P( Y y) X P(1 e y) X P( e 1 y) l(1 y) PX ( ) 1e l(1 y) ( ) 1 (1 y) y(0 y1). This is the CDF of the uiform distributio o [0,1]. The above proof shows that the parameter λ has o cotributio to F(y). That meas o matter what value of λ is selected, the distributio after trasformatio is still a uiform distributio. The value of λ ca be arbitrarily selected at the begiig of the statistical simulatio. The selectio of the iitial value of λ will ot chage the distributio of the test statistic defied i (1). 5

13 CHAPTER II METHODOLOGY I the curret research, the power of two goodess-of-fit tests, G test ad Kolmogorov-Smirov test, will be compared whe they are used to test whether the uderlyig distributio is a expoetial distributio. The followig are doe i the preset research: 1) Fid the critical values of the two test statistics for differet sample sizes; 2) Various alterative distributios are used to compare the power of these two tests. The power of the G test is compared to the power of Kolmogorov- Smirov test i this study. The details related to power will be metioed i Chapter III. 2.1 Fidig Critical Values G Test The followig are the steps for fidig critical values: (a) Geerate a pseudo radom sample u1, u2,,u from the uiform distributio o [0,1]; l(1 u ) (b) Choose a value of λ arbitrarily, say λ=1. Calculate i xi ( i 1,2,..., ) ; (c) Compute the sample mea x 1. The the estimate of λ is ˆ ; x (d) Calculate ˆ x i y 1 e ( i 1,2,..., ) ; i (e) Sort y1, y2,,y to fid the correspodig order statistic y(1), y(2),,y(), ad defie y(0)=0, y(+1)=1; (f) Calculate G(y1,y2,,y) usig equatio (1); (g) Repeat (a)-(f) k times (k=10,000,000); 6

14 (h) Sort all the values of G ad fid the 90 th, 95 th, 99 th, 99.5 th ad 99.9 th quatiles. For give 0<α<1, the critical values of the G test are listed i Table 1. The decisio rule will be to reject the ull hypothesis at α level of sigificace if the test statistic is greater tha G1-α Kolmogorov-Smirov Test Let X1, X2,,X be observatios of a radom sample from a populatio distributio with a distributio fuctio F(x), ad F * (x) be the correspodig empirical distributio fuctio. The the Kolmogorov-Smirov test statistic is: D F x F x D D * sup ( ) ( ) max(, ) x * where D * sup[ F ( x) F( x)] ad D sup[ F( x) F ( x)]. x Let X(1), X(2),,X() be the correspodig order statistic. Defie X(0), X( 1). i The F * ( x) for X ( i) x X ( i 1) (i=0,1,,). x D i max sup F( x) 0 i X ( i) x X ( i1) i max if F( x) 0 i X ( i) xx ( i1) i max F( X() i ) 0 i i max max F( X() i ),0. 1 i (3) D i 1 max max F( X () i ),0. 1 i (4) The procedure for fidig critical values of the Kolmogorov-Smirov test is as follows: (a) Geerate a pseudo radom sample u1, u2,,u from the uiform distributio o 7

15 [0,1]; l(1 u ) (b) Choose a value of λ arbitrarily, say λ=1. Calculate i xi ( i 1,2,..., ) ; (c) Compute the sample mea x 1. The the estimate of λ is ˆ ; x (d) Calculate y ˆ x (i=1,2,,); i i (e) Sort all the y1, y2,,y to fid the correspodig order statistic y(1), y(2),,y(); (f) Calculate D ad D usig equatios (3) ad (4), ad fid out the bigger oe of D ad D which is the test statistic D ; (g) Repeat (a)-(f) k times (k=10,000,000); (h) Sort all the values of D ad fid the 95 th quatiles. For give 0<α<1, the critical values of the K-S test eeded i power study are listed i Table 2. The decisio rule will be to reject the ull hypothesis at α level of sigificace if the test statistic is greater tha D1-α. 2.2 Decisio Rules The hypotheses are: H0: The populatio distributio is a expoetial distributio. H1: The populatio distributio is ot a expoetial distributio G Test The procedure for fidig power of the G test statistic is as follows: (a) Geerate a pseudo radom sample u1, u2,,u from the uiform distributio o [0,1]; 8

16 (b) For a particular alterative distributio, covert u1, u2,,u to a sample x1,x2,x from that alterative distributio (Details will be discussed i Chapter III); (c) Compute the sample mea x. The the estimate of λ is ˆ 1 ; x (d) Calculate ˆ x i y 1 e ( i 1,2,..., ) ; i (e) Sort y1, y2,,y to fid the correspodig order statistic y(1), y(2),,y(), ad defie y(0)=0, y(+1)=1; (f) Calculate G(y1,y2,,y) usig equatio (1). If G(y1,y2,,y) is greater tha the correspodig critical value i Table 1 (here oly α=0.05 is used), reject H0. The record rejectio cout; (g) Repeat (a)-(f) k times (k=1,000,000). Iterate rejectio cout k times; (h) Compute the power which is rejectio cout/k; (i) Repeat procedure (a)-(h) for differet sample sizes Kolmogorov-Smirov Test The followigs are the steps for fidig power of the Kolmogorov-Smirov test statistic: (a) Geerate a pseudo radom sample u1, u2,,u from the uiform distributio o [0,1]; (b) For a particular alterative distributio, covert u1, u2,,u to a sample x1,x2,x from that alterative distributio (Details will be discussed i Chapter III); (c) Compute the sample mea x. The the estimate of λ is (d) Calculate y ˆ x (i=1,2,,); i i ˆ 1 ; x (e) Sort y1, y2,,y to fid the correspodig order statistic y(1), y(2),,y(); 9

17 (f) Calculate D ad D usig equatio (3) ad (4), ad fid out the bigger oe which is the test statistic D. If Table 2, reject H0. The record rejectio cout;; D is greater tha the correspodig critical value i (g) Repeat (a)-(f) k times (k=1,000,000). Iterate rejectio cout k times; (h) Compute the power which is rejectio cout/k; (i) Repeat procedure (a)-(h) for differet sample sizes. Usig the procedures above icludig the procedure i 2.2.1, the power of G test ad K-S test for testig differet alterative distributios with differet sample sizes ca be foud. 10

18 CHAPTER III POWER COMPARISON The power of a hypothesis test is the probability of rejectig the ull hypothesis correctly whe the alterative hypothesis is true. A test with a high power (high rejectio rate) is cosidered to be a good test method. The ideal power of a test is 1, that is, always reject the ull hypothesis whe the ull hypothesis is ot true. I particular, the power of the test statistics discussed i this research is to reject the expoetial hypothesis whe populatio distributio is ot expoetial. Whe the power is closed to 1, the test ca be cosidered to be a good test. I the preset study, the power is estimated usig the rate of rejectio. The same test procedure will be repeated k times to test k sets of pseudo radom samples from specified alterative distributio. The rejectio rate amog these k repetitios will be the power of this goodess-of-fit test. I this research, various alterative distributios such as triagle distributio, V-shaped triagle distributio will be used to coduct Mote Carlo simulatio. The value of k is set to be 1,000,000 to guaratee the accuracy of power compariso. The sample size is also a ifluetial factor to the power. The power will icrease whe becomes large. I this study, =5, 10, 20, 30, 40, 50 will be used. Sigificace level α will be set as Selected Alterative Distributios Triagle Alterative Distributio The probability desity fuctio of the triagle distributio is 11

19 2 x,0 xh h 2(1 x) f( x), h x 1 1 h 0, elsewhere; ad the cumulative distributio fuctio is: 0, x x,0 x h h F( x) 2 (1 x) 1, h x 1 1 h 1, x 1. Let U=F(X). Accordig to the probability itegral trasformatio, U has a uiform distributio o [0, 1]. This is because whe 0 X<h, F( u) P( U u) 1 P X u h P X 2 ( ) 2 ( hu) P( X hu ) 1 hu h u u h); ad whe h X<1, F( u) P( U u) 2 (1 X ) P(1 u) 1 h P X u h 2 ((1 ) (1 )(1 ) P(1 X (1 u)(1 h)) P( X 1 (1 u)(1 h)) (1 1 (1 u)(1 h)) 1 1 h 1 (1 u) uh u 1). 2 12

20 Above is the cdf of the uiform distributio o [0, 1]. I this study, a pseudo radom sample from uiform distributio is geerated first. The the iverse fuctio of U X i hu,0u h 1 (1 U)(1 h), h U 1 has a triagle distributio with parameter h (i=1,2,,).sas/iml ca be used to perform the calculatio after applyig the trasformatio. Here h is a costat betwee 0 ad 1. The selected values are h=0.25, 0.5, ad 0.75 i this power study. Alterative distributio 1 Select h=0.25. This is a left-skewed triagle distributio. Figure 1 shows that the power is icreasig alog with the sample size becomes large. The K-S test performs better tha G test i all cases. Whe sample size is large eough (=50), the power curve of these two tests merges together ad the power is very close to 1. Alterative distributio 2 Select h=0.5. This is a symmetric triagle distributio. Comparig to alterative distributio 1, the result is showed similarly i Figure 2. The power icreases with icreases, ad the K-S test still performs better tha the G test. Whe the sample size icreases to 20, the power of K-S test is approximate 1. However, the power of G test approaches to 1 whe =40. Two curves are mergig faster tha the previous case. 13

21 Alterative distributio 3 Select h=0.75. This is a right-skewed triagle distributio. It ca be foud i Figure 3, that the K-S test is more powerful tha the G test whe <30. After reaches 30, these two tests perform almost same V-shaped Triagle Distributio The probability desity fuctio of the V-shaped triagle distributio is: 2x 2,0 x h h 2(1 x) f( x) 2, h x 1 1 h 0, elsewhere; ad the cumulative distributio fuctio is: 0, x x x,0 x h h F( x) h h x 1 h 1, x 1. 2 ( 2 0) 2 ( x h), 1 I a trasformatio is used. Similarly, let U=F(X). Accordig to the probability itegral trasformatio, U has a uiform distributio o [0, 1]. This is because whe 0 X<h, F( u) P( U u) 1 2 P(2 X x u) h P X hx hu P X h h hu X h h hu 2 2 (( ( ))( ( )) 0) Sice 0 X<h, the X h h hu 2 ( ) 0, 14

22 F u P X h h hu 2 ( ) ( ( ) 0) P X h h hu 2 ( ) h h hu h u h hu u u h). Whe h X<1, F( u) P( U u) 2 ( X h) P( h u) 1 h P X hx h u hu 2 ( 2 0) P(( X ( h ( h 1)( h u)))( X ( h ( h 1)( h u))) 0) Sice h X<1, the X ( h ( h 1)( h u) 0, F( u) P( X ( h ( h 1)( h u))) 0) P( X h ( h 1)( h u)) ( h 1)( h u) h 1 h h h u uh u 1). Above is the cdf of the uiform distributio o [0, 1]. I this study, a pseudo radom sample from uiform distributio is geerated. The X i 2 h h hu,0 U h h ( h 1)( h U ), h U 1 has a V-shaped triagle distributio (i=1,2,,). Here h is a costat betwee 0 ad 1. Select h=0.25, 0.5, ad 0.75 i this power study. Alterative distributio 4 Select h=0.25. This is a left-skewed V-shaped triagle distributio. Figure 4 shows that the K-S test performs better tha the G test whe 30. The powers of both tests are similar, ad approach to 1 whe sample size is greater tha

23 Alterative distributio 5 Select h=0.5. This is a symmetric V-shaped triagle distributio. I Figure 5, it ca be easily foud that the G test is better tha the K-S test i all cases. As the sample size icreases, the powers of both tests are icreasig dramatically. The powers approach to 1 whe =50. Alterative distributio 6 Select h=0.75. This is a right-skewed V-shaped triagle distributio. Whe =5, K-S test performs slightly better tha G test. Figure 6 shows that the G test performs much better tha K-S test whe >10. The power of G test icreases faster tha the K-S test does. However, compare to the previous 2 cases, the powers of both of the tests are low. 3.2 Summary of the Results Based o the above power aalysis, it ca be foud that: (a) For all the triagle alterative distributios, icludig h=0.25, 0.5, 0.75, the K-S test performs better tha the G test. (b) For the left-skewed V-shaped triagle alterative distributio, the K-S test is better tha the G test. However, for the symmetric ad right-skewed V-shaped triagle alterative distributio, the G test performs better tha the K-S test iversely, especially for the right-skewed case. (c) For all the left-skewed alterative distributios, the K-S test performs better tha the G test. 16

24 CHAPTER IV CONCLUSION AND DISCUSSION The goodess-of-fit test is widely used whe checkig whether the uderlyig populatio distributio differs from a specified distributio. I this research, expoetial distributio is cosidered as a specific case. The cocept of the goodessof-fit test is to compute the differece betwee observed values ad expected value i the give model. There are various commoly used goodess-of-fit tests such as the Chi-square test, the Kolmogorov-Smirov test, the Cramer-Vo Mises test, ad the Aderso-Darlig test. I additio, there is also a alterative G test statistic was proposed by Che ad Ye (2009). It was proposed for testig uiformity origially. However, the probability itegral trasformatio makes it possible to use this test to test for ay distributio. Power study is the core sectio of this research. The power of G test ad Kolmogorov-Smirov test are compared by usig the Mote Carlo simulatio. Some alterative distributios such as triagle distributio ad V-shaped triagle distributio are used to compare the power of these two tests. The result shows that Kolmogorov- Smirov test performs better tha G test whe the alterative distributio has a triagle distributio. For the left-skewed V-shaped triagle alterative distributio, the K-S test is better tha G test. However, for the symmetric ad right-skewed V-shaped triagle alterative distributio, G test performs better tha K-S test. 17

25 Table 1 Critical Values of G Test Statistic G0.900 G0.950 G0.975 G0.990 G0.995 G

26 Table 2 Critical Values of K-S Test Statistic D Table 3 Power Compariso: Triagle (h=0.25) G-TEST K-S TEST Table 4 Power Compariso: Triagle (h=0.5) G-TEST K-S TEST Table 5 Power Compariso: Triagle (h=0.75) G-TEST K-S TEST

27 Table 6 Power Compariso: V-shaped Triagle (h=0.25) G-TEST K-S TEST Table 7 Power Compariso: V-shaped Triagle (h=0.5) G-TEST K-S TEST Table 8 Power Compariso: V-shaped Triagle (h=0.75) G-TEST K-S TEST

28 Power Power Power Sample Size G Test K-S Test Figure 1 Alterative Distributio 1: Triagle (h=0.25) Sample Size G Test K-S Test Figure 2 Alterative Distributio 2: Triagle (h=0.5) Sample Size G Test K-S Test Figure 3 Alterative Distributio 3: Triagle (h=0.75) 21

29 Power Power Power Sample Size G Test K-S Test Figure 4 Alterative Distributio 4: V-shaped Triagle (h=0.25) Sample Size G Test K-S Test Figure 5 Alterative Distributio 5: V-shaped Triagle (h=0.5) Sample Size G Test K-S Test Figure 6 Alterative Distributio 6: V-shaped Triagle (h=0.75) 22

30 REFERENCES Che, Z., Ye, C. (2009), A Alterative Test for Uiformity, Iteratioal Joural of Reliability, Quality ad Safety Egieerig, 16: Diae L. Evas, Joh H. Drew & Lawrece M. Leemis (2008) The Distributio of the Kolmogorov-Smirov, Cramer-vo Mises, ad Aderso-Darlig Test Statistics for Expoetial Populatios with Estimated Parameters, Commuicatios i Statistics- Simulatio ad Computatio, 37:7, Hubert W. Lilliefors (1969), O the Kolmogorov-Smirov Test for the Expoetial Distributio with Mea Ukow, Joural of the America Statistics Associatio, 64: Taylor B. Arold, Joh W. Emerso (2011), Noparametric Goodess-of-fit Tests for Discrete Null Distributios, The R Joural, 3/2: Vijay K. Rohatgi, A. K. Md. Ehsaes (2001), A Itroductio to Probability ad Statistics: W. J. Coover (1971), Practical Noparametric Statistics: