A Comparison of the Power of the Discrete Kolmogorov-Smirnov and Chi- Square Goodnessof-Fit

Transcription

1 Bond University From the SelectedWorks of Mike Steele 22 A Comparison of the Power of the Discrete Kolmogorov-Smirnov and Chi- Square Goodnessof-Fit Tests. Mike Steele, Bond University Neil Smart, Bond University Cameron Hurst, Queensland University of Technology Janet Chaseling, Griffith University Available at:

2 A comparison of the power of the discrete Kolmogorov-Smirnov and Chi-Square goodness-of-fit tests Michael Steele, Neil Smart 2, Cameron Hurst 3 and Janet Chaseling 4 School of Business, Bond University, Australia. 2 Faculty of Health Science and Medicine, Bond University, Australia 3 School of Public Health, Queensland University of Technology, Australia. 4 Griffith School of Environment, Griffith University, Australia This is the original author version of the paper. The definitive version is: Chaseling, J., Steele, M., Smart, N., & Hurst, C. (22). A comparison of the power of the discrete Kolmorgorov- Smirnov and Chi-Square Goodness of Fit tests. In K. Kumar & A. Chaturvedi (Eds.). Some Recent Developments in Statistical Theory and Applications (pp. 4-9). Boca Raton, Florida, U.S.A: Brown Walker Press Abstract Goodness-of-fit tests for discrete data are widely used in the health and economic disciplines but only a limited number of published power studies exist which make recommendations on relative power of these test statistics. This paper compares the power of Pearson s Chi-Square test with the discrete Kolmogorov-Smirnov test for a uniform null distribution against a number of predefined alternative distributions and makes recommendations on which is the more powerful discrete goodness-of-fit test.. Introduction Goodness-of-fit tests are widely used in economic and health research areas however limited power studies are available in the literature (Choulakian et al. 994; Pettitt and Stephens 977; Steele and Chaseling 26; Ampadu et al. 29). Although Chi-Square type tests are widely used on discrete data for economic and health projects this paper shows situations where the power of other goodnessof-fit test statistics are relatively higher. In this paper the power of Pearson s Chi-Square test statistic (Pearson 9) is compared with that of the discrete Kolmogorov-Smirnov test statistic defined by Pettitt and Stephens (977). The power for each test statistic defined in Section 2 is approximated for a uniform null against various alternative distributions defined in Section 3. The power study results are given in Section 4 and in Section 5 recommendations are made on which of the two test statistics the researcher should use in relation to power. 2. The test statistics defined The two test statistics in this power study are the discrete Kolmogorov-Smirnov (Pettitt and Stephens 977) and Pearson s Chi-Square (Pearson 9) and are defined in equations 2. and 2.2. max i k Z (2.) i k 2 i O E 2 i E i i where k is the number of cells, O i and E i are the observed and expected frequencies for cell i, and Z i is the cumulative sum of the differences between the observed and expected frequencies up to and including cell i. (2.2)

3 3. Techniques to determine power To obtain consistent results a uniform null distribution over ten cells is used for each alternative distribution defined in Table below. The powers of each test statistic is calculated for sample sizes of, 2, 3, 5, and 2 (or, 2, 3, 5, and 2 observations per cell under the uniform null distribution). The power is estimated from simulated random samples by generating a null and alternative distribution of each test statistic. The simulated null distribution of each test statistic is discrete so an exact significance level of 5% is generally not possible. To account for this the power is linearly interpolated about the 5% level to enable consistent estimates of the power for each test statistic. Table. Alternative distributions used in the power studies Description Cell Probability (2 Decimal Places) Decreasing Step Triangular Platykurtic Leptokurtic Bimodal Results of the power study 4. Decreasing alternative For the smaller sample sizes Figure shows that the test statistic has greater power than the χ 2 test statistic. For larger sample sizes of at least five observations per cell this difference is shown to be negligible Step type alternative Fig.. Power of and χ 2 for a uniform null and decreasing alternative In Figure 2 the results are similar to the trend type alternative in Section 4. with the power of being substantially greater than that of the χ 2 test statistic for sample sizes of up to per cell under the null distribution.

4 Power Fig. 2. Power of and χ 2 for a uniform null and step type alternative 4.3 Triangular alternative distribution The power of is shown in Figure 3 to be less than that of the χ 2 test statistic for most of the sample sizes. The exception being the larger sample sizes where the power of both test statistics is approximately equal. Clearly the χ 2 test statistic is the more powerful for this particular alternative distribution. 4.4 Platykurtic alternative Fig. 3. Power of and χ 2 for a uniform null and triangular alternative The power of the test statistic is shown in Figure 4 to be very low for all sample sizes. Although the power of the χ 2 test statistic is shown to be much greater than the test statistic for larger sample sizes its power for smaller sample sizes is however quite poor. Although the χ 2 test statistic is shown to have relatively higher power than its use for the smaller sample sizes is still with very low power.

5 Power Fig. 4. Power of and χ 2 for a uniform null and platykurtic alternative 4.5 Leptokurtic alternative The power of is shown in Figure 5 to be relatively less than the χ 2 test statistic for smaller sample sizes however for sample sizes of at least five per cell the powers of both test statistics are shown to be approximately the same and high for this alternative distribution. Overall the χ 2 test statistic could be used with higher power than the test statistic for the leptokurtic alternative distribution, particularly for smaller sample sizes Fig. 5. Power of and χ 2 for a uniform null and leptokurtic alternative 4.6 Bimodal alternative The test statistic is shown in Figure 6 to be very low regardless of the sample size and the power of the χ 2 test statistic is also shown to be very low for sample sizes less than ten per cell under the null distribution. Clearly unless the sample size is quite large that none of these two test statistics can be used with high power for a bimodal alternative distribution.

6 Fig. 6. Power of and χ 2 for a uniform null and bimodal alternative 5 Conclusions and Recommendations The power study results show that neither of the test statistics can realistically recommended to the applied econometrician as having higher power for all situations. These results and the summary in Table 2 can at least give the applied econometrician some guide to the choice of alternative goodnessof-fit test statistics with respect to power. Table 2. General summary of the power of the two test statistics Alternative Ranking of the Powers Distribution Decreasing > χ 2 Step > χ 2 Triangular χ 2 > Platykurtic Χ 2 > Leptokurtic χ 2 > Bimodal χ 2 > References Ampadu C, Wang D, Steele M (29) Simulated power of some discrete goodness-of-fit test statistics for testing the null hypothesis of a zig-zag distribution. Far East Journal of Theoretical Statistics 28:57-7 Choulakian V, Lockhart RA, Stephens MA (994) Cramér-von Mises test statistics for discrete distributions. The Canadian Journal of Statistics 22:25-37 Pearson K (9) On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine Series Five 5:57-75 Pettitt AN, Stephens MA (977) The Kolmogorov-Smirnov goodness-of-fit statistic with discrete and grouped data. Technometrics 9:25-2 Steele M, Chaseling J (26) Powers of discrete goodness-of-fit test statistics for a uniform null against a selection of alternative distributions. Communications in Statistics Simulation and Computation 35:67-75.