A comparison study of the nonparametric tests based on the empirical distributions

통계연구 (2015), 제 20 권제 3 호, 1-12 A comparison study of the nonparametric tests based on the empirical distributions Hyo-Il Park 1) Abstract In this study, we propose a nonparametric test based on the empirical distribution and its quantile function and compare its performance with the well-known tests for the two-sample problem. For this, first of all, we review some existing nonparametric tests and then propose a new one. We consider to apply the permutation principle to obtain the null distribution. Then we show an example and compare the performance by obtaining empirical powers through a simulation study. Finally, we discuss briefly the re-sampling methods and comment about importance of the continuity assumption for the underlying distribution for the nonprametric test. Key words : Nonparametric test, permutation principle, quantile function, two-sample problem 1. Introduction When we compare the efficiency between two treatments, we may consider to apply the two-sample -test under the normality assumption or a nonparametric test by only assuming continuity of the underlying distribution under some specified model. There have been proposed several nonparametric tests for the two-sample problem based on the linear rank statistics under the location and scale models. Randles and Wolfe (1979) extensively studied and well summarized these results with the asymptotic relative efficiency under the location translation model. For example, one may apply the Wilcoxon rank sum test (cf. Wilcoxon, 1945) and the median test (Mood, 1950) for the logistic and double exponential distributions, respectively. As another famous and widely used model in the survival analysis, one may consider the proportional hazards model (cf. Cox, 1972). Under this model, one may apply the log-rank test (cf. Cox and Oakes, 1983) for the two-sample case as a nonparametric procedure. On the other hand, when it may be difficult to assume any suitable model, there have been proposed several test procedures up to now. One is the so-called 1) Professor, Dept. of Statistics, Chongju University, Chongju, Choongbook 360-764, Korea. E-mail: hipark@cju.ac.kr

2 Hyo-Il Park versatile test (Fleming and Harrington, 1991) which applies several nonparametric statistics together. Also Hong and Park (2010) and Park (2011) considered applying several quantile statistics for the various situations including the one-sided alternative. Also one may consider to take a multi-aspect test approach (Marozzi, 2004) which may split the null hypothesis into several aspects, take a suitable test for each aspect and then combine them with a chosen combining function to obtain an overall -value with the application of the nonparametric combination of dependent tests (cf. Pesarin, 2001). Another well-known approach is the simultaneous test (cf. Lepage, 1973) which applies both testing procedures for the location and scale parameters. This approach has been applied extensively and modified by many statisticians (cf. Duran et al., 1976, Murakami, 2007 and Neuhäuser et al., 2011). Also Park and Kim (2012) proposed the nonparametric simultaneous tests with the Mood test (1954) for the scale parameter and compared efficiency with Lepage's ones (1973). Park and Han (2013) applied this approach to the one-sample problem by applying the likelihood ratio principle with the normality assumption for the underlying distribution. However historically the Kolmogorov-Smirnov or Cramér-von Mises tests are famous for this situation which do not assume any specific underlying distribution. We note that the testing statistics have been constructed via the empirical distribution functions. Also one may use the two-sample Anderson-Darling test (cf. Pesarin, 2001) whose statistic is a modified form of the one-sample case. Also Baumgartner et al. (1998) studied extensively the relation of another type of Anderson-Darling test with the simultaneous one. In this study, we will consider to propose a nonparametric test without any assumption for the underlying distribution except the continuity using the empirical distribution function and its inverse, i.e., quantile function. This paper will be organized in the following order. In section 2, first of all, we review the well-known nonparametric tests which can be applied for this case and then propose a new nonparametric test. We apply the permutation principle (cf. Good, 2000) to obtain the null distribution of the proposed test. Then we illustrate our procedure with an example and compare the efficiency of the new test with the reviewed tests by obtaining empirical powers through a simulation study in section 3. Finally, we discuss briefly the re-sampling methods and comment about importance of the continuity assumption for the underlying distribution for the nonprametric test. 2. Nonparametric tests Suppose that we have two independent random samples, and

A comparison study of the nonparametric tests based on the empirical distributions 3 from populations with continuous distribution functions and, respectively. Then it is of our main interest to test against without any further assumption or model about and. For this case, the Kolmogorov-Smirnov (), Cramer-von Mises () and Anderson-Darling () tests are famous among the nonparametric tests. Before we propose a new nonparametric test, we review their statistics in the following to clarify our discussion for the construction of our new statistic. Let is an indicator function. Then the two empirical distributions functions and can be defined as and for any real number. In the following, in order to simplify our arguments, we define the order statistics from the combined sample, where. (1) statistic The statistic can be defined as sup. We note that the statistic is a function of the difference between two empirical functions. (2) statistic The statistic can be defined as. (3) statistic The statistic can be defined as,

4 Hyo-Il Park where is the empirical function for the combined sample. We note that all the reviewed statistics consist of difference between two empirical distribution functions. Also we note that one may consider to use quantile functions which are the inverse images of the distribution function. The Mood median test (Mood, 1950) is a famous example in this approach. Also Hong and Park (2010) proposed quantile tests based on the multiple use of quantiles. In order to discuss this subject more precisely, let be a th sample quantile from the combined sample, for any given,. Then a quantile test statistic can be defined as for any given,. Then it is well-known that the mean and variance of under can be obtained as and, where and means that the largest integer part of. Then we may propose a supremum type of nonparametric statistic, for testing against as sup sup. We note that can be considered a difference between two empirical distribution functions since is the empirical probability from the combined sample. Then the testing rule would be to reject in favor of for large values of. Then it is necessary to obtain the distribution of or a limiting one under to complete the test. However it is well-known that these types of statistics do not have the normal distribution as its limiting one. For this reason, we consider applying the permutation principle with the Monte-Carlo approach in this study.

A comparison study of the nonparametric tests based on the empirical distributions 5 3. An example, simulation results and concluding remark The following data set in <Table 3.1> is the perceived degree of job satisfaction measured by a proper psychological index consisting of a sum of a finite subresponses, each related to a specific sub-aspect among individuals classified as extroverted ( ) and introverted ( ) in Pesarin (2001). The respective sample sizes are and and so. Then it is of interest to test whether there exists any difference between the two groups, and. Since it is difficult to assign any specific underlying distribution or assume any model between two populations, it would be appropriate to consider using a test which is based on the difference between two empirical distribution functions. <Table 3.1> Data for job-satisfaction 66 57 81 62 61 60 73 59 80 55 67 70 64 58 45 43 37 56 44 42 We have applied four different tests based on,, and and obtained the respective -values based on 10,000 iterations using the permutation principle with the Monte-Carlo method which may be called the conditional Monte-Carlo method (cf. Pesarin 2001) for all cases. All the computations have been carried out by SAS/IML with PC version and the results are summarized in <Table 3.2>. One may conclude that the two groups, and may differ in the job-satisfaction except the test for the significance level 0.05. <Table 3.2> -values for 4 different tests Tests -value 0.0162 0.0493 0.0521 0.0349 From now, we compare the performance among tests considered in the previous section by obtaining empirical powers through a simulation study under the location translation model. Therefore the two distribution functions, and, have the following relation: for any real number, there is a real number such that, (3.1)

6 Hyo-Il Park where is the location translation parameter. Then by varying the value of, we carry out the simulation study. The distributions considered for this simulation study are normal, exponential, double exponential, Weibull when with unit variance and Cauchy with 0 and 1 for the location and scale parameters, respectively for the generation of pseudo-random numbers. The value of varies from 0.0 to 1.0 with increment by 0.2 under (3.1). For the Weibull distribution, we allowed the range of variation for the location parameter () from 0 to 0.2 with increment by 0.04 since the Weibull distribution is extremely skewed to 0. Therefore we may observe that a small shift of the value of can make big jump of the corresponding empirical power. We note that holds when for all cases. We considered three cases for the sample sizes such as i), ii) and iii),. The nominal significance level is 0.05 for all cases. The respective empirical powers under (3.1) are summarized in Tables 3-8. All the results in the tables are based on the 10,000 simulations with the Monte-Carlo method and within a simulation we applied the permutation principle by 5,000 iterations also with the Monte-Carlo method to estimate the distribution. In general, the and tests show high performance while the and tests yield low efficiency. However we note that the test achieves high empirical powers for the exponential and Weibull distribution. Therefore we may conclude that the test may be efficient when the underlying distributions are skewed. <Table 3.3> Normal distribution Test 0.0 0.2 0.4 0.6 0.8 1.0 0.0493 0.0739 0.1242 0.2096 0.3504 0.4797 0.0619 0.0767 0.1440 0.2675 0.4192 0.6001 0.0617 0.0734 0.1513 0.2712 0.4299 0.6104 0.0514 0.0722 0.1297 0.2237 0.3652 0.5329 0.0246 0.0547 0.1342 0.2993 0.5057 0.7117 0.0461 0.0761 0.2030 0.4142 0.6493 0.8198 0.0483 0.0788 0.2054 0.4203 0.6517 0.8243 0.0449 0.0753 0.1733 0.3486 0.5647 0.7432 0.0398 0.0856 0.2182 0.4273 0.6636 0.8474 0.0357 0.0868 0.2617 0.4964 0.7505 0.9111 0.0362 0.0833 0.2590 0.4984 0.7563 0.9184 0.0444 0.0839 0.2123 0.4236 0.6565 0.8414

A comparison study of the nonparametric tests based on the empirical distributions 7 <Table 3.4> Cauchy distribution Test 0.0 0.2 0.4 0.6 0.8 1.0 0.0493 0.0624 0.0911 0.1303 0.1907 0.2576 0.0619 0.0699 0.0874 0.1336 0.1952 0.2653 0.0617 0.0667 0.0864 0.1356 0.1941 0.2618 0.0514 0.0586 0.0873 0.1299 0.1763 0.2424 0.0246 0.0431 0.0763 0.1489 0.2513 0.3611 0.0461 0.0596 0.1055 0.1762 0.3040 0.4074 0.0483 0.0606 0.1041 0.1763 0.2949 0.3967 0.0449 0.0542 0.0911 0.1596 0.2594 0.3667 0.0398 0.0523 0.1266 0.2418 0.3684 0.5222 0.0357 0.0599 0.1236 0.2311 0.3689 0.5135 0.0362 0.0621 0.1200 0.2213 0.3538 0.4966 0.0444 0.0563 0.1062 0.2049 0.3117 0.4464 <Table 3.5> Exponential distribution Test 0.0 0.2 0.4 0.6 0.8 1.0 0.0493 0.0971 0.2579 0.5094 0.7463 0.8808 0.0619 0.1047 0.2836 0.5120 0.7222 0.8808 0.0617 0.1084 0.2966 0.5173 0.7264 0.8861 0.0514 0.0941 0.2643 0.5247 0.7568 0.9052 0.0246 0.0915 0.3904 0.7268 0.9224 0.9799 0.0461 0.1443 0.4569 0.7687 0.9268 0.9771 0.0483 0.1593 0.4868 0.7961 0.9439 0.9826 0.0449 0.1855 0.6397 0.9115 0.9877 0.9976 0.0398 0.1534 0.5442 0.8577 0.9689 0.9971 0.0357 0.1989 0.5705 0.8369 0.9648 0.9911 0.0362 0.2243 0.6128 0.8686 0.9715 0.9947 0.0444 0.3530 0.8162 0.9799 0.9953 1.0000 <Table 3.6> Double Exponential distribution Test 0.0 0.2 0.4 0.6 0.8 1.0 0.0493 0.0933 0.1954 0.3472 0.5318 0.7146 0.0619 0.0942 0.2143 0.3888 0.5953 0.7553 0.0617 0.0947 0.2136 0.3859 0.5931 0.7530 0.0514 0.0922 0.1858 0.3454 0.5342 0.7074 0.0246 0.0773 0.2614 0.5130 0.7706 0.8992 0.0461 0.1155 0.3271 0.5943 0.8209 0.9314 0.0483 0.1136 0.3233 0.5918 0.8162 0.9261 0.0449 0.0964 0.2801 0.5204 0.7639 0.9051 0.0398 0.1261 0.3807 0.6842 0.9044 0.9745 0.0357 0.1352 0.4066 0.7108 0.9133 0.9801 0.0362 0.1332 0.3984 0.6949 0.9080 0.9752 0.0444 0.1123 0.3359 0.6222 0.8631 0.9630

8 Hyo-Il Park <Table 3.7> Logistic distribution Test 0.0 0.2 0.4 0.6 0.8 1.0 0.0493 0.0777 0.1451 0.2460 0.3964 0.5718 0.0619 0.0826 0.1624 0.3018 0.4744 0.6615 0.0617 0.0814 0.1643 0.3029 0.4735 0.6582 0.0514 0.0734 0.1458 0.2561 0.4183 0.5917 0.0246 0.0576 0.1728 0.3626 0.5845 0.7992 0.0461 0.0856 0.2333 0.4619 0.7132 0.8623 0.0483 0.0872 0.2344 0.4676 0.7145 0.8619 0.0449 0.0781 0.2058 0.4006 0.6250 0.8093 0.0398 0.0933 0.2656 0.4949 0.7454 0.9217 0.0357 0.0955 0.2993 0.5584 0.8091 0.9528 0.0362 0.0924 0.2963 0.5599 0.8107 0.9513 0.0444 0.0925 0.2557 0.4820 0.7226 0.8979 <Table 3.8> Weibull distribution for Test 0.0 0.04 0.08 0.12 0.16 0.2 0.0493 0.1507 0.3659 0.5627 0.7056 0.8041 0.0619 0.1247 0.2638 0.4109 0.5443 0.6512 0.0617 0.1287 0.2706 0.4181 0.5513 0.6567 0.0514 0.1585 0.3881 0.5879 0.7313 0.8232 0.0246 0.2585 0.6042 0.8092 0.9089 0.9545 0.0461 0.2213 0.4735 0.6698 0.8064 0.8856 0.0483 0.2534 0.5261 0.7237 0.8490 0.9189 0.0449 0.5860 0.8723 0.9598 0.9852 0.9955 0.0398 0.4176 0.7596 0.9082 0.9617 0.9835 0.0357 0.2861 0.5717 0.7641 0.8720 0.9302 0.0362 0.3516 0.6514 0.8276 0.9140 0.9551 0.0444 0.7876 0.9561 0.9874 0.9968 0.9989 Up to now, we have applied the permutation principle to obtain the null distribution of the test statistics as a re-sampling method. As another re-sampling method for the hypothesis test, one may also consider to use the bootstrap method (cf. Efron, 1979). While the permutation method re-samples without replacement, the bootstrap method does with replacement. The two methods are asymptotically equivalent for some testing situations and test statistics (cf. Romano, 1989) but often they yield quite different results (cf. Good, 2000). When nuisance parameters are involved, variables may not be exchangeable. Practical example may arise when comparing variances when the means unknown and testing for interaction in an experimental design when main effects are present (cf. Good, 2000). Then one can not apply the permutation principle but do the bootstrap method to those situations. When both methods for a testing problem can be applicable, the

A comparison study of the nonparametric tests based on the empirical distributions 9 permutation test is preferable since it is exact and conditional on a set of sufficient statistics (cf. Pesarin, 2001). Now we discuss the assumption of the continuity for the underlying distributions in the nonparametric testing problems. The continuity of the underlying distribution defines the ranks unambiguously and provides easily the null distribution for any rank-based statistics. If the underlying distribution is discrete, the null distribution for the rank-based statistics should depend on the underlying distribution and so the test procedure can not be nonparametric. However in reality one may have very often tied value or values even though the underlying distribution is continuous because of various reasons such as a consequence of rounding-off or inaccuracy of measurement device. One may overcome this difficulty by using the mid-rank scheme which averages the possible ranks if they are observed and recorded more precisely. For more discussion for the treatment of tied observations, you may refer to Hajek and Sidak (1967). Finally we note that all the test statistics considered in this paper have the form of the difference between two empirical distributions or quantile functions. Also one may show that the statistic can be expressed as the difference of two empirical distributions asymptotically. (2015 년 3 월 5 일접수, 2015 년 4 월 13 일수정, 2015 년 4 월 29 일채택 ) Acknowledgement The author wishes to express his appreciation to the three anonymous referees for pointing out errors and useful suggestions to improve this paper. This work was supported by the research grant of Cheongju University in 2014-2015.

10 Hyo-Il Park References Baumgartner, W., Weiß, P. and Schindler, H. (1998). A Nonparametric Test for the General Two-Sample Problem, Biometrics, 54, 1129-1135. Cox, D. R. (1972). Regression Models and Life-Tables (with discussions), Journal of Royal Statistical Society, B, 34, 187-220. Cox, D. R. and Oakes, D. (1983). Analysis of Survival Data, Chapman and Hall, London. Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife, Annals of Statistics, 7, 1-26. Fleming, T. R. and Harrington, D. P. (1991). Counting Processes and Survival Analysis, Wiley, New York. Good, P. (2000). Permutation Tests-A Practical Guide to Resampling Methods for Testing Hypotheses, Springer-Verlag, New York. Hong, S. M. and Park, H. I. (2010). A Nonparametric Test Procedure for Two Distributions Equality using the Right Censored Data, Journal of the Korean Data Analysis Society, 12, 2963-2972. Lepage, Y. (1973). A Table for a Combined Wilcoxon Ansari-Bradley Statistic, Biometrika, 60, 113-116. Marozzi, M. (2004). A Bi-Aspect Nonparametric Test for the Multi-Sample Location Problem, Computational Statistics and Data Analysis, 46, 81-92. Mood, A. M. (1950). Introduction to the Theory of Statistics, McGraw-Hill, New York. Mood, A. M. (1954). On the Asymptotic Efficiency of Certain Nonparametric Two- Sample Tests, Annals of Mathematical Statistics, 25, 514-522. Murakami, H. (2007). Lepage Type Statistic based on the Modified Baumgartner Statistic, Computational Statistics and Data Analysis, 51, 5061-5067. Neuhäuser, M., Leuchs, A.-K. and Ball, D. (2011). A new Location-Scale Test based on a Combination of the Ideas of Levene and Lepage, Biometrical Journal, 53, 525-534. Park, H. I. (2011). Nonparametric Tests for the One-Sided Alternatives, Journal of the Korean Data Analysis Society, 13, 63-70. Park, H. I. and Kim, J. S. (2012). A Study on the Bi-Aspect Procedure with Location and Scale Parameters, Journal of the Korean Official Statistics, 17, 19-26. Park, H. I. and Han, K. J. (2013). A Simultaneous Test for the Mean and Variance based on the Likelihood Ratio Principle, Journal of the Korean Data Analysis Society, 15, 1733-1742. Pesarin, F. (2001). Multivariate Permutation Tests, Wiley, New York.

A comparison study of the nonparametric tests based on the empirical distributions 11 Randles, R. H. and Wolfe, D. A. (1979). Introduction to the Theory of Nonparametric Statistics, Wiley, New York. Romano, J. P. (1989). Bootstrap and Randomization Tests of Some Nonparametric Hypotheses, Annals of Statistics, 17, 141-159. Wilcoxon, F. (1945). Individual Comparisons by Ranking Methods, Biometrics, 1, 80-83. Short title : Comparison Study of Nonparametric Tests

12 Hyo-Il Park 경험적분포에바탕을둔비모수검정절차에대한비교연구 박효일 1) 요약 이표본문제에서경험적분포와이에따른분위수함수에의거한비모수검정절차를제시하며기존의유명한검정들과검정능력을비교한다. 이를위하여우선기존의검정절차를되짚어보고새로운검정을제시한다. 자료를이용하여제시한검정절차를예시하며모의실험을통하여경험적검정을구하여검정능력을비교한다. 마지막으로재표본방법을검토하며비모수검정절차에대하여모집단의분포에대한연속성가정의중요성을간단히언급한다. 주요용어 : 비모수검정, 순열원칙, 분위수함수, 이표본문제 1) 청주대학교통계학과교수. E-mail: hipark@cju.ac.kr