Non-parametric confidence intervals for shift effects based on paired ranks

Transcription

1 Journal of Statistical Computation and Simulation Vol. 76, No. 9, September 2006, Non-parametric confidence intervals for shift effects based on paired ranks ULLRICH MUNZEL* Viatris GmbH & Co. KG, Weismüllerstr. 45, Frankfurt, Germany (Received 5 August 2004; in final form 2 February 2005) Non-parametric approaches to derive confidence intervals for the shift effect in paired samples are discussed. These approaches are based on the sign statistic, Wilcoxon s signed ranks (WSR) and paired ranks. While the intra-individual differences of the observations are ranked for the WSR approach, the observations are first ranked and the differences are considered afterwards for the paired rank approach. Confidence intervals by using paired ranks are derived by iterating the possible shift effects. In this context, an asymptotic as well as an exact approach is proposed. In simulation studies, the performance of all approaches are compared. Finally, an example from a clinical study is analysed. For all approaches the results are almost equal under hypothesis and under alternative. The differences between the asymptotic and the exact version of the paired rank test are negligible. In the case of normal distributions, the approaches based on paired ranks and signed ranks lose only little efficiency while a huge gain in efficiency can be expected in other situations. This is especially true for the paired rank approach, which is more efficient than the approach based on the sign statistic in all investigated situations and more efficient than the signed rank approach in very most situations. Keywords: Paired samples; Sign test; Wilcoxon s signed rank test; Paired rank test; Exact test 1. Introduction Paired data occur frequently in practice, for example in clinical trials when different topical treatments are compared in the same subject at different location. The standard method for analysing such data is the paired t-test. This test, however, is based on the assumption of normally distributed treatment differences and the resulting confidence intervals may become very large for other distributions. Therefore, non-parametric approaches in a location model may be used to shorten the length of the confidence interval. Consider independent pairs of continuous observations (X k1,x k2 ), where k = 1,...,n denotes the independent replication. For the sake of simplicity, we will denote the replications as subjects in the following. Moreover, we assume a location model (X k1,x k2 µ) (X k2 µ, X k1 ) (1) with an unknown shift parameter µ (interchangeability, [1]). * ullrich.munzel@viatris.de Journal of Statistical Computation and Simulation ISSN print/issn online 2006 Taylor & Francis DOI: /

2 766 U. Munzel Non-parametric standard approaches to derive confidence intervals for µ are based on the sign statistic and the Wilcoxon s signed rank (WSR) statistic [2]. The sign statistic is based on the signs of the differences only and it is well known that this loss of information implies a loss in efficiency. A possible approach to improve the power of the sign test is to rank either the differences which lead to the WSR statistic or to rank the observations first and to compute the intra-individual differences afterwards (paired ranks). The latter approach was proposed first by Conover and Iman [3] who discussed a rank transform statistic of the paired t-test (paired rank test). The asymptotic and exact distribution of paired rank statistics were investigated by Munzel [4] and Munzel and Brunner [5], respectively. Blair and Higgins [6] demonstrated that neither of the paired rank test and the WSR test uniformly is superior to the other. Therefore, it can be expected that the paired rank approach can be an important alternative to the WSR approach when deriving confidence intervals. Throughout the article we briefly describe how confidence intervals are derived by using the sign statistic (section 2.1) and the WSR statistic (2.2), respectively. In section 2.3, the asymptotic and exact versions of the paired rank approach are discussed. Simulation studies on the coverage and length of 95% confidence intervals are presented in section 3 and all methods are applied to an example from a clinical trial in section Computation of confidence intervals 2.1 Sign statistic Let D k = X 2k X 1k denote the intra-individual difference of the kth subject. The sign statistic is the number of positive differences and can be expressed as T s = c(d k ), where c(x) = 0 or 1 according as x< or >0. It should be noted that T s (µ) = c(d k µ) has a binomial distribution B(0.5,n)with parameters p = 0.5 and n in the model (1). Let D (k) denote the order statistic, i.e. exactly n k differences are larger than D (k).to derive a (1 α) confidence interval for the location parameter µ, we compute indexes l and u with P [D (l) µ D (u) ] 1 α. (2) A suitable approach to derive the lower bound D (l) is based on the identity p [ D (l) µ ] [ ] = P c(d k µ) n l. It follows P [D (l) µ] 1 α/2ifn l is larger than the (1 α/2)-quantile q 1 α/2 of the B(0.5,n) distribution. Consequently, l should be chosen as the largest integer n q 1 α2. Analogously, the upper index u should be chosen as the smallest integer n q α/2. The resulting interval [D (l) ; D (u) ] is a (1 α) confidence interval for µ.

3 Non-parametric confidence intervals Wilcoxon s signed ranks An approach similar to section 2.1 leads to confidence intervals based on the WSR statistic T WSR = c(d k ) R k, where R k denotes the rank of D k among all differences. In the location model (1), the test statistic of the shifted differences T WSR (µ) = c(d k µ) R k, follows the null-distribution of the WSR statistic (the ranks of the differences D k are equal to those of the shifted differences D k µ ). Further, let M ij = D i + D j, 2 i = 1,...,n 1; j = i + 1,...,n; denote the Walsh means. It is well known [2] that T WSR = n 1 i=1 j=i+1 c(m ij ). In analogy to equation (2), indexes l and u have to be identified with P [M (l) µ M (u) ] 1 α, where M (s), s = 1,...,n(n+ 1)/2 is the ordered sequence of the Walsh means. The derivation is straightforward and the resulting indexes are l = w α/2 + 1 and u = n(n + 1)/2 w α/2 [7]. The (α/2) quantile of the null-distribution w α/2 can be computed by means of fast shift algorithms [8, 9]. 2.3 Paired ranks In section 2.2, the WSR statistic is derived by ranking the intra-individual differences of the observations. An alternative approach is to rank the marginal observations first (paired ranks) and to compute the intra-individual differences of the ranks afterwards, i.e. to compute the differences Y k = R k2 R k1, where R ks denotes the rank of X ks among all 2n observations Asymptotic approach. Conover and Iman [3] proposed to rank the original observations and to apply ANOVA methods to these ranks (rank transform method). Although this heuristic approach is not suitable in general [10 14], for the special situation of paired data it leads to a very useful statistic. Today, it is well known [4] that the following statistic provides

4 768 U. Munzel a suitable test for the hypothesis µ = 0 in model (2): T PRA = Ȳ. n s, where Ȳ. denotes the mean of the n differences Y k and s is the empirical standard deviation of these differences, i.e., s 2 = 1 (Y k Ȳ. ) 2. n 1 Under hypothesis µ = 0, T PRA has asymptotically a standard normal distribution. For small sample sizes, the distribution is well approximated by a t n 1 distribution [4]. Let T PRA (δ) denote the corresponding test statistics derived from the ranks of the shifted observations (X k1,x k2 δ).ast PRA (δ) is monotonic in δ,a(1 α) confidence interval can be derived by iterating δ over all n 2 possible rank differences D ij = R i2 R j1. The lower bound is the maximal δ {D ij i, j = 1,...,n} with T PRA (δ) t n 1;1 α/2. Analogously, the upper bound is the minimal δ {D ij i, j = 1,...,n} with T PRA (δ) t n 1; α/ Exact approach. As the approaches for the sign test and the WSR test are exact, only the asymptotic treatment of paired ranks is described in section However, Munzel and Brunner [5] proposed an exact version based on the statistic 2 Y k. It should be noted that factor 2 is used only to treat the situation with and without ties, simultaneously. For continuous distributions and assuming that the absolute rank differences R k2 R k1 are known, it is sufficient to consider the number of positive differences T PRX = c(y k ). In the model (1), each difference Y k can be either positive or negative. This leads to 2 n possible permutations, which are equally distributed under the hypothesis of interchangeability. Munzel and Brunner [5] propose a recursion formula and a fast shift algorithm to determine the distribution and quantiles. Using these techniques, a (1 α) confidence interval can be derived in analogy to section This means that the statistic T PRX (δ) is computed from the ranks of the shifted observations (X k1,x k2 δ) and δ is iterated over the rank differences D ij = R i2 R j1. The lower bound is the maximal δ {D ij i, j = 1,...,n} with T PRX (δ) v 1 α/2, where v 1 α/2 denotes the (1 α/2) quantile of the exact null-distribution. Analogously, the upper bound is the minimal δ {D ij i, j = 1,...,n} with T PRA (δ)<v α/2. 3. Simulation Studies 3.1 Design The four approaches to derive confidence intervals for shift effects discussed in section 2, were compared by means of simulation studies. The coverages of 95% confidence intervals

5 Non-parametric confidence intervals 769 were assessed, as well as the mean lengths of these intervals as an indicator of the efficiency. Throughout these studies, also the paired t-test was considered as benchmark. Simulated were the following three models. (a) Normal distribution: X kj = µ j + B k + ε kj, where k = 1,...,n denotes the subject; j = 1, 2 the component; µ j = 0,µ according as j = 1, 2; B k N(0, 1/4) are independent block effects; ε kj N(0, 1) are independent error terms. (b) Log-normal distribution: X kj = µ j + exp(b k ) + exp(ε kj ). (c) Squares: X kj = µ j + (B k + ε kj ) 2. The normal model is used to assess the efficiency in comparison to the optimal t-test in this set-up, i.e. to determine the possible loss of using non-parametric methods. The log-normal model was chosen to represent skew distributions. Therefore, while the block effects vanish in both models when computing intra-individual differences, the block effects remain in the square model which was used in addition. Two situations were considered when choosing the shift effect µ. To analyse confidence intervals under hypothesis, simulations were performed with µ = 0, and for a considerable alternative µ = 1 was used. Sample sizes were n = 7, 10, and 20 and 10,000 simulations were performed for each situation by using a SAS-IML program. 3.2 Results The results of the simulation studies are displayed in table 1 (relative frequency of coverage in %), table 2 (mean lengths of simulated confidence intervals) and figure 1, where the mean lengths of the confidence intervals are presented in relation to the mean length of the confidence intervals calculated for the paired t-test. First, the results clearly indicate that a coverage of at least 95% is guaranteed for all considered approaches (table 1). Moreover, for all approaches the results are almost equal under hypothesis and under alternative (tables 1 and 2). Therefore, only the results under alternative are shown in figure 1. The Table 1. Percent coverage of shift effect µ by simulated confidence intervals. Hypothesis (µ = 0) Alternative (µ = 1) PT PT Distribution n Exact Asym. WSR ST t-test Exact Asym. WSR ST t-test Normal Log-normal Square PT, paired rank test; ST, sign test.

6 770 U. Munzel Table 2. Mean lengths of simulated confidence intervals. Hypothesis (µ = 0) Alternative (µ = 1) PT PT Distribution n Exact Asym. WSR ST t-test Exact Asym. WSR ST t-test Normal Log-normal Square PT, paired rank test; ST, sign test. % Normal Log-normal Squares Sample size n paired ranks (asymptotic); Wilcoxon signed ranks; sign statistic Figure 1. Mean lengths of 95%-confidence intervals of nonparametric tests in percent of the length of the corresponding confidence interval of the paired t-test. differences between the asymptotic and the exact version of the paired rank test are negligible. Consequently, both tests are not distinguished in the following and only the results for the asymptotic version are displayed in figure 1. For normal distributions (see Model (a) in section 3.1), the paired rank approach and the WSR approach lead to only slightly larger confidence intervals than t-test, while a substantial loss of efficiency is observed for the sign test (figure 1 and table 2). For log-normal distributions (see Model (b) in section 3.1), the confidence interval derived from paired ranks is slightly smaller than those derived from the WSR statistic. Both are substantially smaller than the confidence intervals derived from the sign statistic. The t-statistic approach lies somewhere in-between the WSR and the sign test. For larger sample sizes (n = 20) the t-statistic approach is worst.

7 Non-parametric confidence intervals 771 For square distributions (see Model (c) in section 3.1), the confidence intervals based on paired ranks test are considerably smaller than all others. Second best are the WSR or sign approaches depending on the sample size and worst is the t-statistic approach, especially for larger sample sizes. 4. Example Patients suffering from epilepsy who participated in a randomised, double-blind clinical study were asked to join an open-label extension to assess the long-term efficacy of the treatment. The primary evaluation criterion was the monthly rate of partial seizures as assessed at the end of the double-blind part and the end of the open-label extension. Descriptive statistics of the intra-individual changes are displayed in table 3. The statistics indicate a very skewed distribution which is confirmed by the normal quantile quantile plot in figure 2. The skewness is explained by two outliers which are typical and occur frequently when conducting epilepsy studies. To assess the quantitative treatment effect throughout the open-label extension, the nonparametric confidence intervals (see section 2) and the confidence interval from the paired t-statistic are computed. The results are displayed in table 4. The confidence interval of the paired rank approach is considerably shorter than those derived from the sign and the WSR statistic, respectively. The confidence interval derived from the t-statistic is very large. Table 3. Descriptive statistics of intra-individual of monthly rates of partial seizures. n Mean Standard deviation Median Minimum Maximum Skewness Percentiles 20 0,01 0,05 0,25 0,50 0,75 0,90 0, Observed Values Theoretic Quantiles Figure 2. Normal quantile-quantile plot of the intra-individual changes of monthly rates of partial seizures.

8 772 U. Munzel Table 4. Several 95% confidence intervals for treatment effect in open-label extension. 95% confidence interval Approach Lower bound Upper bound Length Paired ranks (asymptotic) WSR Sign test Paired t-test Conclusions Non-parametric approaches to derive confidence intervals for shift effects can be very useful when the normal distribution of the intra-individual differences are doubted. Moreover, in the case of normal distributions the approaches based on paired ranks and signed ranks lose only little efficiency, whereas a huge gain in efficiency can be expected in other situations. This is especially true for the paired rank approach, which is more efficient than the approach based on the sign statistic in all investigated situations and more efficient than the signed rank approach in most situations. There is no evidence that either the signed rank approach or the approach based on the sign statistic are uniformly better than the other. When considering the paired ranks, it is sufficient to use the asymptotic approach which can be implemented more easily than the exact approach, and takes considerably less computational time. References [1] Randles, R.H. andwolfe, D.A., 1979, Introduction to the Theory of Nonparametric Statistics (NewYork, London, Sidney, Toronto: Wiley). [2] Hettmansperger, T.P., 1984, Statistical Inference Based on Ranks (Malabar: Krieger). [3] Conover, W.J. and Iman, R.L., 1981, Rank transformations as a bridge between parametric and nonparametric statistics (with discussion). The American Statistician, 35, [4] Munzel, U., 1999, Nonparametric methods for paired samples. Statistica Neerlandica, 53, [5] Munzel, U. and Brunner, E., 2002, An exact paired rank test. Biometrical Journal, 44, [6] Blair, R.C. and Higgins, J.J., 1985, A comparison of the power of the paired samples rank transform statistic to that of Wilcoxon s signed ranks statistic. Journal of Educational Statistics, 10, [7] Büning, H. and Trenkler, G., 1978, Nichtparametrische statistische Methoden (Berlin: de Gruyter). [8] Streitberg, B. and Roehmel, J., 1986, Exact distribution for permutation and rank tests: an introduction to some recently published algorithms. Statistical Software Newsletter, 12, [9] Zimmermann, H., 1985, Exact calculation of permutational distributions for two dependent samples I. Biometrical Journal, 27, [10] Brunner, E. and Neumann, N., 1986, Rank tests in 2 2 designs. Statistica Neerlandica, 40, [11] Blair, R.C., Sawilowski, S.S. and Higgins, J.J., 1987, Limitations of the rank transform statistic in tests for interaction. Communications in Statistics, Series B, 16, [12] Akritas, M.G., 1990, The rank transform method in some two-factor designs. Journal of the American Statistical Association, 85, [13] Akritas, M.G., 1991, Limitations on the rank transform procedure: a study of repeated measures designs, part I. Journal of the American Statistical Association, 86, [14] Akritas, M.G., 1993, Limitations on the rank transform procedure: a study of repeated measures designs, part II. Statistics and Probability Letters, 17,