A note on vector-valued goodness-of-fit tests

Transcription

1 A note on vector-valued goodness-of-fit tests Vassilly Voinov and Natalie Pya Kazakhstan Institute of Management, Economics and Strategic Research Almaty, Kazakhstan ( Abstract. Vector-valued goodness-of-fit tests based on a single sample with possibly dependent components are considered. Sometimes, e.g., when combining both correlated and uncorrelated non-parametric or parametric tests of approximately the same power, vector-valued tests can provide a gain in power. Several examples of such a gain are presented. Keywords: Vector-valued tests, Power of goodness-of-fit tests. 1 Introduction There are several possible ways to combine test statistics to obtain, say, a more powerful test, or a test sensitive to a specific alternative, or a test for checking the consensus of a set of tests, etc. Combined statistics can be both dependent and independent. Many results concerning the combination of independent test statistics based on the probability integral transformation are known. E.g., [Van Zwet and Oosterhoff, 1967], [Wilk and Shapiro, 1968], [Littell and Folks, 1971], [Koziol and Perlman, 1978], [Marden, 1982], [Rice, 1990], [Mathew et al, 1993]). Combining independent tests in linear models was considered by [Zhou and Mathews, 1993]. [Brown, 1975] considered the same problem if combined tests are not jointly independent. He proposed a method for combining non-independent one-sided tests about a location parameter. To combine information from several sources the above cited authors did not use vector-valued statistics, which can also be useful for obtaining, e.g., more powerful tests. [Zhakharov et al, 1969] proposed a sequential m-dimensional chi-squared vector-valued test X 2 n = (X 2 n 1, X 2 n 2,..., X 2 n m ) T based on m embedded into each other subsamples of a sample of size n such that n 1 < n 2 < < n m n, where Xn 2 i, i = 1,..., m, are standard Pearson s sums. A null hypothesis H 0 is accepted if one of the events A k, k = 1,..., m, occurs, where } A k = {X 2 n1 > x 1,α,..., X 2 nk 1 > x k 1,α, X 2 nk x k,α and x i,α, i = 1,..., n k, are critical values. If for all k = 1,..., m X 2 n k > x k,α, then H 0 is rejected.

2 2 Voinov and Pya [Mason and Schuenemeyer, 1983] proposed a vector-valued test statistic (ω 1 L n,1, ω 2 L n,2, K n, ω 3 U n,1, ω 4 U n,2 ) T, where K n is the Kolmogorov-Smirnov test, L n,1, L n,2, U n,1, U n,2 are Rényitype tests, ω 1,..., ω 4 being nonnegative weights, and 0 < c < is a constant depending on the level of significance α. A null hypothesis is rejected if max{ω 1 L n,1, ω 2 L n,2, K n, ω 3 U n,1, ω 4 U n,2 } > c. Note that all components of the proposed statistic are based on the same single sample. [Voinov and Grebenyk, 1989] used a two-dimensional vector-valued test V n = (K n, R n ) T, where K n was the Kolmogorov-Smirnov and R n - signed rank statistics correspondingly. In spite of correlation between K n and R n the test based on a rejection region, which was the intersection of corresponding rejection regions of components of V n, permitted to recognize a pattern on an image with the signal/noise ratio less than one. In this note we present several examples of combining correlated and uncorrelated non-parametric and parametric tests based on the same single sample. Throughout the paper vectors are boldfaced. 2 Combining two independent tests Consider the following artificial example of testing simple null hypothesis about a probability distribution against a simple alternative: H 0 : P {X x} = F (x; θ), H a : P {X x} = G(x; θ). Let Y 2 1n and Y 2 2n be two independent statistics such that: lim P {Y 1n 2 y H 0 } = lim P {Y 2n 2 y H 0 } = P {χ 2 4 y}, n n lim P {Y 1n 2 y H a } = lim P {Y 2n 2 y H a } = P {χ 2 6 y}, n n where χ 2 k means a central chi-squared distributed random variable with k degrees of freedom. For brevity in the sequel we shall omit the sign of limit. Consider a vector-valued test U n = (Y1n, 2 Y2n) 2 T with the rejection region S 1 = (Y1n 2 > y 1 ) (Y2n 2 > y 1 ), where y 1 = 4.73 is the critical value. Since Y1n 2 and Y2n 2 are independent, then the probability to fall into S 1 under the hull hypothesis or the level of significance of the test will be (see Figure 1): P {(Y 2 1n > y 1 ) (Y 2 2n > y 1 )} = P {Y 2 1n > y 1 } P {Y 2 2n > y 1 } = = 0.1.

3 vector-valued goodness-of-fit tests 3 Fig. 1. Probability density functions of Y 2 in, i = 1, 2, under the null and the alternative hypotheses. In this case the power of the vector-valued test under consideration is P {U n S 1 H a } = = At the same time the power of each component of U n for the same level of significance α = 0.1 is P {Y 2 1n > y 2 H a } = P {Y 2 2n > y 2 H a } = 0.255, where y 2 = We see that that the power of components is less than the power of U n. Consider the vector-valued test U n = (Y1n, 2 Y2n) 2 T with the rejection region S 2 = (Y1n 2 > y 3 ) (Y2n 2 > y 3 ), where y 3 = For such a rejection region the power of U n is P {U n S 2 H a } = 0.28 and again it is more than the power of Y 2 1n and Y 2 2n. This simple example shows that using vector-valued tests may result in power increase as compared with the power of components of a vector-valued test. 3 Combining non-parametric and parametric tests Consider a two-dimensional vector-valued test with correlated components using as an example a test with a modified chi-squared and the non-parametric statistics as its components V n = (Y 2 2 n(θ n ) U 2 n(θ n ), A 2 n) T, (1) where Y 2 2 n(θ n ) is the Hsuan-Robson-Mirvaliev (HRM) statistic ([Hsuan and Robson, 1976], [Mirvaliev, 2001]), U 2 n(θ n ) is the ([Dzhaparidze and Nikulin, 1992]) test, θ n is the method of moments estimator (MME) of a true parameter θ, and A 2 n is the Anderson-Darling test

4 4 Voinov and Pya ([Anderson and Darling, 1954]). The test statistic (Y 2 2 n(θ n ) U 2 n(θ n )) was described, e.g., in [Voinov et al, 2007]. Under some regularity conditions the HRM test Y 2 2 n(θ n ) possesses in the limit χ 2 r 1 distribution under the hull hypothesis ([Mirvaliev, 2001]), r being a number of grouping intervals. For any n- consistent estimator θ n of the parameter θ including the MME θ n the Dzhaparidze-Nikulin test U 2 n(θ n ) follows in the limit χ 2 r s 1 distribution ([Dzhaparidze and Nikulin, 1992], [Mirvaliev, 2001]), where s is the number of unknown parameters. The modified chi-squared test (Y 2 2 n(θ n ) U 2 n(θ n )) is distributed in the limit as χ 2 s. It is shown ([Voinov et al, 2007]) that for equiprobable intervals of grouping data the latter test can be significantly more powerful than Y 2 2 n(θ n ). Suppose that one needs to test a composite null hypothesis about the logistic probability distribution against alternative normal distribution. To compare the power of (1) with that for the scalar tests Y 2 2 n(θ n ), U 2 n(θ n ), Y 2 2 n(θ n ) U 2 n(θ n ), and the non-parametric test A 2 n we used the Monte Carlo simulation. Since theoretical and simulated critical values corresponding to a given level of significance α do not differ within statistical errors, in this study we used (except for the A 2 n) the theoretical ones. The rejection region for V n was the intersection of rejection regions of the components of V n. Since on a plain one can select a rejection region of a given level α by infinitely many ways, we tried several possible variants to choose one, which gives the highest power. The results of simulation for random samples of size n = 200 and α = 0.05 are shown in Figure 2. All modified chi-squared tests considered were calculated for r equiprobable fixed intervals (number of runs was 10,000). Fig. 2. Powers of V n, Y 2 2 n(θ n), U 2 n(θ n), Y 2 2 n(θ n) U 2 n(θ n), and A 2 n for the normal alternative as functions of the number of cells r.

5 vector-valued goodness-of-fit tests 5 From Figure 2 one can see that despite of the correlation between components, the vector-valued statistic for testing the compound hypothesis about the logistic null distribution against the alternative normal distribution possesses not too much but, nevertheless, higher power than the scalar tests based on components under consideration. 4 Combining two non-parametric tests In this section we consider a combination of two correlated non-parametric goodness-of-fit tests that are based on empirical distribution function: the Kolmogorov-Smirnov test K n and the Cramer-Von Mises test W 2 n R n = (K n, W 2 n) T. (2) Consider a problem of testing the simple null hypothesis about the logistic probability distribution with mean zero and variance one against the triangular alternative distribution with the same mean and variance. In this case the power of both K n and Wn 2 implemented separately is approximately the same. To compare power of vector-valued test (2) with the powers of its components two types of rejection regions were used: union and intersection of rejection regions of vector components. Because of the mathematical intractability of some aspects of this study the computer simulation was used. Table 1 shows the results of power simulation for two different rejection regions of the same significance level taken as a union of the rejection regions of the components of (2) (n = 200, α = 0.05, number of runs was 10,000): S 1U = (K n > ) (W 2 n > ), S 2U = (K n > ) (W 2 n > ). S 1U S 2U K n W 2 n R n ±0.004 Table 1. Powers of K n, Wn, 2 and R n against triangular alternative probability distribution for the rejection region of R n taken as a union. The statistical error shown corresponds to one standard deviation From Table 1 we see that in the above considered case the vector-valued test (2) may have higher power than the scalar tests based on the components of the vector R n. It is evident also that the power of (2) definitely depends on the structure of a rejection region.

6 6 Voinov and Pya Consider the following rejection regions of the vector-valued test (2): S 1I = (K n > ) (W 2 n > ), S 2I = (K n > ) (W 2 n > ). The results of the Monte Carlo experiment for random samples of size n = 200 for this case are presented in Table 2. S 1I S 2I K n ±0.004 W 2 n R n Table 2. Powers of K n, Wn, 2 and R n against triangular alternative probability distribution for the rejection region of R n taken as an intersection. The statistical error shown corresponds to one standard deviation From this table we again see that the power of vector-valued test may be both higher and lower than that of scalar components depending on the structure of a rejection region. We would like to note also that that power have been considered only for very simple rejection regions (intersection and union). Evidently that much more complicated regions can be constructed. From this it follows that we only posed the problem, which needs further investigation. 5 Conclusion Several examples of vector-valued goodness-of-fit tests have been considered. The results of the investigation show that when combining both correlated and uncorrelated non-parametric or parametric tests of approximately the same power, vector-valued tests can provide a gain in power as compared with the power of components of a vector-valued statistic. Examples considered show that the power of vector-valued goodness-of-fit tests depends on the structure of a rejection region, correlation between components of a test, and, quite possibly, on the dimensionality of a vector. All these problems need further thorough both theoretical and experimental investigation. References [Anderson and Darling, 1954]T.W. Anderson, D.A. Darling. A test of goodness of fit. JASA, 49: , [Brown, 1975]M.B. Brown. A method for combining non-independent, one-sided tests of significance. Biometrics, 31: , 1975.

7 vector-valued goodness-of-fit tests 7 [Dzhaparidze and Nikulin, 1992]K.O. Dzhaparidze, and M.S. Nikulin. On evaluation of statistics of chi-squared type tests. In Problems of the Theory of Probability Distributions, St. Petersburg: Nauka, 12:59 90, [Hsuan and Robson, 1976]A. Hsuan, D.S. Robson. The χ 2 goodness-of-fit tests with moment type estimators. Commun.Statist. Theory and Methods, A5: , [Koziol and Perlman, 1978]J.A. Koziol and M.D. Perlman. Combining independent chi-squared tests. JASA, 73: , [Littell and Folks, 1971]R.C. Littell and J.L. Folks. Asymptotic optimality of Fisher s method of combining independent tests. JASA, 66: , [Marden, 1982]J.I. Marden. Combining independent noncentral chi-squared or F tests. The Annals of Statistics, 10: , [Mason and Schuenemeyer, 1983]D.M.Mason and J.H. Schuenemeyer. A modified Kolmogorov-Smirnov test sensitive to tail alternatives. The Annals of Statistics, 11: , [Mathew et al, 1993]T. Mathew, B.K. Sinha, L. Zhou. Some statistical procedures for combining independent tests. JASA, 88: , [Mirvaliev, 2001]M. Mirvaliev. An investigation of generalized chi-squared type statistics. Doctoral thesis, Academy of Science of the Republic of Uzbekistan. Tashkent, [Rice, 1990]W.R. Rice. A consensus combined P-value test and the family-wide significance of component tests. Biometrics, 46: , [Van Zwet and Oosterhoff, 1967]W.R. van Zwet and J. Oosterhoff. On the combination of independent test statistics. Annals of Math. Statist., 38: , [Voinov and Grebenyk, 1989]V.G. Voinov and A.P. Grebenyk. Combining dependent criteria and statistical filtration of patterns. MMPR-IV (Math. methods of pattern recognition), Abstracts, Part 3, Sec. 2, Riga [Voinov et al, 2007]V. Voinov, R. Alloyarova, N. Pya. Recent achievements in modified chi-squared testing. In F. Vonta, M. Nikulin, N. Limnios, and C. Huber, editors, Statistical Models and Methods for Biomedical and Technical Systems, Birkhäuser, Boston, Ch.18: [Wilk and Shapiro, 1968]M.B. Wilk and S.S. Shapiro. The joint assessment of normality of several independent samples. Technometrics, 10: , [Zhakharov et al, 1969]V.K. Zhakharov, O.V. Sarmanov, B.A. Sevastyanov. A sequential chi-squared test. Sbornik Mathematics, 79(121): , [Zhou and Mathews, 1993]L. Zhou and T. Mathews. Combining independent tests in linear models. JASA, 88: , 1993.