Non-Parametric Weighted Tests for Change in Distribution Function

Transcription

1 American Journal of Mathematics Statistics 03, 3(3: DOI: 0.593/j.ajms Non-Parametric Weighted Tests for Change in Distribution Function Abd-Elnaser S. Abd-Rabou *, Ahmed M. Gad Statistics Department, Faculty of Economics Political Science, Cairo University, Cairo, Egypt Abstract For a given data set it may be required to discover if a change has been occurred. This is can be conducted using change-point analysis. Let X, X,,X n be independent romvariables with respective continuous distribution functions F, F,, F n such that F i (0=0 for all i.we consider the problem of testing the null hypothesis that F = F = = F n against the alternative of r-changes in the distribution functions of this sequence at unknown times <[nτ <[nτ <. <[nτ r <n, where[ y is the integer part of y. We study the asymptotic theory of change-point processes which are defined in terms of the empirical process. We propose study new weighted non-parametric change-point test statistics for a possible change in distribution function of a data set. Keywords Change Point Problem, EmpiricalProcesses, Kiefer Process, Limit Theorems, Monte Carlo Simu lation. Introduction In many applications it is necessary to discover if a data set comes from a single distribution or there is a change in the distribution function. The change-point inference is an effective powerful statistical tool for determining if when a change in a set of data has occurred. Let X, X,,X n, be independent rom variables with continuous distribution functions (DF F, F,, F n, respectively such that F i (0=0for all i. We are interested in testing the null hypothesis of no change HH oo : FF = FF = = FF = FF, ( where F is unknown against the alternative of at most r-changes (AMRC, GG, ii [ττ GG, [ττ < ii < [ττ (rr.. HH : FFii = (.... GG rr +, [ττ rr < ii, where rr is specified, the distribution functions GG ii, ii =,, rr + the change-point positions ττ ii, ii =,, rr are unknown GG ii GG ii +, ii =,,rr. Note that[ττ denotes the integer part of ττ. The aim of this article is to introduce weighted non-parametric tests for distributional change in a data set. The asymptotic distributions of these test statistics are derived. These tests also can be applied to the mean change ina data set. * Corresponding author: abdrfeps@feps.edu.eg (Abd-ElnasrAbd-Rabou Published online at Copyright 03 Scientific & Academic Publishing. All Rights Reserved The paper is organized as follows. In Section we will consider the above multiple change-point problem in the case of at most two change-points (AMTC, i.e. r=. In Section 3 we generalize the AMTC results to the case of rr. The proposed new change-point test statistics are presented in Section 4. Also, the asymptotic distributions of the proposed test statistics are derived in Section 4. In Section 5, we propose new test statistics for the case of at most one change point. We study the applicability of the proposed tests through a Monte Carlo study in section 6.. The Case of at Most Two Change Points (AMTC In this section we treat the case of at most -changes (AMTC, i.e. testing HH 0 of ( against the alternative of (, when r =. Many authors have discussed the change-point problem, testing (detection, estimation using both Bayesian non-bayesian approaches. Most of the work done in the change-point analysis is concerned with the case of at most one change (AMOC. Csörgő Horváth[ gave avery excellent extensive treatment review for the related work. Forthe AMTC, recent works are done by[,3. They propose weighted CUSUM tests for the multiple changes in the variance of a sequence of independent rom variables. By using the general form of U-statistic, they studied some CUSUM test statistics for the AMTC in variance. Hawkins[4 developed a dynamic program to test for a mu ltip le change point in the parameters of the general exponential family using repeated maximum likelihood algorithm. His algorithm, involves the application of the at most one change-point ma ximu m like lihood detector for every

2 58 Abd-Elnaser S. Abd-Rabou et al.: Non-Parametric Weighted Tests for Change in Distribution Function segmented group of data available. Syzsykowicz[5studied weighted approximations for various versions of the empirical processes under the null continuous alternatives of the AMOC. She approximated these processes by appropriate Gaussian processes. Let the smoothed two-parameter empirical process be given by (ss, tt = [(+tt {II[FF(XX ii ii = ss ss}, 0 ss, tt. (3 Let tt = (tt, tt, where 0 tt tt <. For 0 tt tt <, 0 ss,, define ββ ss, tt = [( + tt αα (ss, tt + [( + tt {αα (ss, tt αα (ss, }, (4 ss, tt = {( tt KK(ss, tt + tt KK(ss, tt tt KK(ss, }/, (5 where {KK(ss, yy; 0 ss, 0 yy < } is a Kiefer process, i.e., a mean zero two-parameters Gaussian process with EE(KK(ss,yy KK(ss,yy = (ss ss ss ss (yy yy. (6 The following result follows from Theorem 8.. of[5. Theorem. Assume that HH 0 of Eq. ( holds true. Then, there exists a Kiefer process K(.,. such that as, ββ (ss, tt (ss, tt = oo pp (, 0 ss 0<tt tt < whereββ (.,. is as in Eq. (4 ππ (.,. is as in Eq. (5. Let (.,. be as in Eq. (3 ββ (ss, tt be as in Eq. (4. Define AA (ss, yy = (ss, yy [( +yy (ss,. (7 Note that ββ ss, tt = [ ( + tt AA (ss, tt + [( + tt AA (ss, tt. Let Q be the class of positive functions on (0,, which are non-decreasing in a neighbourhood of zero non-increasing in the neighbourhood of one. A function q defined on (0, is called positive if inf δδ tt δδ qq(tt > 0, δδ 0,. Let qq, qq QQ, the AMTC weighted-process, ββ ww (.,. is defined as ββ ww(ss, tt = [ (+ tt AA (ss,tt + qq [ (+ tt AA (ss,tt. (8 qq (tt Let K(.,. be a Kiefer process. Define Γ (ss, tt = KK (ss, tttt(ss,, 0 ss, tt, Γ ww ss,tt = ( tt Γ ( ss,tt qq + tt Γ (ss,tt. (9 qq (tt Note that Γ (ss, tt Γ(ss, tt = KK(ss, tt tttt(ss, BB (ssbb (tt, 0 ss, tt, (0 where BB (. BB (. are two independent Brownian bridge. Now we give the main theorem of this section. Theorem. Under HH oo of (, there exists a Kiefer process K(.,. such that as,. if for i=, tt( tt log log (tt( tt / lim = 0, tt 0 qq ii (tt then tt( tt llllll llllll (tt( tt / lim = 0, tt 0 qq ii (tt 0 ss 0 tt,tt <. if for i=, then 0 ss 0 ss ββ ww ss, tt ΓΓ ww ss, tt = oo pp ( tt( tt log log (tt( tt / lim <, tt 0 qq ii (tt tt( tt log log tt( tt lim <, tt qq ii (tt 0 tt,tt < ββ ww ss, tt ΓΓ ww ss, tt = OO pp ( ββ ww ss, tt DD 0 tt,tt < 0 ss Γ ww ss, tt. 0 tt,tt < Proof of Theorem (Sketch First we can easily notice that AA (ss, tt ii αα (ss, tt ii, 0 ss, tt ii, ii =, ( where the right-h side is the two-time parameter empirical process of[5. Second if we put qq ii (tt ii =, ii =, in (6..3 of[5, we get as 0 ttii tt ii [( +tt ii = oo(. ( Using the definition of the processes of Eq. (8 Eq. (9, the statements of Eq. (, Eq. ( Theorem 8.3. of[5, we complete the proof of this theorem. Now, let ββ ww.,. be the process defined in Eq. (8, then by Theorem ( the relations in Eq. (9 Eq. (0, we have ββ ww ss, tt DD Γ ww ss, tt, where ββ ww ss, tt = BB (ss ( tt BB + tt BB (tt, (3 qq qq (tt BB ii (., ii =, are as in Eq. (0. As in Pouliot[3, we may use the one weight function ww tt = qq ( tt +qq, 0 < tt tt <, for the whole process. It is very easy to see that Theorem remains true under the one-weight function ww.. In this case the corresponding limiting process of Eq. (3 becomes Γ ww ss, tt = ww tt BB ( ss BB tt, where BB tt = ( tt BB + tt BB (tt,

3 American Journal of Mathematics Statistics 03, 3(3: BB ii (., ii =, are independent Brownian bridge. It is clear that Γ ww ss, tt is a mean zero Gaussian process with covariance function where CC ss,tt, yy = EE Γ ww ss,tt Γ ww ss, yy = (ss ss ss ss CC BB tt, yy, ww tt ww yy CC BB tt, yy = ( tt [( rr yy + yy yy + tt [( rr (tt yy + yy (tt yy tt yy. 3. The Case of at Most r Change Points (AMRC We consider here the general case of rr. Following the definition of the change-point processes in (6.5 of Pouliot (00, we define the weighted r-change point empirical process its corresponding weighted Gaussian process as follows. Assume that qq jj (. QQ, jj =,, rr, satisfy the two assumptions of part ( of Theorem, we define the weighted r-time parameter empirical process 3/ ([(+ tt mm + [(+tt mm rr mm = qq mm MM ww ss, tt AA (ss, tt mm + ( ttrr ( ttrr[(+ttrrqqrr(ttrr (ss,, (4 where 0 = tt 0 tt tt tt rr tt rr+ = the processes αα (.,. AA (.,. are defined by Eq. (3 Eq. (7 respectively. We also define the weighted r-time parameter limiting Gaussian process as follows; Λ ww ss, tt : = rr + tt mm KK (ss,tt mm mm= + ttrr ttrrttrrqqrr(ttrrkk(ss,, (5 where K(.,. is the Kiefer process of Eq. (6 KK (ss, = KK(ss, tttt(ss,, 0 ss, tt. Now, following the steps of the proof of Theorem, we can state the general weighted- metric approximation for the r-time parameter empirical process of Eq. (4. Theorem 3. Under the null hypothesis HH 0 of (, there exists a Kiefer process K(.,. such that with the sequence of processes MM ww (.,. ww (.,. of (4 (5 respectively, we have as MM ww (ss, tt ΛΛ ww (ss, tt = oo pp (. 0 ss 0<tt tt tt rr < Under the conditions of Theorem 3, we obtain 0 ss 0<tt tt tt rr < MM ww ss, tt DD ΛΛ ww (ss, tt, where ΛΛ ww (ss,. is the process in Eq. (5 when n=. 4. The Proposed AMRC Test Statistics To introduce our proposed multiple change-point test statistics, we need the following integrated processes. Let αα (.,. be the smoothed two-parameters empirical process defined by Eq. (3.Then integrating over 0 ss, we get αα (tt = αα (ss, ttdddd = [ (+tt [(+ tt FF(XX ii = ii 0 (6 define the integrated empirical process difference of Eq. (4 as; AA (tt ii = αα (tt ii [ (+tt ii αα (, ii =,,., rr. (7 The generalized test statistics integrated processes in the case of rr, AMRC, tt = (tt 0, tt,, tt rr, tt rr+, such that 0 = tt 0 < tt tt rr < tt rr+ = are given by MM tt = MM ss, tt dddd 0 rr = 3 ([( + tt mm + [( + tt mm AA mm = + ( [( + tt rr ( [( + tt rr ([( + tt rr αα ( where αα (. is defined by Eq. (6 its corresponding limiting Gaussian process is Λ tt = Λ ss, tt dddd 0 rr = BB (ssdddd + tt mm BB, 0 mm = where Λ(.,. is defined by Eq. (5 B(. is a stard Brownian bridge defined on the same probability space. Next, we define the weighted processes MM ww (. Λ ww (., that are needed to construct the AMRC test statistics. For tt = (tt 0, tt,, tt rr,tt rr +, such that 0 = tt 0 < tt tt rr < tt rr+ =, we define the following weighted processes. ([(+ tt mm + [(+tt mm qq mm MM tt = 3 rr mm = AA + ( [( + tt rr ( [( + tt rr ([(+tt rr Λ ww tt 0 BB (ssdddd qq rr (tt rr αα ( } (8 + tt mm rr mm = BB, (9 qq mm where αα (. AA (. are given by Eq. (6 Eq. (7 respectively qq jj (. QQ, jj =,,, rr are the weight functions of Theorem 3. Theorem 4. Let B(. be a Brownian bridge assume that HH oo of ( holds. Then as MM ww tt DD ΛΛ ww tt, where tt = (tt 0, tt,, tt rr,tt rr +, such that 0 = tt 0 < tt tt rr < tt rr+ = MM ww. ΛΛ ww. are the processes given by (8 (9 respectively. The proof of this theorem can be deducted easily from that of Theorem 3.

4 60 Abd-Elnaser S. Abd-Rabou et al.: Non-Parametric Weighted Tests for Change in Distribution Function Corollary. By the continuous Mapping Theorem for rr, we have TT (rr = tt MM ww (tt DD ΛΛ ww (tt = TT tt TT (rr = MM ww tt = ΛΛ ww tt tt The asymptotic distribution of TT (rr TT (rr are, up to our knowledge, are unknown. For this reason we present the special case; at most one change point test statistics. Then we study the applicability of the proposed teststhrough a Monte Carlo simulation study. 5. The at Most One Change (AMOC Test Statistics First, we present the two-weight function test statistics for the CDF on change-point change. For 0 yy, consider the following weight functions; WW (yy = yy( yy, (0 WW (yy = yy( yy log log. ( yy( yy Let αα (. is the empirical counterpart of the process in Eq. (6, defined by replacing the CDF, F(., by its sample one FF (.. We define the test process {AA (kk; kk }, by AA (kk αα (kk kk αα (, = TT 4 ( kk tt FF (XX ii kk ii = FF (XX ii = ( ii where FF (tt = II(XX ii = ii tt, tt RR, is the sample empirical distribution function. The above test process is the natural cidate in case of testing for a change in the CDF of a sequence of independent rom variables. Let II ( (aa, bb = (, II ( (aa, bb = [, [, where (a, b = ( , , see[3. Now, we p ropose the follo wing AMOC test statistics; TT ( max kk AA (kk, (3 AA TT ( max (kk kk WW ( kk / (4 ( (kk kk II (aa, bb TT 3 ( max kk AA (kk (, kk II (aa, bb, WW kk (5 AA (kk kk II ( WW ( kk (aa, bb / AA (kk kk II ( WW ( kk (aa, bb, / (6 TT 5 ( AA (kk ( II (aa,bb + AA (kk ( II (aa,bb / WW ( kk WW ( kk / / WW ( kk (7 TT 6 ( AA (kk ( II (aa,bb + AA (kk ( II (aa,bb (8 where WW (., WW (. AA (. are given by Eq. (0, Eq. ( Eq. ( respectively. Note that the first four test statistics are CDF change point versions analogues to Pouliot (00. The last two TT 5 ( TT 6 ( are new proposed test statistics. The limiting distributions of the above test statistics are unknown in literature. Thus we conduct a Monte Carlo study to determine the performance of these test statistics. 6. Estimated Critical Values Powers The critical values of the proposed tests in (3-(8 have been evaluated via simulation. Also, the power of the proposed tests have been estimated. These estimation tasks are conducted using three simulation studies. 6.. Simulation The aim of this simu lation is two-folds. First, to estimate the critical values of each test at different sample sizes under different distributions.second, to show that the critical values are stable. The upper 5% critical values for each test of the six tests given in (3-(8 have been obtained via simulation study. A sample of each distribution has been simulated. The sample sizes are fixed at 5, 0, 5, 30, 40, 50, 00, 00, 500, 000. The underlying distributions are the normal distribution, the chi-square distribution, the exponential distribution the uniform distribution. Each test value is evaluated for each sample. This process is replicated 0000 times. Then, each test values are sorted the upper 95% percentile is obtained. The simulation results are displayed in Table Table. The other distributions critical values have a similar behaviour. From these results we can notice that the second test has a higher critical values followed by the third test, for all the four distributions. The fifth test has the lowest critical values across all the four distributions. The critical values of the first, fourth sixth tests are close. Generally, the critical values of all tests starts at a higher (lower levelfor smaller sample size; n = 5, but they converge reasonably as the sample size increase, see the table. This convergence appears from a sample size as large as 00.

5 American Journal of Mathematics Statistics 03, 3(3: Ta ble. The estimated upper 5% critical values of the normal chi-square distributions The normal distribution n T T T3 T4 T5 T The chi-square distribution n T T T3 T4 T5 T Ta ble. The estimated upper 5% critical values of the exponential uniform distributions The exponential distribution n T T T3 T4 T5 T The uniform distribution n T T T3 T4 T5 T

6 6 Abd-Elnaser S. Abd-Rabou et al.: Non-Parametric Weighted Tests for Change in Distribution Function 6.. Simulation The aim of this simulation study is to estimate the power of the tests (3 - (8 assuming there is one change point in the mean. A sample of fixed size is generated fromeach distribution. The sample sizes are fixed at 0, 50, 00 units, to cover small, moderate large sample size. The samples are generated from the normal distribution, the chi-square distribution, the exponential distribution the uniform distribution. The change point positions are fixed at the first tail (5%n of the sample in the middle (50%n of the sample. Different change shifts have been used, name ly = 0.5, =.0 =.5. The replications number is The percentage of times the test statistic exceeds the estimated critical values is reported for each change, test statistic, sample size. The results are displayed in Table 3 Table 4. The simulation results show that the estimated power of each test undereach distribution increase as the change position moves to the middle of the sample. The estimated powers of all tests increase as the change shift increases the sample size increases. From the results we can see that the fourth test has the highest powerfollowed by the sixth test for all distributions in the different settings. The third test has the lowest power in the different setting. The estimated powers of the first the second tests are comparable in the different settings. The powers under the uniform distribution has the highest values, whereas the lowest powers are under the chi-square distribution. This is not surprising because any change in the mean of the uniform rom variable affect the distribution boundaries too. However, the chi-square distribution will change its shape very slowly with such minor location changes. Ta ble 3. The estimated powers of the normal chi-square distributions (change in the mean The normal distribution n Δ position T T T3 T4 T5 T The chi-square distribution n Δ position T T T3 T4 T5 T

7 American Journal of Mathematics Statistics 03, 3(3: Table 4. The estimated powers of the exponential the uniform distributions (change in the mean The exponential distribution n Δ position T T T3 T4 T5 T The uniform distribution n Δ position T T T3 T4 T5 T Simulation 3 The aim of this simulation is to estimate the power of the proposed testassuming that there is change point in the distribution rather than only the distribution mean. A sub-sample is simulated from a given distribution augmented by another sub-sample of another distribution. This means that there is a change in distribution. The change positions are the first 5% point of the sample the middle of the sample. The sample sizes are fixed at n = 0, n = 50 n= 00. Four different distributions have been used; namely the normal distribution, the chi-square distribution, the exponential distribution the uniform distribution. The results in Table 6 show the estimated power of the tests for the normal distribution against the other three distributions (the chi-square distribution, the exponential distribution the uniform distribution. Also, the table shows the estimated power of the tests assuming the chi-square distribution against the exponential the uniform distribution in addition to the exponential distribution against the uniform distribution. From the results we can notice that the fourth test has the highest powerfollowed by the sixth test. Generally, the third test has the lowest estimated powers. The highestpowers are obtained for the chi-square distribution against the rest of the distributions. 7. Discussion Conclusions In this paper we presented new non-parametric weighted type test statisticsfor a change in the cumulative distribution function of a set of data. These proposed test statistics are based on the empirical processes. The asymptotic distributions of these test statistics are unknown intractable to be studied theoretically.

8 64 Abd-Elnaser S. Abd-Rabou et al.: Non-Parametric Weighted Tests for Change in Distribution Function We conducted a simulation study to estimate the critical values powersof the proposed tests in the at most one change point. Our weighted proposed tests have good performance in all settings; different distributions, different Table 5. The estimated powers of a distribution change sample sizes different change positions. The difficulty of tracing the limiting distributions of the proposed weighted test statistics encouragethe search for a simple new weighted test statistics. The normal distribution against the chi-square distribution n position T T T3 T4 T5 T The normal distribution against the exponential distribution n position T T T3 T4 T5 T The normal distribution against the uniform distribution n position T T T3 T4 T5 T The chi-square distribution against the exponential distribution n position T T T3 T4 T5 T The chi-square distribution against the uniform distribution n position T T T3 T4 T5 T The exponential distribution against the uniform distribution n position T T T3 T4 T5 T

9 American Journal of Mathematics Statistics 03, 3(3: REFERENCES [ Csörgő, M. Horváth, L., 979, Limit Theorems in Change-Point Analysis, John Wiley & Sons, New York. [ Orasch, M. R., 999, Multiple change-points with an application to financial modelling, Ph.D. thesis, Carleton University, Ottawa, Canada. [3 Pouliot, W. J., 00, Non-parametric, Non-sequential change-points analysis, Ph.D. thesis, Carleton University, Ottawa, Canada. [4 Hawkins, D. M., 00, Fitting multiple change-point models to data, Computational Statistics & Data Analysis, 37, [5 Syzsykowicz, B., 99, Weak convergence of stochastic processes in weighted metrics their applications to contiguous change-points analysis, Ph.D. thesis, Carleton University, Ottawa, Canada. [6 Aly, E.-E., Abd-Rabou, A. S. Al-Kari, N. M., 003, Tests for multiplechange points under ordered alternatives, Metrika, 57, 09-.