CHANGE POINT PROBLEMS IN THE MODEL OF LOGISTIC REGRESSION 1. By G. Gurevich and A. Vexler. Tecnion-Israel Institute of Technology, Haifa, Israel.

Transcription

1 CHANGE POINT PROBLEMS IN THE MODEL OF LOGISTIC REGRESSION 1 By G. Gurevich and A. Vexler Tecnion-Israel Institute of Technology, Haifa, Israel and The Central Bureau of Statistics, Jerusalem, Israel SUMMARY The paper considers generalized maximum likelihood asymptotic power one tests which aim to detect a change point in logistic regression when the alternative specifies that a change occurred in parameters of the model. A guaranteed nonasymptotic upper bound for the significance level of each of the tests is presented. For cases in which the test supports the conclusion that there was a change point, we propose a maximum likelihood estimator of that point and present results regarding the asymptotic properties of the estimator. An important field of application of this approach is occupational medicine, where for a lot chemical compounds and other agents, so-called threshold limit values or TLVs are specified. We demonstrate applications of the test and the maximum likelihood estimation of the change point using an actual problem that was encountered with real data. 1 AMS 2000 subject classifications. Primary 62F03, 62H15, 62F10; Secondary 62F05, 62L10. Key words and phrases. change point, logistic regression, logit, maximum likelihood estimation, martingale. This research was supported by the Israel Science Foundation and by the Marcy Bogen Chair of Statistics at the Hebrew University of Jerusalem. 1

2 1 Introduction We consider a finite sequence of independent observations Y i, x i, i 1 where x i = [x i1,..., x id ] T, i 1 are fixed d-vector explanatory variables and Y i = [Y i1,..., Y imi ] T, i 1, where Y ij are Bernoulli variates whose exact distribution depends on the predictor x i. We assume that P {Y ij = 1 x i } = 1 + exp x T i β 0 1 I{i < ν} exp x T i β 1 1 I{i ν}, β k = [β k1,..., β kd ] T, k = 0, 1, j = 1,..., m i, i = 1,..., n, β 0 β 1, where I{ } is the indicator function and β 0, β 1, ν are parameters of the model. The model corresponds to a situation where up to an unknown change point ν > 0, the observations satisfy the logistic regression model with parameter β 0. Beyond that if ν n, the parameter changes to β 1. Formally, the problem examined in this study is a change point problem of hypothesis testing, where H 0 : ν / [1, n], versus H 1 : 1 ν n; ν > 0 is unknown. 1.2 One of the main problems considered in the change point literature relates to testing, which focuses on a sequence of previously obtained independent observations in an attempt to determine whether they all share the same distribution. It is usually assumed that if a change in the distribution did occur, then it is unique and the observations after the change all have the same distribution, which differs from the distribution of the observations before the change e.g. Page 1954, Sen and Srivastava 1975, Pettitt 1980, Worsley 1983, Gombay and Horvath 1994, Ferger A number of 2

3 studies have dealt with independent observations which are not identically distributed e.g. Quandt 1960, Brown, Durbin and Evans 1975, Kim and Siegmund 1989, Ulm 1991, Pastor and Guallar Kim and Siegmund 1989 considered likelihood ratio tests which aim to detect a change point in simple linear regression when the alternative specifies that the intercept and the slope change. They obtained approximations for the significance level of the tests based on asymptotic analysis. Ulm 1991 proposed a test for detecting a change in logistic regression parameters based on a quasi one-sided χ 2 1 distribution. Except for the simulation study, the author has not represented rigorous theoretical substantiation of his method. Note that many epidemiological studies are based on a two-segmented logistic regression model, in which the linear term associated with a continuous exposure in standard logistic regression is replaced by a two-segmented polynomial function with an unknown change point. In these studies, the parameters of the model including the change point are often estimated without previous formal testing of the existence of the change point e.g. Pastor and Guallar 1998, Gossl and Kuchenhoff It is reasonable to assume that estimation of a change point without testing its existence may be an erroneous procedure. The main goal of the paper is to present maximum likelihood asymptotic power one tests with bounded significance levels for the problem under consideration. In this way, we obtain non-asymptotic bounds for the significance levels of our tests that do not depend on explanatory variables x. Therefore, no assumptions are made about behaviour of x. We represent Monte Carlo simulations to demonstrate the accuracy of suggested bounds for the significance levels and powers of our tests. Our approach is based on methods developed through a sequential testing of a change in the distribution s parameters of 3

4 independent identically distributed i.i.d observations in the ranges before and after the possible change. In this context, Robbins and Siegmund 1973 constructed a power one class of tests based on estimation of the unknown parameters that appeared after the change point occurred prior to that point the parameters were known. In order to preserve an H 0 -martingale property of the sequence of likelihood ratios, they proposed that the estimator of the parameter used in conjunction with the n-th observation be a function of the first n 1 observations only. Based on this idea they produced an upper bound for the level of significance. This method was used by Yakir, Krieger and Pollak 1999 as well as Lorden and Pollak 2004 in the sequential approach for two-phase linear regression and i.i.d observations. We propose retrospective tests for the non-linear model 1.1, which are also based on a statistic that has a martingale structure. Note that if the application of the test ends with the acceptance of hypothesis H 1 then we propose the maximum likelihood estimator MLE of the change point parameter and present results for its asymptotic properties. The paper is organized as follows. The next sections presents a test for the simple case of the problem when β 0 and β 1 are known, and the MLE of the change point is defined. Section 3 considers a more complicated case, where β 0 is known and β 1 is unknown. Section 4 presents a situation where both β 0 and β 1 are unknown. Section 5 represents the results of Monte Carlo simulations. An example of real data for the application of the test and the MLE is demonstrated in Section 6. 4

5 2 Procedure for both initial and final parameters known We consider the model 1.1 and test the hypothesis 1.2, where β 0, β 1 are known. The analysis is relatively clear, and has the basic ingredients for more general cases. We posit that this simple case can be practically associated with the following examples: a technical tool manual will indicate β 0 and β 1, if a functional change in the tool s smooth work is expected by the manufacturer; if there are large tails of the sample, i.e., numerous observations are made before and after a possible change in the distribution; in many studies of occupational medicine, the distribution function of being ill is known a priori and the dependence on some agents varies before and after some possible TLVs. Let P H0 denote the distribution of the sequence Y 1,..., Y n under H 0 and P ν=k denote the distribution of the sequence Y 1,..., Y n under H 1 with ν = k. Likewise, let E H0 and E ν=k denote expectation under P H0 and P ν=k, respectively. Denote ni=k mi Λ n j=1 P ν=k {Y ij x i } k β 0, β 1 x k,..., x n = ni=k mi j=1 P H0 {Y ij x i } ni=k mi j=1 P ν=i {Y ij x i } = ni=k mi j=1 P H0 {Y ij x i } n m i n = exp x T i β 1 x T i β 0 Y ij 1 + expx T mi i β 0, i=k j=1 i=k 1 + expx T i β where P ν=k {Y ij x i } = P ν=k {Y ij = 1 x i } Y ij 1 P ν=k {Y ij = 1 x i } 1 Y ij, P H0 {Y ij x i } = P H0 {Y ij = 1 x i } Y ij 1 P H0 {Y ij = 1 x i } 1 Y ij. 5

6 We propose the following test: reject H 0 iff max 1 k n Λn k β 0, β 1 x k,..., x n > C. 2.4 Significance level of the test. In 2.4 we proposed a generalized maximum likelihood ratio test for the problem 1.2. It is widely known in change point literature that such tests have high power. Therefore, evaluation of their significance level is a major issue. Most results deal with the significance level of the generalized maximum likelihood ratio tests even in the simplest cases of i.i.d observations before and after the change are asymptotic n e.g. Gombay and Horvath In Proposition 2.1 we obtain a guaranteed non-asymptotic upper bound for the significance level of the proposed test. Proposition 2.1 The significance level α of the test satisfies: { } α = P H0 max 1 k n Λn k β 0, β 1 x k,..., x n > C 1/C. Proof. In Appendix. We have the upper bound that does not involve n and no conditions are considered on explanatory variables x 1,..., x n for the significance level of the test 2.4: α 1/C. Thus, C = 1/α determines a test with a level of significance that does not exceed α. Note that even if we assume that x i c, then we have i.i.d random variates under H 0 and use of the known results of the renewal theory regarding overshooting will lead us to the asymptotic n result for α e.g. Woodroofe 1982: α = Zβ 0, β 1, c/c + o 1/C, where 0 < Zβ 0, β 1, c 1 is some function of β 0, β 1 and c, o 1/C C 0 as C. 6

7 Power of the test. We define for all 1 i n m i P a i E ν ln ν {Y ij x i } 2.5 j=1 P H0 {Y ij x i } x T = m i i β 1 x T expx T i β expx T i β expx T i β 1 + ln i β expx T i β 1 it is clear that if i < ν than a i 0 and if i ν: a i 0. The following proposition presents an expression for the power of the test 2.4. Proposition 2.2 Assume: for some N > 0 and n ν > N: n a i > lnc and n a i lnc 2 n m i x T 2, as n ν. i β 1 β 0 Then for all 0 < δ < 1/2 { } P ν max 1 k n Λn k β 0, β 1 x k,..., x n C n exp a i lnc 2 2 n m i x T i β 1 β 0 2 and lim exp n δ a i lnc 2 n ν n m i x T i β 1 β 0 2 { } 1 P ν max 1 k n Λn k β 0, β 1 x k,..., x n > C = Proof. In Appendix. Proposition 2.2 shows that the probability of type II error is bounded by an exponentially vanishing term. Remark 2.1. Consider the assumption in Proposition 2.2 on a simple example. Let m i = 1 be for all i [ν, n]. Hence, by 2.5 we have a i = E ν ln PH0 {Y i1 x i } P H0 {Y i1 x i } ln E ν = P ν {Y i1 x i } P ν {Y i1 x i } 7

8 Suppose that in addition ɛ 1 > 0, ɛ 2 > 0 exists, such that for all i [ν, n]: a i ɛ 1 and x T i β 1 β 0 ɛ 2. Obviously, if n ν is large enough, then for a fixed C: n a i > n νɛ 1 > lnc and n a i lnc 2 n m i x T i β 1 β 0 2 ɛ2 1n ν 2 2 lnc n a i ɛ 2 2n ν, as n ν. Therefore, according to Proposition 2.2 constants c 1 > 0 and c 2 > 0 are such that for the large N { } P ν max 1 k n Λn k β 0, β 1 x k,..., x n C c 1 exp c 2 n ν. Maximum Likelihood Estimation of a change point. If H 0 is rejected, then, following Borovkov 1998, the following estimator of ν is considered ν n = arg max 1 k n Λn k β 0, β 1 x k,..., x n. 2.8 The following proposition delineates several properties of this estimator. Proposition 2.3 For all t 1 > ν and t 2 < ν n P ν { ν n t 1 } k t 1 exp k 1 a i 2 2 k 1 m i x T i β 1 β and P ν { ν n t 2 } exp k t 2 ν 1 i=k a 2 i 2 ν 1 i=k m i X T i β 1 β If, additionally, we assume that for some N > 0, c > 0, p > 2 and n ν > N: k 1 2 n a i exp k t 1 2 c k 1 m i x T i β 1 β 0 2 t 1 ν, 2.11 p exp k<t 2 ν 1 i=k a i 2 2 ν 1 i=k m i x T i β 1 β c t 2 ν p,

9 then E ν ν n = ν + O1 and the estimator ν n is asymptotically homogeneous see Borovkov 1998 as n ν, where O1 is bounded and a i by 2.5. Proof. In Appendix. Remark 2.2. Consider the conditions 2.11 for the simple example in Remark 2.1. Note that for i [1, ν 1] a i = E H0 ln Pν {Y i1 x i } P ν {Y i1 x i } ln E H0 = 0. P H0 {Y i1 x i } P H0 {Y i1 x i } Suppose additionally that for all i [1, ν 1]: a i ɛ 1 and x T i β 1 β 0 ɛ 2. It is obvious that in this case there are constants c 3 > 0, c 4 > 0, c 5 > 0 and c 6 > 0 so that k 1 2 n a i t > ν : exp k t 2 k 1 m i x T i β 1 β n k=[t] 1 t < ν : [t] k=1 c 3 e c4k ν e c4[t] ν 1 e c 4n ν = c 3, 1 e c 4 ν 1 i=k exp a 2 i k<t 2 ν 1 i=k m i x T i β 1 β 0 2 c 5 e c 6ν k = c 5 e c 6ν [t] e c 6ν 1 e c 6 1 where [t] is an integer value of t. Here, it is clear that the condition 2.11 is fulfilled. 3 Procedure for known initial parameters and unknown final parameters This Section considers the same problem as in Section 2, but β 1 is unknown. This case has been dealt with extensively in the literature e.g. Yakir, Krieger, Pollak 1999 and Lorden, Pollak 2004 and corresponds to a control problem where the baseline in-control 9

10 distribution is known and the post-change out-of-control distribution is not. One application of this problem is found in epidemiological studies where it is often assumed that explanatory variables have no effect on a response prior to a certain unknown change point. We denote Λ n k β0, β x k,..., x n = n m i i=k j=1 expx T 1 i β 0 Y ij expx T i β 0 expxt i β i+1,n Y ij 1 + expx T i β i+1,n, where β i+1,n I {Y r, x r } n r=i+1, 1 i n is any estimator of β1 in the σ- algebra generated by {Y r, x r } n r=i+1 β i+1,n based upon {Y r, x r } n r=i+1, and where β n+1,n = β 0. Therefore, Λ n k Λ n k β 0, β 1 x k,..., x n from 2.3. We propose the following test: reject H 0 iff β0, β x k,..., x n is the estimator of the likelihood ratio max 1 k n Λn k β0, β x k,..., x n > C Significance level of the test. Proposition 3.1 The significance level α of the test satisfies: { α = P H0 max 1 k n Λn k β0, β x } k,..., x n > C 1/C. Proof. In Appendix. Power of the test. The next result gives an expression for the power of the test

11 Proposition 3.2 Assume: for some 0 < ρ < 1, N > 0 and n ν > N: 1 ρ n a i > lnc and 1 ρ n a i lnc 2 n m i x T i β 1 β 0 2, as n ν, where a i > 0 is defined in 2.5. Then { 0 1 P ν max 1 k n Λn k β0, β x } k,..., x n > C exp 1 ρ n a i lnc 2 2 n m i x T i β 1 β τ ν,n, 3.15 where } n a i { n τ ν,n P ν 2m i x T i β i+1,n β 1 ρ Proof. In Appendix. From this proposition, we ascertain that the probability of type II error is bounded by an exponentially vanishing term plus a term τ ν,n that depends on the estimates used. Remark 3.1. In the case that the explanatory variables x are such that n a i + see for example, Remark 2.1. At this rate, if we suppose consistency of the estimator β i+1,n β i+1,n β 1, since n i is arbitrarily large, i ν, it can thus reasonably assumed that n 2m i x T i β i+1,n β 1 / n a i 0. Hence, for 3.15, 3.16 τ ν,n 0, as n ν. Maximum Likelihood Estimation of a change point. If H 0 is rejected, then following Borovkov 1998, the following estimator of ν is considered ν n = arg max n m i 1 k n i=k j=1 expx T 1 i β 0 Y ij expx T β i k,n Y ij 1 + expx T i β expx T β i k,n, 3.17 where β k,n is the MLE of β 1 in the σ-algebra generated by Y k, x k,..., Y n, x n. 11

12 Proposition 3.3 Assume: for some N > 0, 0 < ρ 1 < 1, 0 < ρ 2 < 1, c > 0, t 1 > ν, t 2 < ν, p > 2 and n ν > N: t 1 k n 1 k t 2 P ν where a i by 2.5. { k 1 P ν { ν 1 i=k 2m i x T i β k 1 k,n β 1 ρ 1 } a i } 2m i x T i β ν 1 k,n β 1 ρ 2 a i exp 1 ρ 1 2 t 1 k n exp 1 k t 2 i=k k 1 2 a i 2 k 1 m i x T i β 1 β ρ 2 2 ν 1 i=k a 2 i 2 ν 1 i=k m i x T i β 1 β 0 2 c t 1 ν p, 3.18 c t 2 ν p, 3.19 c t 1 ν p, 3.20 c t 2 ν p, Then E ν ν n = ν + O1 and the estimator ν n is asymptotically homogeneous see Borovkov 1998 as n ν, where O1 is bounded. Proof. In Appendix. Remark 3.2. An analysis of the conditions 3.18, 3.19 may be applied, as in Remark 3.1. The investigation of a condition similar to 3.20 is presented in Remark Procedure for both initial and final parameters unknown This section considers situations in which both β 0 and β 1 are unknown, and proposes a test based on the Bayesian approach. Denote for some 1 L n and L k n Λ n k β x k,..., x n = k 1 m i i=l j=1 f{y ij x i, β 1,i 1 } f{y ij x i, β L,n }

13 n m i i=k j=1 f{y ij x i, β k,i 1 } f{y ij x i, β L,n }, where L may be arbitrarily chosen depending on the dimension of parameter β on the a priori assumption that the first L 1 observations come before the change. Such observations are usually called a learning sample e.g. Yakir The condition probability function f{y x, β} exp x T βy exp x T β, = 1, m=1 β L,n is the MLE of β 0 under the hypothesis H 0 based upon Y L, x L,..., Y n, x n : β L,n arg max β n m r r=l h=1 f{y rh x r, β }, β l 1,l 2 I Y l1, x l1,..., Y l2, x l2, 1 l 1 l 2 n 1 is an estimator of β based upon Y l1, x l1,..., Y l2, x l2 for example, the MLE based upon Y l1, x l1,..., Y l2, x l2 observations and for l 1 > l 2 : β l 1,l 2 = β 1,l2, β 1,0 = [0,..., 0] T. Therefore, Λ n k β x k,..., x n is the estimator of the likelihood ratio Λ n k β 0, β 1 x k,..., x n from 2.3. We propose the following test: reject H 0 iff n β x k,..., x n > C, 4.22 k=l ω k Λ n k where {ω k, k = L,..., n} is a sequence of positive numbers such that n k=l ω k = 1, e.g., we can choose ω k = 1/n L + 1. In other words, we consider ω k, L k n as a prior distribution of the change point: P ν = k = ω k. Proposition 4.1 The significance level α of the test satisfies: { n } α = P H0 ω k Λ n k β x k,..., x n > C 1/C. k=l 13

14 Proof. In Appendix. If H 0 is rejected, then the estimator of ν is considered ν n = arg max n m i 2 k n i=k j=1 expx T i β 1,k 1 Y ij 1 + expx T i β 1,k 1 1 expx T i β k,n Y ij 1 + expx T i β k,n, 4.23 where β j,m I Y j, x j,..., Y m, x m, 1 j m n is some esimator of β e.g. MLE based upon Y j, x j,..., Y m, x m. 5 Monte Carlo Simulation Study The results reported in this section focus on the performance of the model 1.1, where m i = 1. The samples were generated from the following scheme for all of the results presented in Tables 1-4: We generated x i = [1, ζ i ] T, where ζ i Uniform[0, 1] and applied β 0 = [ 0.1, 1] T, β 1 = [ 0.1, 3.7] T to model 1.1. Then 3000 simulations of the random sequence {Y i1, 1 i n} were generated, which satisfy 1.1. In our application of the tests 3.14, 4.22 and the estimators 3.17, 4.23 of ν the change point, we used MLEs of β. At this rate, β l,n = β 0, n l 5 was applied in test 3.14, and L = 5, ω k = 1/n 4 were used in the formulation of Table 1 displays the simulated significance levels for the proposed tests 2.4, 3.14 and 4.22 for different sample sizes n and thresholds C, where ν > n in 1.1. From these results, we can experimentally conclude that the upper bounds by Propositions 2.1, 3.1 and 4.1 are reasonable in the considered case. -Table 1 here- Table 2 presents the simulated power of the test 2.4, the boundary by Proposition 2.2, the Monte Carlo mean and the standard error SE of estimator ν n 2.8. The values 14

15 of the boundaries B in Table 2 asymptotically approach the simulated power. -Table 2 here- Tables 3-4 display the simulated powers of the tests 3.14, 4.22, the Monte Carlo means and the standard errors of the estimators 3.17, Tables 3 and 4 here- Additionally to Table 3, we point out that for the situation where n = 75 and ν = 60, if we use the correct critical value C = 16.5 obtained by simulations corresponding to significance level α = 0.05, then the simulated power of the test is P = This result confirms existence of practical meaning of the upper bounds for the significance level of the proposed test. 6 An example: Change in fertility The Israel Central Bureau of Statistics conducted a research aimed at determining whether there was a change in the distribution of the number of children per Israeli woman over the past few decades, and if so, in what year it occurred. The survey, based on a random sample from the 1995 population census of Israel, assumed that the possible change in the distribution may be a result of immigration to Israel. Therefore, all of the women who arrived in Israel after the year in which the change in the distribution occurred would be designated as New Immigrants NI. This designation is relevant for certain sociological applications. Clearly, if there is no change in the distribution then there is no need to define the status NI. 15

16 Consider the following simplified model. Let 1 m i x i = 1, i, T a ij m i j=1 and define the random variables: 1, if the number of children per woman is 2 by 1995 Y ij = 0, if the number of children per woman is > 2 by 1995, where i = 80,..., 95 is the year of arrival in to Israel, a ij the age of the woman ij in the year i, 1 j m i, m i represents the number of women who arrived in Israel during year i and appear in the data. It is known that the average number of children per woman in Israel is about two. Table 5 lists the characteristics of observations Y and the explanatory variables x. -Table 5 here- Now we consider the model 1.1, where d = 3, j = 1,..., m i, i = 80..., 95. Then we test the hypothesis, H 0 : ν / [80, 95], versus H 1 : 80 ν 95; ν > 79 is unknown. There was a prior decision that the definition of the specific status NI for women who arrived in Israel up 1980 is irrelevant. Therefore we use the logistic regression model that is commonly accepted at Central Bureau of Statistics in Jerusalem, based on observations related to the years prior to The model assumes β 0 = [ , , ] T. By applying 3.13, we have ln max 80 k 95 Λ95 k β0, β x k,..., x 95 = ln Λ β0, β x 89,..., x 95 = , where β is the MLE of β 1 β l,n = β 0, n l 7. Therefore, based on the test 3.14 and Proposition 3.1 we reject H 0 with a p value exp

17 By applying 3.17, we obtain ν = 90 as the estimator of the change point. Thus, we can define the status NI for women that came to Israel starting from Appendix 7.1 Proofs of the Propositions Proof of Proposition 2.1. In order to find the relation between C and the significance level of the test 2.4 note: Lemma 7.1 Under H 0, the sequence {Λ n k β 0, β 1 x k,..., x n } n k=1 is a non-negative reverse martingale with respect to {Y k, x k } n k=1 and E H0 Λ n k β 0, β 1 x k,..., x n = 1 for all k = 1, 2,..., n. Proof. Noting that, for all k = 1,..., n 1 we have by 2.3 = E H0 Λ n k β 0, β 1 x k,..., x n Y n, x n,..., Y k+1, x k+1 n m i P ν=k {Y ij x i } P H0 {Y ij x i } E m k P ν=k {Y kj x k } H 0 {Y r, x r } n r=k+1 P H0 {Y kj x k } i=k+1 j=1 = Λ n k+1 β 0, β 1 x k+1,..., x n = Λ n k+1 β 0, β 1 x k+1,..., x n. m k j=1 j=1 E H0 P ν=k {Y kj x k } P H0 {Y kj x k } Therefore E H0 Λ n 1 β 0, β 1 x 1,..., x n = E H0 Λ n 2 β 0, β 1 x 2,..., x n =... = E H0 Λ n n β 0, β 1 x n = 1. This completes the proof of Lemma

18 Denote M n = max 1 k n Λn k β 0, β 1 x k,..., x n 7.24 and define the stopping rule N n = min {τ n C, n}, where for C > 0, τ n C = inf { l > 1 : = if no such l exists. Λ n } 1 β 0, β 1 x 1,..., x n Λ l 1 1 β 0, β 1 x l 1,..., x n > C Now, by Lemma 7.1 and Doob s theorem for a nonnegative submartingale we have α = P H0 {M n > C} = P H0 { Λ n Nn β 0, β 1 x Nn,..., x n > C } 7.25 E H 0 Λ n N n β 0, β 1 x Nn,..., x n C = E H 0 Λ n n β 0, β 1 x n C = 1 C. Proof of Proposition 2.2. We define for all 1 i n m i P λ i ln ν {Y ij x i } a i 7.26 j=1 P H0 {Y ij x i } = x T i β 1 x T m i i β 0 Y ij expxt i β 1, 1 + expx T i β 1 j=1 where a i by 2.5. Noting that, by 7.24 and 2.3 { n } 0 1 P ν {M n > C} P ν λ i + a i lnc { n = P ν λ i } n a i lnc, where, by 2.5 and 7.26 E ν λ i = 0 and λ i m i x T i β 1 x T i β 0, i ν. Therefore, by Hoeffding 1963 we have n 0 1 P ν {M n > C} exp a i lnc 2 2 n m i x T i β 1 β The proof of Proposition 2.2 is now complete. 18

19 Proof of Proposition 2.3. Note that = P ν { ν n t 1 } n P ν k t 1 n { } max 1 j n Λn j β 0, β 1 x j,..., x n = Λ n k β 0, β 1 x k,..., x n k t 1 P ν {Λ n ν β 0, β 1 x ν,..., x n Λ n k β 0, β 1 x k,..., x n }. Hence, by definitions 2.3 of Λ n k β 0, β 1 x k,..., x n and 2.5, 7.26 we have P ν { ν n t 1 } { n k 1 P ν k t 1 λ i } k 1 a i Therefore, by applying the same principle as in the proof of Proposition 2.2, we obtain 2.9. In a similar manner for t 2 < ν. P ν { ν n t 2 } = { } P ν max 1 j n Λn j β 0, β 1 x j,..., x n = Λ n k β 0, β 1 x k,..., x n k t 2 k t 2 P ν {Λ n ν β 0, β 1 x ν,..., x n Λ n k β 0, β 1 x k,..., x n } { ν 1 P ν k t 2 i=k } ν 1 λ i a i exp k t 2 i=k ν 1 i=k a 2 i 2 ν 1 i=k m i X T i β 1 β 0 2 Now, additionally, if condition 2.11 is satisfied then by 2.9, 2.10 and elementary inequalities E ν ν n = n 0 ν + E ν ν n ν 2 = = ν, P ν { ν n > t}dt 7.28 n ν ν P ν { ν n > t}dt 0 maxν 2,n ν 2 0 ν 2 P ν { ν n t}dt, 0 maxν 2,n ν 2 + ν 2 P ν { ν n ν 2 > z}dz P ν { ν n z 1/2 + ν} + P ν { ν n z 1/2 + ν}dz P ν { ν n z 1/2 + ν}dz, 19

20 it follows that the proof of Proposition 2.3 is complete. Proof of Proposition 3.1. Inasmuch as for all k = 1,..., n by 3.13 E H0 Λ n k β0, β x k,..., x n Yn, x n,..., Y k+1, x k+1 = n m i i=k+1 j=1 expx T i β i+1,n Y ij 1+expx T i β i+1,n expx T i β 0Y ij 1+expx T i β 0 m k E H0 j=1 = Λ n k+1 β0, β x k+1,..., x n and EH0 Λ n k expx T k β k+1,n Y kj 1+expx T k β k+1,n expx T k β 0Y kj 1+expx T k β 0 β0, β x k,..., x n = E H0 E H0 Λ n k β0, β x k,..., x n {Yr, x r } n r=k+1 = E H0 Λ n k+1 β0, β x k,..., x n = {Y r, x r } n r=k+1 E H0 E H0 Λ n k+1 β0, β x k,..., x n {Yr, x r } n r=k+2 =... = 1, is a non-negative reverse martin- hence under H 0 the sequence { Λ n k gale with respect to {Y k, x k } n k=1. β0, β x k,..., x n } n k=1 Combining these results with considerations similar to 7.25 completes the proof of Proposition 3.1. Proof of Proposition 3.2. By definitions 3.13, 2.5, 7.26 as well as a Taylor series expansion and some rearranging of terms, the probability of type II error for test 3.14 is bounded { P ν max 1 k n Λn k β0, β x } k,..., x n C { P ν ln Λ n ν β0, β x } i,..., x n lnc n m i = P ν 1 + expx x T T i β 1 β 0 Y ij + m i ln i β 0 j=1 1 + expx T i β 1 n m i + x T i β 1 + expx i+1,n T β 1 Y ij + m i ln i β expx T β i i+1,n j=

21 } { n n lnc = P ν λ i + a i n + x T i β i+1,n m i expθ β 1 i Y ij m i lnc j=1 1 + expθ i { n } n P ν λ i + 1 ρ a i lnc + τ ν,n, where θ i x T i β i+1,n, x T i β 1. Applying to 7.29, the same principle as in the proof of Proposition 2.2 yields { 0 1 P ν max 2 k n Λn k β0, β x } k,..., x n > C exp 1 ρ n a i lnc 2 2 n m i x T i β 1 β τ ν,n. where τ ν,n by 3.16, thereby completing the proof. Proof of Proposition 3.3. P ν { ν n t 1 } For t 1 > ν by 3.17 we have expx n T m β i k,n Y ij i 1+expx P T β i k,n ν expx t 1 k n T i i=k j=1 β 0Y ij 1+expx T i β 0 n m i j=1 expx T β i ν,n Y ij 1+expx T β i ν,n. expx T i β 0Y ij 1+expx T i β 0 Therefore, by definition of β ν,n in 3.17, a Taylor series expansion and some rearranging of terms, we have P ν { ν n t 1 } expx n T m β i k,n Y ij i 1+expx P T β i k,n ν expx t 1 k n T i i=k j=1 β 0Y ij 1+expx T i β 0 { k 1 P ν t 1 k n where a i by 2.5, λ i by λ i k 1 k 1 a i 2 n m i j=1 expx T i β 1Y ij 1+expx T i β 1 expx T i β 0Y ij 1+expx T i β 0 m i x T i β k,n β 1 By applying the same principle as in the proofs of Propositions 2.2, 2.3 and we obtain P ν { ν n t 1 } 21 2c t 1 ν p },

22 In a similar manner P ν { ν n t 2 } 2c t 2 ν p By 7.30, 7.31 and elementary inequalities 7.28, it follows that the proof of Proposition 3.3 is complete. Proof of Proposition 4.1. Without loss of generality, we assume that L = 1 and for all 1 i n: m i = 1. Noting that by definition of β 1,n in 4.21 we have Λ n k β X k,..., X n k 1 Now, for all k = 1,..., n k 1 = E H0 E H0 i=1 k 1 = E H0 i=1 k 1 E H0 f{y i1 x i, β 1,i 1 } f{y i1 x i, β 0 } i=1 f{y i1 x i, β 1,i 1 } f{y i1 x i, β 0 } i=1 Therefore, by 7.32 E H0 Λ n k k 1 = E H0 i=1 n i=k f{y i1 x i, β 1,i 1 } f{y i1 x i, β 0 } n i=k f{y i1 x i, β k,i 1 } f{y i1 x i, β 0 } f{y i1 x i, β 1,i 1 } f{y i1 x i, β 0 } f{y i1 x i, β k,i 1 } f{y i1 x i, β 0 } n f{y i1 x i, β k,i 1 } i=k f{y i1 x i, β 0 } {Y r, x r } n 1 r=1 n 1 i=k E H0 f{yn1 x n, β k,n 1 } f{y n1 x n, β 0 } f{y i1 x i, β 1,i 1 } f{y i1 x i, β 0 } we complete the proof of Proposition 4.1. n 1 i=k f{y i1 x i, β k,i 1 } f{y i1 x i, β 0 } f{y i1 x i, β k,i 1 } f{y i1 x i, β 0 } {Y r, x r } n 1 r=1 =... = 1. β x k,..., x n 1 and by applying Markov s inequality to α, Acknowledgments. The authors are indebted to Professor Moshe Pollak for many helpful discussions and comments. We thank the two referees and Professor Marie Hušková for some helpful comments. 22

23 References Borovkov, A. A Asymptotically optimal solutions in the change-point problem. Theory Probab. Appl Brown, R. L., Durbin, J. and Evans, J. M Techniques for testing the constancy of regression relationships over time with discussion. J. R. Statist. Soc. B Gombay, E. and Horvath, L An application of the maximum likelihood test to the change-point problem. Stochastic Processes and their Applications Hoeffding, W Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc Gossl, C. and Kuchenhoff, H Bayesian analysis of logistic regression with an unknown change point and covariate measurement error. Statistics in Medicine Kim, Hyune-Ju and Siegmund, D The likelihood ratio test for a change-point in simple linear regression. Biometrika Lorden, G. and Pollak, M Non-Anticipating Estimation Applied to Sequential Analysis and Changepoint Detection. Under revision for The Annals of Statistics. Page, E. S Continuous inspectation schemes. Biometrika Pastor, R. and Guallar, E Use of two-segmented logistic regression to estimate change-point in epidemiologic studies. Am. J. Epidemiol

24 Pettitt, A. N A simple cumulative sum type statistic for the change-point with zero-one observations. Biometrika Quandt, R. E Tests of the hypothesis that a linear regression system obeying two separate regimes. J. Amer. Statist. Assoc Robbins, H. and Siegmund, D A class of stopping rules for testing parametric hypotheses. Proc. Sixth Berkeley Symp., Math. Statist. Prob. 4, Univ. of Calif. Press. Sen, A. and Srivastava, M. S On tests for detecting change in mean. Ann. Statist Stefanski, L. A. and Carroll, R. J Covariance measurement error in logistic regression. Ann. Statist Ulm, K A statistical method for assessing a threshold in epidemiological studies. Statistics in Medicine Woodroofe, M Nonlinear Renewal Theory in Sequential Analysis. SIAM, Philadelphia. Worsley, K. J The power of likelihood ratio and cumulative sum tests for a change in a binomial probability. Biometrika Yakir, B On the average run length to false alarm in surveillance problems which possess an invariance structure. Ann. Statist Yakir, B., Krieger, A. M. and Pollak, M Detecting a change in regression: first-order optimality. Ann. Statist

25 Dr. Gregory Gurevich Dr. Albert Vexler Faculty of Industrial Engineering Central Bureau of Statistics and Management 66, Kanfey Nesharim Technion-Israel Institute of Technology Jerusalem 91130, Israel Haifa 32000, Israel Phone: Phone: Fax:

26 Table 1: The simulated significance levels α 1, α 2, α 3 of the tests 2.4, 3.14 and 4.22, where L = 5 respectively ν > n n = 75 n = 120 C Upper Bound: 1/C α 1 α 2 α 1 α 2 α Table 2: The simulations of the power P test 2.4, the boundary B= exp n a i lnc 2 n by 2.6 and the MLE ν 2 m i x T i β 1 β 0 2 n by 2.8 C = 20 ν = 10 ν = 50 n P 1-B ν n SE ν n P 1-B ν n SE ν n Table 3: The simulations of the power P test 3.14 and the MLE ν n by 3.17 C = 20 ν = 10 ν = 50 ν = n 15 n P ν n SE ν n P ν n SE ν n P ν n SE ν n

27 Table 4: The simulations of the power P test 4.22, where L = 5 and the MLE ν n by 4.23 C = 20 ν = 50 n P ν n SE ν n Table 5: Characteristics of the observations used in the example i m i 1 mi m i j=1 a ij 1 mi m i j=1 Y ij i m i 1 mi m i j=1 a ij 1 mi m i j=1 Y ij