Detection and Diagnosis of Unknown Abrupt Changes Using CUSUM Multi-Chart Schemes

Transcription

1 Sequential Analysis, 26: , 2007 Copyright Taylor & Francis Group, LLC ISSN: print/ online DOI: 0.080/ Detection Diagnosis of Unknown Abrupt Changes Using CUSUM Multi-Chart Schemes Dong Han Department of Mathematics, Shanghai Jiao Tong University, Shanghai, China Fugee Tsung Department of Industrial Engineering Logistics Management, Hong Kong University of Science Technology, Kowloon, Hong Kong Abstract: A cumulative sum (CUSUM multi-chart scheme that consists of multiple CUSUM control charts is studied for detecting diagnosing an unknown abrupt change in a stochastic system on the basis of sequential observations. We prove that the CUSUM multi-chart not only has a high diagnostic capability but also possesses a better detection performance than individual CUSUM charts when the in-control average run length is large. We also present an optimal design of the CUSUM multi-chart two illustrative examples involving the normal exponential distributions. Moreover, numerical comparisons of the average run lengths are made via Monte Carlo simulation among the CUSUM, generalized likelihood ratio, exponentially weighted moving average (EWMA, multi-chart, CUSUM multi-chart. The numerical results indicate that the CUSUM multi-chart has the best performance on the whole among the five schemes in detecting the unknown mean shift. Keywords: Asymptotic optimality; Kullback Leibler information distances; Online detection diagnosis; Sequential analysis. Subject Classifications: 62L0; 62N0.. INTRODUCTION The problem of quick detection diagnosis of abrupt changes in a stochastic system has many important applications, including industrial quality control, Received December 8, 2005, Revised March 3, 2006, Accepted April 9, 2006 Recommended by W. Schmid Address correspondence to Fugee Tsung, Department of Industrial Engineering Logistics Management, Hong Kong University of Science Technology, Kowloon, Hong Kong; season@ust.hk

2 226 Han Tsung automated fault detection in controlled dynamical systems, segmentation of signals, so on. To deal with the problem, various control charts, such as Shewhart charts, cumulative sum (CUSUM charts, exponentially weighted moving average (EWMA charts, cumulative score (Cuscore charts, etc. have been proposed. Many studies on these control charts have been conducted by Crowder (987, 989, Lucas Saccucci (990, Montgomery Mastrangelo (99, Baxley (995, Mastrangelo Montgomery (995, Lai (995, Reynolds (996a,b, Box Luceño (997, Ramírez (998, Hawkins Olwell (998, Luceño (999, Mastrangelo Brown (2000, Jiang et al. (2000, Jones et al. (200, Shu et al. (2002. Moustakides (986 Ritov (990 proved that the performance in detecting the mean shift of the one-sided CUSUM control chart with the reference value,, which is related to the mean shift magnitude of particular interest, is optimal in terms of the average run length (ARL if the real mean shift is. In reality, we rarely know the exact shift value of a process before detecting the mean shift. That is to say, the performance of the CUSUM chart in detecting the mean shift depends on the given reference value, which is the magnitude of the mean shift to be detected quickly. For the same reason, the detecting performance of many other control charts, such as the EWMA, the optimal EWMA, Cuscore, is closely related to the given reference value or the reference pattern. Moreover, it is challenging to diagnose the possible size of the mean shift quickly by using a single control chart. There are several schemes that do not depend on a specific shift size. For example, using the generalized likelihood ratio (GLR statistic, Siegmund Venkatraman (995 presented a CUSUM-like control chart, called the GLR chart, that does not depend on the value of. Their simulation results show that the GLR chart is better than the CUSUM control chart in detecting a mean shift that is larger or smaller than is only slightly inferior in detecting a mean shift of size. Also, by taking the maximum weighting parameter in the EWMA control chart, Han Tsung (2004 proposed a generalized EWMA (GEWMA control chart that does not depend on the reference value proved that the GEWMA control chart is better than the optimal EWMA in detecting a mean shift of any size when the incontrol ARL is large. However, these methods usually require complex computing thus have not been regularly applied to real online problems in practice. To solve the problem of detecting the unknown magnitude of the change, various detection schemes have been proposed studied. The pioneering work on this issue was done by Lorden (97 Lorden Eisenberger (973 on charting a set of CUSUM statistics. Dragalin (993, 997 investigated the design analysis of a combination of two CUSUM charts. Sparks (2000 further explored this idea studied a combination of three CUSUM charts in particular via simulation. On the other h, Willsky Jones (976 introduced the windowlimited GLR scheme, which was theoretically investigated by Lai (995, 998 by Lai Shan (999. Although the window-limited GLR scheme the GLR control chart have good performance in detecting the unknown magnitude of the changes, their computational complexity lack of a capability in diagnosing the possible magnitude of the changes restrict their application in real online problems. To make the GLR scheme practicable, Nikiforov (2000 proposed a suboptimal recursive approach that is based on a collection of L parallel recursive 2 -CUSUM charts established a direct relation between the efficiency of the detection

3 Detection Diagnosis of Unknown Abrupt Changes 227 scheme its computational complexity. Motivated by Nikiforov s recursive approach, we further extend the study of the CUSUM multi-chart. The objective of this paper is to demonstrate that the CUSUM multi-chart can meet to a considerable extent the following three goals in detecting diagnosing the unknown abrupt change in a stochastic system: ( signal an alarm as quickly as possible when having an abrupt change, (2 accurately indicate the possible type/amount/size/etc. of the change, (3 easily hle computational complexity. The remainder of this paper is organized as follows. In Section 2, the CUSUM multi-chart a charting performance index (CPI to evaluate the detection power of a control chart over a whole region of possible abrupt changes are introduced. The asymptotic ARLs, optimal design, diagnostic capability of the CUSUM multi-chart are presented in Section 3. Two illustrative examples simulation comparisons are demonstrated in Section 4. Conclusions problems for further study are discussed in Section 5, with the proofs of two lemmas two theorems given in the appendix. 2. THE CUSUM MULTI-CHART SCHEME Let X i i= 2 be the ith independent observation on a process with a known common probability distribution, f 0. Suppose that at some time period,, which is usually called a change point, the probability distribution of X i changes from f 0 to f, where f f 0 if only if 0. In other words, from time period onwards, X i has the common distribution f. Note that the parameter may not be the characteristic number of the distribution f, for example, the mean, deviation, etc., which is often used just to distinguish the different distributions. In practice, the change point the postchange distributions are usually unknown, that is, are two unknown parameters. However, we may assume that the possible unknown postchange distributions f belong to a closed domain D, where D = f D D is a closed boundary set of the parameters, the boundary D of D is known. The possible change domain its boundary (including the size form of the boundary about the observation process may be determined by engineering knowledge, practical experience, or statistical data if the possible unknown postchange, f, is a dominated family of distributions with parameters. For example, when f is the normal density function = a b, where a b denote the mean stard deviation respectively, we can take the set D = a b a a a 2 0 <b b b 2, where a a 2 b, b 2 are known numbers, which means that we know the domain of the possible postchange distributions, that is, the boundary D of the parameter set is known. Note that the mean of the process with gamma distribution may keep with a constant when the postchange occurs since the mean of the gamma distribution is /, where = are the parameters of gamma distribution. If f is subject to a family of multidistributions, for example, multinormal distributions N , we can similarly choose a closed boundary set D of the parameters = 2 2 If the possible postchange distributions consist of two kinds of family distributions (normal stable distributions, it is not clear

4 228 Han Tsung easy for us to choose a closed boundary set satisfying conditions I II defined in Section 3. In this paper we mainly consider the case that the possible postchange distributions belong to a set that consists of a dominated family of distributions f with the parameter. Moreover, we assume that 0 D f f if only if, where f, f D, D. In order to detect diagnose the unknown abrupt changes, the CUSUM multi-chart, T MC, is defined as follows: fixed m 2 T MC = min T kd k k m { n T k d k = min nmax j n i=j log f } k X i f 0 X i >d k where d k > 0 k m are control limits; k D f k D, k m are prespecified known reference values density functions, respectively; f 0 is a known common probability density of X i before the change point. Let E, E 0, E k k m denote the expectations corresponding to the unknown postchange density, f ; the known common probability density, f 0 ; the known reference density functions, f k k m, respectively. We often use ARL ARL 0 to denote E E 0, respectively. The unknown abrupt change, f, will be detected diagnosed according to the following procedure. Take the control limits, d k k m, such that E 0 T k d k = L for all k m, where the number L is a positive constant. If the CUSUM multi-chart test, T MC, sends out an alarm of having an abrupt change, the alarm certainly comes from one of the CUSUM charts, for example, T l d l. That is, T l d l is the first one to tell us that the change occurs. In this case, we may say that the unknown postchange probability density, f, is near f l since the CUSUM chart with the reference density function f l has the best detection performance in terms of the ARL if the real postchange is f l. Here the chart has the best detection performance means that it has the smallest ARL among the m CUSUM charts with common ARL 0, that is, E l T l d l <E l T k d k for k l, k m. In order to evaluate the detection power of a control chart, T with E 0 T = L, over a whole region of possible abrupt changes, we present the CPI in the following: { CPIT = exp D w E T ARL } ARL d where ARL is a reference ARL value of well-known procedure at the postchange probability density, f, w is a positive weighting function with wd = D to emphasize various postchanges within the region D based on prior knowledge experience with the process. Usually, we may take w such that w > w when I 0 > I 0 if the large change is considered more important than the small change. Thus, we can compare the charts by the CPI to know which performs better in detecting such a large change. If no prior information preference are provided, we may use a equal weight w = MD throughout the region, where M is a measure of the region D.

5 Detection Diagnosis of Unknown Abrupt Changes 229 Let I 0 denote the Kullback Leibler information distance (number defined by [ I 0 = E log f ] X f 0 X Taking ARL log L (2. I 0 which is Lorden s (97 asymptotic lower bound, that is, as L where inf E T log L TE 0 T L I 0 E T = sup ess sup E T n + + X X n n n is change time, we have 0 < CPIT for large L if E T E T for large L. Obviously, the higher the CPI the better detection performance for the chart. Compared with traditional evaluation methods using ARL at a single point, the proposed CPI can take the whole postchange region into consideration. 3. THE ASYMPTOTIC ARL AND THE DIAGNOSTIC CAPABILITY OF THE CUSUM MULTI-CHART In this section, we first present the asymptotic ARL. We then introduce the optimal design diagnostic capability of the CUSUM multi-chart for the unknown abrupt change, that is, the unknown postchange probability density function f. 3.. The Asymptotic ARL the Optimal Design of the CUSUM Multi-Chart In order to obtain the asymptotic ARL of the CUSUM multi-chart, we first divide the region, D, into several disjointed subsets by using the reference density functions, f k k m according to the Kullback Leibler information distance. Let the prespecified reference density functions be f k D k D, k m. Let { } J k = f D I k min I j j k for k m = J k k = J k J k J j j=

6 230 Han Tsung for 2 k m, where f 0 D is the known density function before the abrupt change. It can be seen that the sets k, k = 2m are disjointed m k= k = D. Since f f if only if, we have a disjointed division of the region D, D k, k = 2m such that m k= D k = D, k D k, D k if only if f k k m. Thus, the CPIT MC can be written as { m CPIT MC = exp w k= Dk E T MC ARL } ARL d (3. To estimate the ARL of the CUSUM multi-chart scheme, we define a generalized Kullback Leibler information distance (number, I k 0,as [ I k 0 = E log f ] k X = I f 0 X 0 I k for k m. Note that the number I k 0 may be negative. Obviously, I k k 0 = I k 0 or 0 if = k or k = 0. Moreover, by the definition of k, we have f k k (or D k D k if only if I k 0 > max j k I j 0 since I k 0 = I 0 I k can only occur on k. Thus, we can say that the unknown postchange probability density function, f,is near the prespecified reference density function, f k (or is near the prespecified reference value, k if only if f k ( D k. Here, we assume that the chosen reference density functions, f k D k m satisfy the following conditions: I. 0 < I 0 k < I 0 k+ for k m. II. I k 0 >0iff k or D k for k m. III. For each f k f k, k m, there exists a positive number such that E 0 exp logf k X/f 0 X < E exp logf k X/f 0 X < Moreover, the rom variables logf k X/f 0 X, k m are not degenerate. Condition I implies that the reference density function, f k+, is farther from f 0 than from f k according to the meaning of the Kullback Leibler information distance (number. The inequality I k 0 >0, that is, I 0 > I k, implies that the information distance between f f 0 is greater than that between f f k. Thus, condition II can hold if we take many f k, k m, such that I 0 > I k for all f k ( k m. Condition III is just Cramèr s condition (see Shiryaev, 995. This condition means that E 0 logf k X/f 0 X n < E logf k X/f 0 X n < for n. It usually holds for observation processes in industrial practice. Next we mention two lemmas three theorems. The proofs of Lemmas Theorems will be laid down in the appendix. Lemma 3.. Let f D. Then, (a T k d k a.s.-p as d k for k m (b the series T k d k /d k d k > 0 is uniformly integrable with respect to P for each f k. Lemma 3. will be used to prove the following Theorem 3., which gives the asymptotic ARL of the CUSUM multi-chart.

7 Detection Diagnosis of Unknown Abrupt Changes 23 Theorem 3.. Let T k d k k m have a common ARL 0 = L, that is, L = E 0 T k d k holds for k m. Iff k, then, a.s-p, for l k, as L. T k d k T l d l max0 I l 0 I k 0 T MC d k = T kd k d k + o T k d k < T l d l I k 0 ( ( TMC Tk d E = E k d + o k d k I k 0 (3.2 (3.3 By Theorem 3., T k d k < T l d l, a.s-p as ARL 0 = L for l k when f k that is, ( ( 0 = P lim T kd k T l d l 0 P lim T kd k T l d l = 0 L L for l k when f k Thus, the case that T l d l = T k d k = mint j d j l k, can be neglected for large L. Usually, comparisons of control chart performance are made by designing the charts to have a common ARL 0 then comparing the ARL s of the control charts. The chart with the smaller ARL is considered to have the better performance. Theorem 3.2 in the following will show that the performance of the CUSUM multi-chart can be better than that of its constituent charts in detecting an unknown abrupt change. That is to say, the CUSUM multi-chart has the better performance than that of using the CUSUM charts separately in detecting the unknown abrupt change. To compare their ARLs, we introduce two common ARL 0 s, L, L, respectively for T k d k T MC in the following. Take the control limits, d k, k m, such that d k >d k E 0 T d = =E 0 T m d m = L >E 0 T d = =E 0 T m d m = L = E 0 T MC (3.4 where T MC = min k mt k d k. To prove Theorem 3.2 we present Lemma 3.2 in the following, which gives some relations between two ARL 0 s, L, L, two control limits, d i d i. Lemma 3.2. If the control limits d k >d k k m satisfy (3.4, then < L L m + o 0 <d k d k log m + o for large L.

8 232 Han Tsung Theorem 3.2. Let p l l m be positive numbers satisfying m l= p l = If condition (3.2 holds for T MC for f k there is f i i k f i i such that I i 0 < I k 0, then, for every f D, for large L. m p l E T l d l >E T MC l= Note that the condition I i 0 < I k 0 in Theorem 3.2 means that I i > I k, that is, the information distance between f f i is greater than that between f f k when f k. The optimal design of the CUSUM multi-chart can be obtained from the following Theorem 3.3. Note that the value CPIT MC depends on f f m.so the value CPIT MC can be denoted by CPI m. Theorem 3.3. There exist the numbers k D k k m such that CPIT MC = CPI m = { = exp k= max CPI m m m } I w 0 + o D k I 0 I k d (3.5 for large L. Proof. It follows from (2., (3., (3.3 that { CPI m = exp { = exp m w Dk E T MC ARL ARL m I w 0 D k I 0 I k k= k= } d } d + o for large L. Thus, Theorem 3.3 holds since m I w 0 D k I 0 I k d k= is a continuous function on m D is a closed set. Remark 3.. The CUSUM multi-chart, T MC, satisfying (3.5 can be called an optimal CUSUM multi-chart, which is denoted as TOMC. Let k = maxi D k. Since min D I 0 > 0, it follows from (3.5 that CPI m as max k m k 0 L. Thus, for any small >0, we can take the smallest positive integral number, m, such that CPI m

9 Detection Diagnosis of Unknown Abrupt Changes 233 for large L, where m = minm CPI m. By this procedure, the optimal design of the CUSUM multi-chart, TOMC, can be obtained. That is, T OMC = min k m T k d k { n T k d k = min nmax j n i=j log f k X } i f 0 X i >d k CPIT OMC = CPI m for large L. However, it is usually not easy to get the optimal numbers, m. Note that the number of k s depends not only on the dem to signal an alarm as quickly as possible when having an abrupt change but also on the accuracy to indicate the possible type/amount/size/etc. of the change. So, the following way may be better to determine the number of k s: Let be any two small positive numbers. Taking the smallest positive integral number, m = m, that is, m = minm 2 CPI m, where the numbers m satisfy CPI m = max CPI m m I k for D k k m, this number m is the one we want to choose The Diagnostic Capability of the CUSUM Multi-Chart First, we give a definition of diagnostic capability for a multi-chart. Definition 3.. Let Z be a rom variable that denotes the possible abrupt changes T be a multi-chart that consists of several control charts, T k k m where T k depends on the reference density function, f k. Assume that PT = T k > 0 for all k m. Then a diagnostic capability of T is defined by DCT = m m P ( Z = f k T = T k k= Since the conditional probability, PZ = f k T = T k can be considered as a diagnostic capability of T DCT is then the average diagnostic capability of T. In other words, DCT denotes a percentage of judging correctly the possible type (or amount, size, etc. of an abrupt change. In the following, we show that the diagnostic capability of the CUSUM multi-chart goes to when the control limit or the in-control ARL (ARL 0 goes to infinity.

10 234 Han Tsung Theorem 3.4. Let Z be a rom variable taking value in with PZ k >0 for k m, T MC = min k m T k d k be the CUSUM multi-chart, T k d k, k m have a common ARL 0 = E 0 T k d k = L. Then as L. P ( Z = f k T MC = T k d k DCT MC Proof. Let T j = T j d j j m. It follows from Theorem 3. that P ( T j >T k j k f k = P ( TMC = T k f k as L. That is, P ( T MC = T j f k 0 j k as L. Thus, P ( Z = f k T MC = T k PZ = f k PT MC = T k f k = PZ = f k PT MC = T k f k + j k PZ = f j PT MC = T k f j as L, so that DCT MC asl. In Theorem 3.4, T MC = T k d k means that the CUSUM chart, T k d k, first sends out an alarm of having an abrupt change among all T j d j j k, we can see that the probability of accurately diagnosing the possible change type (or amount, size, etc. that is near f k is approaching as L goes to infinity. 4. ILLUSTRATIVE EXAMPLES AND SIMULATION COMPARISONS 4.. Illustrative Examples Two examples are given in the following to demonstrate the applications of the theorems. Example 4.. Consider the problem of detecting diagnosing the abrupt mean shift >0 in a stochastic process, X n n= 2 which are independent identically distributed (i.i.d. normally rom variables with variance 2 =. Assume that the prechange mean, 0 = 0, the possible interval of the mean shift are a b, that is, a b, where a b are two known positive numbers. Then, we can constitute the CUSUM multi-chart, T MC = min k m T k d k, by choosing

11 Detection Diagnosis of Unknown Abrupt Changes 235 the reference values, 2 m in the interval. With no loss of generality, it is assumed that the k k m, are strictly increasing with k a< < 2a. Obviously, condition III holds for the normal rom variables. It can be derived that I k 0 = k k /2 I k = k 2 2 D k = k + k /2 < k + k+ /2 We can see that I 0 k < I 0 k for k m. Note that I x 0 attains its maximum value at x =.If D l, then l < 2, l min k l k k < 2 I l 0 >0. Thus conditions I III hold for the example. Hence, by Theorem 3., we have for j l T l d l < T j d j T MC d l l l /2 ( TMC E d l l l /2 as-p as L. Let w = b a. As an application of Theorem 3.3, we can check that the optimal numbers, k 2 k m, are respectively the unique solutions to the integral equations I x = x+ 2 2 a x 2 x 2 x/2 2 d = 0 I kx = x+ k+ 2 k +x 2 x 2 d = 0 (4. x 2 x/22 for 2 k m, where 0 = 0, a< < 2a m < m+ = b. The optimal design k k m can be calculated with a computer with little difficulty. For example, take a = 0, b = 4, m = 5. By using (4., we can obtain the theoretical optimal reference values, k, k m, that is, = = = = 86 5 = 326 therefore CPITOMC = As can been seen, the theoretical value of the optimal CUSUM multichart is quite high. Its exceptional performance is verified by the simulation result in Table, which shows that the optimal CUSUM multi-chart with the five CUSUM charts, T k = Tk d k, has the best performance among the five control charts in terms of CPI, even when ARL 0 is not large. Moreover, it follows from Theorem 3.4 that as L for l m. PZ = J l T MC = T l d l

12 236 Han Tsung Table. Comparisons of the ARLs of the CUSUM, EWMA, GLR, EWMA multi-chart, CUSUM multi-chart, optimal CUSUM multi-chart with ARL 0 = 500 Shifts CUSUM CUSUM Optimal CUSUM EWMA EWMA GLRT GL ( = Multi-chart Multi-chart r = 0 Multi-chart d = ( ( ( ( ( ( (32 262(7 273( ( ( ( ( ( (6 06.5( (39 3.6( (32 36.(2 35.7( ( (3 36.9( ( 8.4(0 9.0(0 5.87(9 8.4(2 8.6( 0.2(6.5(6.9(6 0.3(5.2(6.4( (3 8.05(4 8.2(4 7.65(3 7.79(4 7.79( (2 6.02(3 6.07(3 6.09(2 5.85(3 5.75( ( 3.8(2 3.79(2 4.36( 3.69(2 3.6( ( ( ( (0.7.94(0.9.94( (0.4.56(0.8.34( (0.4.28(0.5.32(0.5 CPI Example 4.2. Let X n n= 2 be i.i.d. with an exponential distribution with parameter >0. Assume that the prechange mean is / 0 the possible interval of the mean shift is /b /a, where b>a>0. Without loss of generality, we choose the reference values, k ( k m, in a b such that 0 < 0 < k < k for k m. Let / /b /a be an unknown mean shift. It can be calculated that I k = k log k I k 0 = 0 k + log k 0 D k = { k k < log k log k k+ k log k+ log k } It can be checked that I 0 k < I 0 k I k 0 >0 for D k k m. Moreover, the example satisfies condition III if < 0. Thus, the results in Theorems hold for the example Comparisons with Numerical Simulation Results To compare the detection performance, numerical simulation results of ARLs of the two-sided CUSUM, EWMA, CUSUM multi-chart, EWMA multi-chart, GLR charts are presented in Table. The numerical results of ARLs were obtained based on,000,000-repetition experiments. The common ARL 0 here is chosen to be 500,

13 Detection Diagnosis of Unknown Abrupt Changes 237 which is a typical value in practice. We compare the simulation results for 0 mean shifts ( = 0 2 = = 4 listed in the first column of Table with change point =. The values in parentheses in every column of the tables are the stard deviations of the ARLs. The simulation results of the ARL s of the CUSUM chart are listed in the second column of Table, that is, T d with the reference value = the control limit d = 5075 such that ARL 0 T 5075 = 500. The CUSUM multi-chart results are listed in the third column, where the CUSUM multi-chart, T MC, consists of five CUSUM charts, T k d k, k 5 with = 0, 2 = 05, 3 =, 4 = 5, 5 = 2. To maintain the overall ARL 0 T MC = 500, we choose their control limits to be d = 24546, d 2 = 5227, d 3 = 607, d 4 = 62572, d 5 = so that ARL 0 T d = 5000, ARL 0 T 2 d 2 = 5000, ARL 0 T 3 d 3 = 5000, ARL 0 T 4 d 4 = 5000, ARL 0 T 5 d 5 = The simulation results for the optimal CUSUM multi-chart, TOMC, are listed in the fourth column with reference values, k, which are determined by (4., that is, = = = = 86, 5 = 326, where the control limits, d = 364d 2 = 524d 3 = 67d 4 = 645, d 5 = 692 are taken in order to obtain ARL 0 TOPM = 500. Also, the simulation results of ARLs for the EWMA chart, T E r d with the parameter r = 0 control limit d = 288, the EWMA multi-chart, T ME are listed, respectively, in the fifth the sixth columns, where T ME consists of five EWMA charts, T E r k d k k 5, with r = 0r 2 = 03, r 3 = 05r 4 = 07, r 5 = 09 Similarly, we choose the control limits, d k k 5, in order to maintain ARL 0 T ME = 500. Moreover, we list the simulation results of the GLR(T GL in the last column with the control limit d = 3497, which leads to the same ARL 0 value. We take w = /39, 0 4 for CPI. The bottom row of Table lists the CPI values, which were calculated based on all the mean shifts listed in the first column to represent the performance for a range of unknown mean shifts. Table shows that each charting scheme has its strengths weaknesses over the range, it is difficult to compare them in terms of ARL. However, in terms of CPI for the overall performance, we can see that the optimal CUSUM multichart is superior to the CUSUM, CUSUM multi-chart, EWMA, EWMA multichart, GLR charts on the whole in detecting various mean shifts in the range. An interesting result in Table is that the stard deviations of the ARLs for the CUSUM multi-chart the optimal CUSUM multi-chart are smaller than those for the other charts in detecting small mean shifts. 5. CONCLUSIONS AND DISCUSSIONS Although we rarely know the exact change of a process before detection, we may assume that the possible unknown postchange f belongs to a known closed domain D. In this case we consider a CUSUM multi-chart scheme that consists of multiple CUSUM control charts for detecting diagnosing the unknown abrupt change. It is proved that the CUSUM multi-chart not only has a high diagnostic capability but also has better performance than using individual CUSUM charts in detecting an unknown abrupt change when the in-control ARL is large. Comparisons of numerical simulation results show that the optimal CUSUM multi-chart has the best performance according to the CPI values among the five

14 238 Han Tsung schemes, CUSUM, EWMA, CUSUM multi-chart, EWMA multi-chart, GLR charts, in detecting various mean shifts. As can be seen, the Kullback Leibler information distance the generalized Kullback Leibler information distance play an important role in evaluating not only the detection performance but also the diagnostic capability of the multi-chart. There are still several issues regarding the multi-chart that merit further research. First, we can see that the main results for the CUSUM multi-chart are based on the assumption that the constituent charts of the CUSUM multi-chart have a common ALR 0. It would be of interest to study whether the same results still hold for the CUSUM multi-chart when its constituent charts have different ALR 0 s. Note that if the small mean shift is considered to be more important than the large one, the ALR 0 of the control chart for a small mean shift may be chosen to be smaller than that of the control chart for a large mean shift, so that the small mean shift can be detected more quickly. In this paper the comparisons of the detection performance of the control charts are based on the ARL. Another interesting alternative would be to compare the charts using the average delay as a criterion. Let N denote the number of false alarms before the change time i for i N + be the consecutive alarm intervals until the detection of the change point. Thus, The average delay time for = t is thus + + N < + + N + N + ADTt = E t + + N + t where E t denotes the expectation when the place of shift is at a fixed time t. When t = 0, it becomes the out-of-control ARL ARL = ADT0. Srivastava Wu (993 have used the so-called stationary average delay time, lim t ADTt, as the main measure for evaluating the performance of a detecting procedure. Usually, the number position of the reference parameters m will be increased more scattered in order to keep T MC to have an effective detection performance when the domain D becomes large. This case is also true for the optimal parameters m. As has been seen, the number of the reference parameters in the paper is fixed. So, it is interesting to study the influence of a (much larger D on the results. Also, it is known that, in many practical applications, the i.i.d. assumption does not hold. Thus, it is worthwhile to extend the theorems in this paper to non-i.i.d. situations such as autocorrelated processes. Such extensions should enhance the potential applications of the multi-chart. APPENDIX: PROOFS OF THE LEMMAS AND THEOREMS Proof of Lemma 3.. It follows that ( m P T k d k n = P max max m n j m i=j ( n P log f k X i f i= 0 X i log f k X i f 0 X i >d k >d k ni k 0 d k

15 Detection Diagnosis of Unknown Abrupt Changes 239 where the last inequality is obtained by using Markov s inequality I k 0 = E log f k X f 0 X Obviously, P T k d k n converges to zero as d k since, by condition III, I k 0 < for D k m. Thus, T k d k in probability as d k. This implies that a subsequence of the T k d k s goes to infinity a.s.-p. It is obvious that T k d k is nondecreasing a.s.-p as each of the d k s increase to infinity. This means that T k d k a.s.-p as d k. Thus, the first part (a of the lemma is proved. Let { n N k d k = min n log f } k X i f 0 X i >d k i= It is clear that T k d k N k d k. Thus, the uniform integrability of T k d k /d k d k follows from the well-known uniform integrability of N k d k /d k (for the proof of the latter see, for example, Gut, 988. Proof of Theorem 3.. We first prove that T k d k < T l d l a.s.-p as L for f k l k. It is known that (see Siegmund, 985, p. 26 L = E 0 T k d k = ed k dk I 0 k (A. Thus, we have for large L, where d k = d l + Ck l + o log L = d k log I 0 k + o (A.2 Ck l = log I 0 k I 0 l From (A. we know that L if only if min k m d k. By condition I (A.2 it follows that d k <d l when k<lsince Ck l < 0 for k<l. By condition II we know that f k means that I k 0 I l 0 for l k I k 0 >0. Assume that I k 0 > I l 0 for l k. By the strong law of large numbers we have max j n n n i=j log f l X i f 0 X i max0 I l 0 a.s.-p (A.3 for l m as n. Writing T j = T j d j j m for short, we have max j T l T l i=j log f l X i f 0 X i >d l T k max log f k X i j T k f 0 X i d k i=j

16 240 Han Tsung It follows from (A.2 that max j T l T l T l i=j log f l X i f 0 X i > d l = d k Cl k + o + T l T l T l T k T l max j T k T k T k i=j log f k X i f 0 X i Cl k + o + T l (A.4 Note that Cl k/t l 0 a.s.-p Lemma 3., we have as L. By (A.3, (A.4, Part (a of max0 I l 0 T k I T k 0 l a.s.-p as L. This means that T k <T l, a.s.-p as L for l k since I k 0 > I l 0 Let I k 0 = I l 0, that is, I k = I l. By the definition of the s we know that if f k I k = I l, it must have l>k. For instance, f can only belong to the boundary of k k+, not to the boundary of k k when f k. Note that max j T k max j T k T k T k i=j T k T k i=j log f k X i f 0 X i > d k T k log f k X i f 0 X i By the strong law of large numbers, this implies that d k T k T k /d k I k 0 a.s.-p as L. Similar result can be obtained for T l. Thus T k d k < T l d l, a.s.-p since d k >d l T k d k d k I 0 I k = I 0 I l T ld l d l a.s.-p as L. Since T k d k < T l d l a.s.-p as L it follows that T MC T k d k 0 a.s.-p as L, therefore, T MC d k I k 0 a.s.-p as L. Note that the family T MC /d k d k > 0 is uniformly integrable with respect to P since T MC T k d k, T k d k /d k d k > 0 is uniformly

17 Detection Diagnosis of Unknown Abrupt Changes 24 integrable with respect to P. Thus, by application of Theorem A.. in Gut (988, we have ( TMC E d k I k 0 as L. Proof of Lemma 3.2. We first show that Lemma 3.2 holds for m = 2. By (3.4, (A., (A.2, we have [ ( ] d i d j = + od Ii i d j = + o ln (A.5 for large L. It follows that L L = E 0 T d E 0 T MC = = + n=0 + n= { ( P0 T d >n ( P 0 Ti d i>n i 2} P 0 ( T2 d 2 n T d >n Writing T i = T i d i i = 2, P = P 0 E = E 0 in short, T i expressed by (see Siegmund, 985, p. 25 I j can be T i = N i + N i 2 + +N i K i (A.6 where S i n = n j= logf i X j /f 0 X j, ET i i = EN EKi N i = inf { ns i n 0d i } N i k = inf { n S i N + +N k +n S i N + +N k 0d i } K i = inf { ks i N i i + +Nk S i N i i + +Nk d i} Moreover, { N i k k } is i.i.d. with mean E ( N i K is geometrically distributed with mean EK i = /P ( S i ( d i N i i. Note that E S = Ii = I 0 i <0. We can further prove that there exists >0 such that E ( i N e < (A.7 for i = 2. In fact, by condition III Chebyshev s inequality it follows that e ni i P ( S i n + ni i ni i E ( e S i n +ni i = h n

18 242 Han Tsung where h i = E ( e logf i X j /f 0 X j +I i. Taking small positive such that I i log h i > 0, we have P ( N i >n P ( S i n 0 = P ( S i n + ni i ni i e ni i log h i <, i = 2 Let pi = This implies that (A.7 holds for N i. Obviously E( N i P ( S i d N i i, Ei = E ( N i f = E ( e N E f 2 = E ( e N 2 +E 2. Set n = d ed /I E n = n/e +, n 2 = n/e 2 for small 0 <<. It follows from (A.6 that + n= P ( T 2 n T >n n P ( T 2 n T >n + where Qd d 2 n= n = n= k= j= + n=n + p 2 p 2 k p p j P ( T >n = Qd d 2 + Qd P ( N 2 + +N 2 k n N + +N j >n Qd = n=n + j= p p j P ( N + +N j >n Taking such that < a i = log f i > 0 for i = 2, we have (by Chebyshev s inequality ( k P l= N 2 l for k>n 2. Similarly, ( j P l= N l for j n. Thus { n Qd d 2 n= ( k n = P l= N 2 l + ke 2 ke 2 n/k exp ke 2 n/k log f 2 exp ka 2 ( j n = P k=n 2 + j= l= N l je jn/j E exp jn/j E log f exp ja ( k p 2 p 2 k P l= l= N 2 l n n ( j } + p p j P N l >n n { n2 + n= k= j=n + p 2 p 2 k p p j }

19 Detection Diagnosis of Unknown Abrupt Changes 243 where n { n= k=n 2 + n } p 2 p 2 k e ka2 + p p j e ja n + p 2 n 2 p n n= n { p 2 p 2 e a2 n 2+ p 2 p 2 e a 2 n= j= + p p e a p e a n p p e a } + Pd d 2 n p 2 p 2 p 2 e a 2 + n p p p e a + Pd d 2 n Pd d 2 = p 2 n 2 p n n= = p E + E p n + E p + p E + p 2 E 2 p n E + p 2 n p E + p 2 E 2 Note that p i = PS i d N i i = E i ET i = E ii i e d i + Od i e d i for i = 2 d 2 d = + o logi 2/I n p 2 d I 2E 2 /I E e d 2 d d E 2/E, n p = d + o. Furthermore, p E + I E e d /E + p 2 E 2 E 2 I 2 E 2 e d 2 /E2 p n E + Oe d 0 p2 n E 2 Oe d 0 as d i i= 2 We have E p Pd d 2 = + [ p E + E ( p 2 2 ][ p E + p 2 E 2 ] + O ( = ed E + I E + E 2 E + Oe d E 2 E + [ ] [ ] = ed 2 I 2 + O + = L Oe d + O + Oe d

20 244 Han Tsung for large d or L small. Thus [ ] Qd d 2 L 2 + O + Oe d + Od /L Next we estimate Qd. Taking a such that a = f a/f a for a>e, it follows that a = f /f is strictly increasing (see Durrett, 996, pp. 7 73, so that E = 0, E + > 0 therefore log f E + > 0. Further, the function gx = x x log f x is increasing for x E since g x = x 2 log f x 0 for x E ge = 0. Thus P ( N + +N j > jn/j exp jn/jn/j log f n/j { [ = exp n n/j j ]} n log f n/j { [ exp n E + ]} E + log f E + = e nge + for j n = n/e +. Hence { Qd n ( p p j P N + +N j >n + n=n + n=n + j= p n e nge + + n=n + p n j=n + +ge e n + e + p ge + p n + = + o I E for large d. Thus we have L L = E 0 T d E 0 T MC Qd d 2 + Qd ( L 2 + O + Oe d + O/L + Od /L p p j } for large L. This implies that L /L 2 + o for large L. It follows from (3.4 (A. that L /L = e d i d i + o. Thus, d i d i log 2 + o for large L. For m 3 we define T k MC = min j kt j d j for k m. Since TMC = T d m TMC = T MC, it follows that Note that m L L = E 0 T d E 0 T MC = ( k E 0 T MC E 0 T k+ MC = + n=0 k= ( E0 T k MC E 0 T k+ MC P ( T k+ d k+ n T jd j>n j k n P ( T k+ d k+ n T jd j >n j k + Qd n=

21 Detection Diagnosis of Unknown Abrupt Changes 245 We can similarly prove that n n= P ( T k+ d k+ n T jd j>n j k n p k+ k p k+ p k+ e a k+ + n p j p j p j e aj + Pd d k+ j= P ( d d k+ n k = p k+ n k+ p j n j = n= j= [ k j= p E j j + p k+ [ ][ k j= p E j j + p k+ ( = ed I k j= E j E j + ( + E k+ k E k+ j= ( = L + O + o kk + E k+ E k+ k + o E j E j + ] E j + j= p j ] + O for large d or L small, where p j = P ( S j d N j, nj = n/e j j + j k, n k+ = n/e k+. Thus ( m L L L + O + o = L ( m kk + + O + o k= therefore L /L m + o d k d k log m + o k m for large L. This completes the proof of Lemma 3.2. Proof of Theorem 3.2. Let f D. Without loss of generalization, we assume f k. This means that I k 0 I i 0 for i k I k 0 >0. By the condition of Theorem 3.2, there is l D l l k such that I k 0 > I l 0. Let T j = T j d j j m It follows from (A.4 that Y l d l = max j T l T l T l i=j log f l X i f 0 X i T l > d l Y l d l > 0

22 246 Han Tsung therefore Note that d l /d k, ( ( Tl T E k E d k d l d k Y l d l ( Tk E d k I k 0 ( Tk E d k Y l d l max0 I l 0 a.s.-p as L. Here, L if only if min k m d k. Thus, by Fatou s lemma we have ( lim inf E Tl T k L d k max0 I l 0 I k 0 ( I = k 0 I k 0 max0 I l 0 therefore ( { lim inf E Tl T k + if I l 0 0 = L d k if I l 0 >0 we have I k 0 I l 0 I k 0 I l 0 Taking c>0 such that c < I k 0 letting c l = I k 0 maxc I l 0 I k 0 maxc I l 0 E T l T k /d k c l > 0 for large L. If there is l such that I l 0 = I k 0, by the definition of the divide domains, j j m, we know that it must have l>k, therefore (see the proof of Theorem 3. d l >d k. Defining a set A k, every j A k satisfies j>k I j 0 = I k 0. The set A k may be empty if there is no j such that I j 0 = I k 0. Let A k = m A k. Similarly, we have E T l T k /d k c l > 0 for l A k large L. On the other h, by Lemma 3.2 (3.3 of Theorem 3. we have d i d i log m, d i /d i for i m, E T k d k E T k d k d k = E T k d k d k d E k T k d k d k d k I k 0 I k 0 = 0

23 Detection Diagnosis of Unknown Abrupt Changes 247 as L. Similarly, we have (see the proof of Theorem 3. E T j /d j I j 0 E T j T k /d k 0 for j A k as L since I j 0 = I k 0 d j /d k asl. Note that E T k d k E T MC. Thus [ m ] p l E T l d l E T MC d k l= = m p d l E T l d l E T k d k + E k d T k d k E T MC k l= = d k l Ā k p l E T l d l E T k d k + d k j A k p j E T j d j E T k d k + E d T k d k E T k d k + E T k d k E T MC k p l c l + o >0 l Ā k for large L, therefore, m p l E T l d l >E T MC l= for large L. ACKNOWLEDGMENTS We thank the special issue guest editor two referees for their valuable comments suggestions, which have improved this work. We are especially grateful to a referee who pointed out a mistake in an earlier proof of Lemma 3.. We also thank Yanting Li for her help in the numerical simulations. This work was supported by RGC Competitive Earmarked Research Grants HKUST6232/04E HKUST6204/05E. REFERENCES Baxley, R. V., Jr. (995. An Application of Variable Sampling Interval Control Charts, Journal of Quality Technology 27: Box, G. E. P. Luceño, A. (997. Statistical Control by Monitoring Feedback Adjustment, New York: Wiley. Crowder, S. V. (987. A Simple Method for Studying Run Length Distributions of Exponentially Weighted Moving Average Control Charts, Technometrics 29:

24 248 Han Tsung Crowder, S. V. (989. Design of Exponentially Weighted Moving Average Schemes, Journal of Quality Technology 2: Dragalin, V. (993. The Optimality of Generalized CUSUM Procedure in Quickest Detection Problem, Proceedings of Steklov Institute of Mathematics: Statistics Control of Stochastic Processes 202: Dragalin, V. (997. The Design Analysis of 2-CUSUM Procedure, Communications in Statistics Simulation & Computation 26: Durrett, R. (996. Probability Theory Examples, 2nd ed., Belmont: Wadsworth. Gut, A. (988. Stopping Time Rom Walks: Limit Theorems Applications, New York: Springer. Han, D. Tsung, F. G. (2004. A Generalized EWMA Control Chart Its Comparison with the Optimal EWMA, CUSUM GLR Schemes, Annals of Statistics 32: Hawkins, D. M. Olwell, D. H. (998. Cumulative Sum Charts Charting for Quality Improvement, New York: Springer. Jiang, W., Tsui, K. L., Woodall, W. H. (2000. A New SPC Monitoring Method: The ARMA Chart, Technometrics 42: Jones, L. A., Champ, C. W., Rigdon, S. T. (200. The Performance of Exponentially Weighted Moving Average Charts with Estimated Parameters, Technometrics 43: Lai, T. L. (995. Sequential Change-Point Detection in Quality Control Dynamic Systems, Journal of Royal Statistical Society, Series B 57: Lai, T. L. (998. Information Bounds Quick Detection of Parameter Changes in Stochastic Systems, IEEE Transactions on Information Theory 44: Lai, T. L. Shan, J. Z. (999. Efficient Recursive Algorithms for Detection for Abrupt Changes in Signals Control Systems, IEEE Transactions on Automatic Control 44: Lorden, G. (97. Procedures for Reacting to a Change in Distribution, Annals of Mathematical Statistics 42: Lorden, G. Eisenberger, I. (973. Detection of Failure Rate Increases, Technometrics 5: Lucas, J. M. Saccucci, M. S. (990. Exponentially Weighted Moving Average Control Schemes: Properties Enhancements, Technometrics 32: 6. Luceño, A. (999. Average Run Lengths Run Length Probability Distributions for Cuscore Charts to Control Normal Mean, Computational Statistics Data Analysis 32: Mastrangelo, C. M. Brown, E. C. (2000. Shift Detection Properties of Moving Centerline Control Chart Schemes, Journal of Quality Technology 32: Mastrangelo, C. M. Montgomery, D. C. (995. SPC with Correlated Observations for the Chemical Process Industries, Quality Reliability Engineering International : Montgomery, D. C. Mastrangelo, C. M. (99. Some Statistical Process Control Charts Methods for Autocorrelated Data, Journal of Quality Technology 23: Moustakides, G. V. (986. Optimal Stopping Times for Detecting Changes in Distribution, Annals of Statistics 4: Nikiforov, I. (2000. A Suboptimal Quadratic Change Detection Scheme, IEEE Transactions on Information Theory 46: Ramírez, G. J. (998. Monitoring Clean Room Air Using Cuscore Charts, Quality Reliability Engineering International 4: Reynolds, M. R., Jr. (996a. Shewhart EWMA Control Charts Using Variable Sampling Intervals with Sampling at Fixed Times, Journal of Quality Technology 28:

25 Detection Diagnosis of Unknown Abrupt Changes 249 Reynolds, M. R., Jr. (996b. Variable Sampling-Interval Control Charts with Sampling at Fixed Times, IIE Transactions 28: Ritov, Y. (990. Decision Theoretic Optimality of the CUSUM Procedure, Annals of Statistics 8: Shiryaev, A. N. (995. Probability, New York: Springer. Shu, L. J., Apley, D. W., Tsung, F. (2002. Autocorrelated Process Monitoring Using Triggered Cuscore Charts, Quality Reliability Engineering International 8: Siegmund, D. (985. Sequential Analysis: Tests Confidence Intervals, New York: Springer. Siegmund, D. Venkatraman, E. S. (995. Using the Generalized Likelihood Ratio Statistic for Sequential Detection of a Change-Point, Annals of Statistics 23: Sparks, R. S. (2000. CUSUM Charts for Signalling Varying Location Shifts, Journal of Quality Technology 32: Srivastava, M. S. Wu, Y. H. (993. Comparison of EWMA, CUSUM Shiryavev- Robters Procedures for Detecting a Shift in the Mean, Annals of Statistics 2: Willsky, A. S. Jones, H. L. (976. A Generalized Likelihood Ratio Approach to Detection Estimation of Jumps in Linear Systems, IEEE Transactions on Automatic Control 2: 08 2.