A New Model-Free CuSum Procedure for Autocorrelated Processes

Transcription

1 A New Model-Free CuSum Procedure for Autocorrelated Processes Seong-Hee Kim, Christos Alexopoulos, David Goldsman, and Kwok-Leung Tsui School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, U.S.A. February 25, 2006 Abstract We present a new CuSum chart for monitoring shifts in the mean of autocorrelated data. The monitoring statistic is the plain cumulative sum of differences between the observations and the target value, and the control limits are derived based on the asymptotic behavior of the monitoring statistic. Although our method requires the asymptotic variance constant that is the sum of covariances of all lags prior to monitoring the underlying process, the constant can be estimated by a variety of non-parametric techniques that are popular in the simulation literature. Therefore, the proposed method is completely model-free. The performance of our new procedure is compared with that of other competitive model-free methods using examples based on various stationary processes with normal or nonnormal marginals. Key Words: Statistical Process Control, CuSum Chart, Autocorrelated Data 1

2 Biographical Note Seong-Hee Kim is an assistant professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. She currently serves on the editorial board of the Simulation Department of IIE Transactions. She is also an associate editor in the simulation area of Operations Research and OR/simulation area of The American Statistician. She is a member of IIE and INFORMS. Her address is skim@isye.gatech.edu. Christose Alexopoulos is an associate professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. He currently serves as Simulation Departmental Editor of IIE Transactions. He a member of INFORMS and an active participant of the Winter Simulation Conference, having been Proceedings Co-Editor in He is also on the editorial board of Networks. His address is christos@isye.gatech.edu. David Goldsman is a Professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. His research interests lie in simulation output analysis and statistical ranking and selection methods. He is a member of INFORMS and IIE, and is currently serving on the Board of the Winter Simulation Conference. His address is sman@gatech.edu. Kwok-Leung Tsui is a Professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. He currently serves as Quality and Reliability Departmental Editor of IIE Transactions. He has also held various offices in the INFORMS Quality, Statistics, and Reliability Section and Data Mining section. He is a Fellow of ASA, and member of ASQ and INFORMS. His address is ktsui@isye.gatech.edu. 2

3 1. Introduction For a control chart, Type I error corresponds to the probability that the chart gives false alarms when the underlying quality of the output remains unchanged from the in-control status, and Type II error is a probability that the chart does not detect a shift when it actually occurs. These Type I and II errors are the main performance measures of interest. The Type I error can be measured through the in-control average run length (ARL 0 ), which is the expected number of observations taken before action is (incorrectly) taken when the process is in control. An analogous measure corresponding to the Type II error is an out-ofcontrol average run length (ARL 1 ). The control limits of an SPC chart are designed so that the actual ARL 0 of the chart is equal to the target ARL 0. Some charts for independent and identically distributed (IID) normal data such as the Shewhart and Tabular CuSum charts (see Chapter 8 of Montgomery 2000 for details concerning those procedures) determine such control limits analytically. Unfortunately, this is a much more difficult task for autocorrelated data. In recent years, considerable effort has been spent on developing SPC methods for autocorrelated data, such as the exponentially weighted moving average (EWMA) charts (Montgomery and Mastrangelo 1991; Tseng and Adams 1994; Lu and Reynolds 1999; and Jiang, Tsui, and Woodall 2000), CuSum charts (Runger, Willemain, and Prabhu 1995; VanBrackle and Reynolds 1997; Timmer, Pignatiello, and Longnecker 1998; and Lu and Reynolds 2001), and residual charts (Alwan and Roberts 1988; and Jiang et al. 2002). These charts have been developed assuming either that the underlying process follows a specific model or that certain characteristics of the underlying process, such as the autocorrelation structure, are known. Then the control charts are constructed from raw observations or residuals using traditional hypothesis tests. We call these model-based or semi-model-based charts. For some charts such as the ARMA chart (Jiang, Tsui, and Woodall 2000) and the Proportional 3

4 Integrated Derivative (PID) chart (Jiang et al. 2002), the model or correlation structure is not needed for computing the monitoring statistics; it is needed only for designing the control chart, i.e., determining the control limits and chart parameters. Overall, the model-based or semi-model-based charts suffer from the limitation that the underlying model may be inappropriate. In this case, the estimation of important parameters that determine control limits may not be valid and may affect negatively the performance of such procedures. Alternatively, model-free methods for autocorrelated processes (e.g., Runger and Willemain 1995) use sequences of batch means (i.e., sample averages of contiguous, but correlated, observations) as the monitoring statistics. However, these methods sometimes delay legitimate out-of-control alarms for highly correlated data and/or large shifts. On the other hand, Johnson and Bagshaw (1974) present a model-free CuSum method that uses raw observations instead of batch means. Ben-Gal et al. (2003) introduce a context-based SPC chart for state-dependent discrete-valued data generated by a finite-memory source. The application of this method is limited to processes with a finite state and, therefore, their procedure is not of our interest in this paper. In this paper we consider a CuSum process as our monitoring statistic that is a bit different than that of Johnson and Bagshaw (1974), and we approximate this CuSum process by a Brownian motion process. By exploiting the known properties of Brownian motion, we derive a new model-free CuSum chart called the MFC chart. The control limits of the MFC chart that are usually straight lines can be done analytically. More specifically, when a monitoring system is in control, it can be shown that under mild conditions the plain cumulative sum of autocorrelated data converges to a Brownian motion process as the number of samples goes to infinity. Then, we can approximate the first-passage time of the monitoring statistic to a constant threshold which is ARL 0 by that of a corresponding Brownian motion process. Although the proposed SPC procedure requires the asymptotic variance constant which 4

5 is the sum of covariances of all lags, the procedure is completely model-free including the design of control limits and chart parameters with the help of non-parametric variance estimation techniques popular in the simulation community (see Alexopoulos, Goldsman, and Serfozo 2006 for the details of various model-free estimators for the asymptotic variance constant of autocorrelated processes). The MFC chart can be used with raw observations or batch means of any size so we can avoid the use of large batches. Our procedure also provides a convenient way to compute control limits like the Shewhart chart does whereas most existing methods require either simulation or prior studies about the structure of the underlying process. The remaining sections are organized as follows. Section 2 contains notation and assumptions. Section 3 presents a new model-free SPC chart for autocorrelated data, and Section 4 contains experimental results that compare the new procedure against existing competitive procedures based on stationary processes with normal or nonnormal marginals. Section 5 contains conclusions. 2. Background In this section we set up notation and assumptions on the monitoring process. To begin with, suppose that the discrete-time process Y = {Y j : j = 1, 2,...} is stationary with mean µ = E[Y j ]. The sample mean of the first n observations is denoted by Ȳ (n), and the standardized CuSum, C(t), is defined as C(t) nt j=1 Y j ntµ v, 0 t 1, (1) n where is the floor (greatest integer) function and v 2 is the asymptotic variance constant, defined as v 2 lim n n Var(Ȳ (n)). The random function C(t) is an element of the Skorohod space D[0, 1], i.e., the space of 5

6 functions on [0, 1] that are right-continuous and have left-hand limits (Chapter 3 of Billingsley 1968). To establish our results, we restrict attention to processes that satisfy the following assumption, related to a Functional Central Limit Theorem (FCLT) (see Billingsley 1968, Chapter 4). Assumption 1 (FCLT) There exist finite real constants µ and v 2 > 0 such that the probability distribution of C(t) over D[0, 1] converges to that of a standard Brownian motion process, W( ), for t [0, 1], as n. Formally, C(t) = W(t), 0 t 1, as n, where = denotes weak convergence. Further, we assume that for every t [0, 1], the family of random variables {C 2 (t) : n = 1, 2,...} is uniformly integrable (see Billingsley 1968, Chapter 5). One set of sufficient conditions for the FCLT is that Y is stationary and φ-mixing with appropriate mixing constants (see also Glynn and Iglehart 1990). Roughly speaking, in a φ-mixing process the distant future is nearly independent of the past and present (see p. 166 of Billingsley 1968 for the exact definition). 3. Design of the MFC Chart In this section we present the MFC chart and show how to determine its control limits. Let µ 0 represent the mean of Y when it is in control. Also, let S(n) = n i=1 (Y i µ 0 ) denote the plain cumulative sum of differences between the observations and a target value. Then, the MFC chart is as follows: 6

7 MFC: Model-Free CuSum Procedure Setup: Choose a target two-sided ARL 0 and determine the control limit b using one of the alternatives in (4) given below. Chart: Continue sampling until S(n) b. If n is sufficiently large and the process is in control, then it follows from (1) and the continuous mapping theorem (Billingsley 1968) that S(n) D = v n C(1) D v n W(1) D = v W(n), (2) where = D denotes equivalence in distribution and D denotes approximate equivalence in distribution. Moreover, by an argument similar to that for Proposition 3.2 of Kim, Nelson, and Wilson (2005), we see that (3) follows from (2) and the continuous mapping theorem as n goes to infinity: Pr{ S(n) b} Pr{ vw(n) b}. (3) The process W(t) for t 0 has the first-passage time to the threshold b > 0 defined by T W = inf { t : t 0 and vw(t) b }. The process S(n) has the first-passage time to the threshold b > 0 defined by T S = min { n : n = 1, 2,... and S(n) b }. Then by (3), the expected first-passage time of the process S(n) to b can be approximated by that of W(n) to b/v. That is, E(T S ) E(T W ). There are a number of approximations for the expected first-passage time of W(t) to a constant threshold. Darling and Siegert (1953) show that E(T W ) b 2 /v 2. On the other hand, Siegmund (1985, p. 27) s first-passage time for IID normal data can be easily extended to the case of a Brownian motion with variance constant v 2. Then we get ( ) 2 b v E(T W ). v 7

8 Therefore, one can compute b as follows: { v ARL b = 0 by Darling and Siegert (1953) v( ARL ) by Siegmund (1985). (4) Since the asymptotic variance constant v 2 is usually unknown, it needs to be estimated first from existing past data before the monitoring starts. There are many non-parametric techniques that are popular in the simulation literature to estimate the asymptotic variance constant. These include the methods of nonoverlapping batch means (Chien, Goldmsan, and Melamed 1997), overlapping batch means (Meketon and Schmeiser 1984, Damerdji 1994, Sargent, Kang, and Goldsman 1992), and standardized time series (Schruben 1983). Goldsman, Meketon, and Schruben (1990) and Song and Schmeiser (1995) compare these estimators based on bias and mean-squared error. It turns out that these estimators only require a modest amount of data to obtain a relatively accurate and precise estimate for v 2. A number of new variance estimators that reduce bias and variance even more have been proposed recently. Some of them cut the variance 50% or more. See Alexopoulos et al. (2005ab) and Goldsman et al. (2005) for those recent estimators for v Experimental Evaluation In this section, we compare the performance of various model-free SPC procedures designed for autocorrelated data. 4.1 Testing Processes The comparisons are based on a stationary autoregressive process of order one (AR(1)), Y j = µ + φ(y j 1 µ) + ɛ j, for j = 1, 2,..., (5) where ɛ IID j N(0, σɛ 2 ) and 1 < φ < 1 so that Y j have mean µ and variance σy 2 = σɛ 2 /(1 φ 2 ). To ensure (5) starts from the stationary state, we set Y 0 = N(µ, σ 2 Y ). If σ 2 ɛ = 1 φ 2, the marginal variance of Y j is σ 2 Y = 1. 8

9 The lag-l covariance for the AR(1) process is Cov l = σ 2 Y φ l = σ 2 ɛ φ l /(1 φ 2 ) for l = 0, ±1, ±2,... ; (6) and the asymptotic variance constant v 2 is v 2 = σ 2 Y ( ) 1 + φ = 1 φ σ 2 ɛ (1 φ) 2. (7) We also consider a stationary process whose marginal is nonnormal. An exponential autoregressive process of order one (EAR(1)) is defined as Y j = { (µ 1/λ) + φ[yj 1 (µ 1/λ)], w.p. φ, (µ 1/λ) + φ[y j 1 (µ 1/λ)] + ɛ i, w.p. 1 φ, } for j = 1, 2,..., (8) where 0 < φ < 1 and ɛ j are IID exponential with mean 1/λ = σ Y so that Y j have mean µ and variance σy 2 = 1/λ 2. To ensure the stationarity of the EAR(1) process, we sample Y 0 from an exponential distribution with mean 1/λ. The covariance function and v 2 have the same forms as (6) and (7) of the AR(1) process, respectively. However, the marginal of the EAR(1) process is not normal anymore. Steiger and Wilson (2001) give more details of the AR(1) and EAR(1) processes. 4.2 Competitive Procedures We compare MFC with two model-free procedures due to Runger and Willemain (1995) and Johnson and Bagshaw (1974). Runger and Willemain (1995) present charting methods based on weighted batch means (a model-based approach) and unweighted batch means (UBMs) (a model-free approach), the latter of which we consider in this paper Procedure of Johnson and Bagshaw (1974) Johnson and Bagshaw (J&B) (1974) and Bagshaw and Johnson (1975) used a similar approach to MFC involving an approximate Brownian motion process to derive a CuSum chart for autocorrelated data. Their procedure is as follows: 9

10 The J&B Two-Sided Chart: Define S + n max(s + n 1 + Y n µ 0, 0) and S n min(s n 1 + Y n µ 0, 0), for n 1, with S + 0 = 0 and S 0 = 0. Choose the target two-sided ARL 0, and set h = v 2ARL 0. (9) Give an out-of-control signal when S + n > h or S n < h. Though MFC is very similar to the J&B chart, our procedure is different for the following reasons: The J&B chart adjusts the CuSum to zero whenever it becomes negative. We observe the plain CuSum of the differences between the observations and the target without any adjustment. For two-sided tests, the J&B chart combines two one-sided charts. The MFC chart is a single two-sided chart. The control limits of the J&B chart are different Control Charts Based on Unweighted Batch Means Runger and Willemain (R&W) (1995) propose a model-free method that uses a sequence of UBMs with the idea that UBMs are roughly uncorrelated for sufficiently large batch sizes. They apply the usual charting techniques say, classical Shewhart and Tabular CuSum charts for IID normal data on the UBMs as described below: R&W Shewhart Chart: Find a batch size m such that the lag-one autocorrelation of the batch means is approximately 0.1. Choose a target ARL 0 and find z ON such that m 1 Φ(z ON ) + Φ( z ON ) = ARL 0. Then give an out-of-control signal if UBM i z ON σ UBM, where UBM i is the ith UBM and σ 2 UBM is the marginal variance of the UBMs (which is assumed to be known). 10

11 R&W One-Sided CuSum Chart: Find a batch size m that gives approximately a 0.1 lag-one autocorrelation of batch means. Form the UBMs of size m. Pick the reference value k and ARL 0. Determine h that gives the target ARL 0 using approximations for IID data as in Brook and Evans (1972) or Siegmund (1985). Then set K = kσ UBM and H = hσ UBM. The disadvantage of the R&W approach is that the charts may have to wait a long time to accumulate enough observations to build batches especially in the presence of strong autocorrelation. This can unnecessarily delay legitimate out-of-control alarms, particularly when the shift is large. Thus, for highly autocorrelated process data, it may be desirable to have an SPC method that is model-free and uses small batches or even single observations. Table 1: Theoretical and experimental ARLs in terms of raw observations for the R&W Shewhart chart based on UBMs for an AR(1) process with φ = 0.9 and σy 2 = 1. The estimates are based on 5000 experiments. Shift in Mean (Multiple of σ Y ) m = 58 Theoretical Experimental Runger and Willemain (1995) state that the choice of batch size m such that the lagone autocorrelation is approximately 0.1 will result in UBMs behaving like IID data, and therefore one can apply the classical charts developed for IID data on the UBMs. This is true for the R&W Shewhart chart, as shown in Table 1, in which we compare the theoretical and experimental ARLs for an AR(1) process with φ = 0.9, σ 2 Y = 1, and m = 58 when the shift varies from 0 to 4σ Y. The choice of m = 58 ensures a lag-one autocorrelation of UBMs of approximately

12 Table 2: Theoretical and experimental ARLs in terms of raw observations for the R&W CuSum chart based on UBMs for an AR(1) process with φ = 0.25, σ 2 e = 1, m = 4, k = 0.5/σ UBM, and h = The estimates are based on 5000 experiments. (Under these parameter settings, σ 2 UBM = ) Shift in Mean Theoretical Experimental (Multiple of σ e ) Although a batch size that brings down a lag-one autocorrelation of batch means approximately 0.1 works well for the Shewhart chart, it is questionable if such batch size would work well for a CuSum chart. Unfortunately, CuSum charts are known to be more sensitive to data dependencies than Shewhart charts, and small amounts of positive autocorrelation reduce ARL 0 significantly as Rowlands and Wetherill (1991) pointed out. Our experimental results in Table 2 clearly show that this is also the case for a CuSum chart with UBMs. For the comparison with the results of Runger and Willemain (1995), we set k = 0.5/σ UBM and target one-sided ARL 0 = 20,000 (so that the two-sided ARL 0 = 10,000), which are their parameter settings. For an AR(1) process with φ = 0.25, the choice of m = 4 gives a 0.1 lag-one autocorrelation, and the approximation of Brook and Evans (1972) gives h = for the one-sided target ARL 0 = 20,000. Table 2 shows theoretical and experimental results for CuSum charts based on UBMs in terms of raw observations for the AR(1) process with φ = 0.25, σɛ 2 = 1, m = 4, k = 0.5/σ UBM, and h = In Table 2, one can notice that the actual ARL 0 is substantially smaller than the theoretical value we expect from the approximation of Brook and Evans (1972); hence this will cause more false alarms than anticipated. 12

13 These findings imply that we need to use a much larger batch size for CuSum charts than for the Shewhart chart. An alternative is to search for h via simulation so that ARL 0 will be approximately equal to a target ARL 0 for a given m. These remedies are neither desirable nor convenient. Therefore, we will not consider CuSum charts based on UBMs for formal comparisons with our procedure. 4.3 Comparison In this subsection, we discuss experimental results AR(1) Processes For the AR(1) process, the marginal variance of Y i is set to σy 2 = 1; therefore, σɛ 2 = 1 φ 2 and v 2 = (1 + φ)/(1 φ) for the AR(1) process. The shift varies over 0, 0.25, 0.5, 0.75, 1, 1.5, 2, 2.5, 3, and 4 in multiples of σ Y, and the coefficient φ is set to 0, 0.25, 0.5, 0.9, 0.95, and The experimental results for the AR(1) process are based on 5000 independent experiments. For R&W, we consider two different values for the batch size m: a batch size that yields a lag-one autocorrelation of approximately 0.1 (m 1 ) and a batch size that minimizes the mean-squared error (m ) of the nonoverlapping batch means estimator (Chien, Goldsman, Melamed 1997). The asymptotically optimal batch size m for the AR(1) process was derived by Carlstein (1986): { } 2/3 2 φ m = n 1/3. 1 φ 2 Since the target ARL 0 for a shift of zero is 10,000, we use n = 10,000 in the above equation to compute m. For MFC, we can determine the control limits using the alternatives in (4). We tested the performance of MFC with both approximation methods and found that they use about the same number of raw observations for low to medium correlation, say 0 φ

14 However, Siegmund s approximation spends slightly fewer observations for high correlation cases. Therefore, we only present the results of MFC with Siegmund s approximation in the current paper. MFC vs. J&B: The accuracy of approximations (4) and (9) for MFC and J&B is quite good since the control charts determined by these approximations gave an actual ARL 0 close to 10,000 for all configurations. But for high correlations, both approximations yield control charts that are somewhat conservative, i.e., ARL 0 s that are a bit larger than the target ARL 0. For all configurations we tested, MFC is more effective in detecting any sizes of shifts and spends approximately 30% fewer observations than J&B. MFC vs. the R&W Shewhart chart: It is well known that the CuSum chart is very effective in detecting small shifts, but not as effective as the Shewhart chart in detecting large shifts for IID data. The case of φ = 0 in Table 3 compares the two CuSum-type charts (J&B and MFC) versus the Shewhart-type chart of R&W for IID data. The J&B and MFC worked better only up to a shift of 1.5σ Y, while R&W was more effective in detecting larger shifts. We observe the same tendency for small-to-medium correlation coefficients φ. When φ is 0.25 or 0.5 with a batch size that gives a lag-one autocorrelation of about 0.1, J&B and MFC detect shifts up to about σ Y more quickly than R&W; but for a shift larger than σ Y, R&W is again more effective. Table 4 tells the same story for large correlation coefficients φ with a batch size that results in lag-one autocorrelation of about 0.1. The effectiveness of R&W in cases of a large shift disappears when a large batch size is required; in such cases, R&W spends exactly one batch for a large shift. This will obviously delay legitimate out-of-control alarms; and the upshot is that J&B and MFC will be more effective for a very large shift. This problem is well demonstrated 14

15 Table 3: Two-sided ARLs in terms of raw observations for an AR(1) process with small or medium φ and σ 2 Y = 1. A number in a box represents the smallest ARL 1 for each shift. φ Shift J&B MFC R&W (µ µ 0 )/σ Y m 1 = m 1 = 4 m = m 1 = 8 m =

16 Table 4: Two-sided ARLs in terms of raw observations for an AR(1) process with large φ and σ 2 Y = 1. A number in a box represents the smallest ARL 1 for each shift. φ Shift J&B MFC R&W (µ µ 0 )/σ Y m 1 = 58 m = m 1 = 118 m = m 1 = 596 m =

17 in Table 4 for φ = 0.99, where MFC outperformed R&W for a shift of 4σ Y EAR(1) Processes The marginal variance of Y i of the EAR(1) process is set to σ 2 Y = 1. Then, λ = 1 and v 2 = (1 + φ)/(1 φ) for the EAR(1) process. The shift varies over 0, 0.25, 0.5, 0.75, 1, 1.5, 2, 2.5, 3, and 4 in multiples of σ Y, and the coefficient φ is set to 0.25, 0.5, 0.9, and The experimental results for the EAR(1) process are based on 5000 independent experiments. For R&W, we need to determine a batch size first. Since the EAR(1) process has the same covariance function as the AR(1) process, lag-one autocorrelation of batch means will be the same for the AR(1) and EAR(1) processes for a given batch size. Therefore, one can easily imagine that if a batch size gives lag-one autocorrelation of batch means below 0.1 for an AR(1) process, then the batch size should do for a corresponding EAR(1) process. However, such a batch size would not guarantee that the R&W chart which is basically the Shewhart charts will work well for the EAR(1) process as (i) the marginal of Y i is not normal but exponential and (ii) the Shewhart chart is very sensitive to normality (Stoumbos and Reynolds 2000). Unfortunately, Table 5 show that for the R&W chart, neither m 1 nor m works well for EAR(1) processes. For example, for an EAR(1) process with φ = 0.25, the batch size of m = 15 is indeed large enough to get a lag-one autocorrelation below 0.1 for batch means. However, the size of 15 is not large enough to achieve normality of batch means and the Shewhart chart fails. More specifically, R&W gives estimated ARL 0 = 3548 which is seriously smaller than the target value 10, 000, resulting far more false alarms. To find an appropriate batch size for the R&W chart, we had to increase the batch size until estimated ARL 0 is close to the target value. By trial-and-error experimentation, we found that the batch sizes m = 140, 150, 180, and 230 would be respectively required for EAR(1) processes with φ = 0.25, 0.5, 0.7, and 0.9 to achieve a lag-one correlation of at most 17

18 Table 5: Two-sided ARLs in terms of number of raw observations for an EAR(1) process with σy 2 = 1 φ Shift J&B MFC R&W (µ µ 0 )/σ Y m = 15 m = m = 27 m = m = 43 m = m = 97 m =

19 0.1 for the batch means. As shown in Table 5, both the J&B and MFC charts deliver estimated ARL 0 close to the target value. The MFC chart exhibits uniform superiority over the J&B chart for EAR(1) processes as well as for AR(1) processes. Due to the large batch size, R&W does better than the MFC chart for small shift sizes ranging from 0.25 to 1.0. But for large shift sizes the R&W chart starts taking one full batch, and the MFC chart detects the shift faster than the R&W chart. However, the actual amount of experimentation of the R&W chart greatly exceeded that of the other procedures due to the trial-and-error search for an appropriate batch size. Also, such a task that requires estimating ARL 0 is not practically possible. 5. Conclusion We presented the MFC chart, a new CuSum chart for autocorrelated data. The new chart is completely model-free and provides a simple way to determine the control limits. Also, it allows for the use of a small batch size or even raw observations while most other procedures require knowledge of the model or autocorrelation structure of the underlying process either in computing the charting statistic or in determining the control limits. In terms of chart performance, our experiments demonstrate that MFC is competitive with existing modelfree control charts. While many CuSum-type charts require an a priori study to determine control limits, both J&W and MFC determine approximate control limits analytically when the variance constant is known. An estimate for the variance constant should be obtained from a training data set using model-free variance estimation techniques. For a process with normal marginal, the MFC chart seems to work better than R&W for small shifts when a batch size that gives a lag-one autocorrelation of at most 0.1 for batch means is used. This is expected since R&W is a Shewhart-type chart that is generally more effective than a CUSUM-type chart in detecting large shifts for IID normal observations. 19

20 R&W works efficiently with a good choice of a batch size and is very sensitive to normality of batch means. A good batch size should ensure both independence and normality of batch means. Unfortunately, such a batch size is often hard to determine in practice. It is shown that a batch size that gives a lag-one autocorrelation of at most 0.1 for batch means is not large enough to ensure normality, and a much larger batch size is likely to be required; this will hinder the performance of R&W, especially for large shifts. The MFC chart uniformly defeats the J&W chart and has potential to be conveniently implementable in practice. However, it should be further studied the impact of using estimated variance rather than a known variance parameter. If the number of training data points is large enough to get a very accurate variance estimate, the MFC chart probably works as advertised in this paper. The real interesting question is how the performance of the MFC chart will be affected when the number of training data points is not large but not too small if too small, there is no hope and how to overcome performance degradation if there is any. This is a topic of ongoing research. Acknowledgement The authors appreciate a number of valuable comments from James R. Wilson. 20

21 References Alexopoulos, C., Argon, N. T., Goldsman, D., Tokol, G., and Wilson, J. R. 2005a. Overlapping Variance Estimators for Simulation: I. Theory. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA. Alexopoulos, C., Argon, N. T., Goldsman, D., Steiger, N. M., Tokol, G., and Wilson, J. R. 2005b. Overlapping Variance Estimators for Simulation: II. Implementation and Evaluation. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA. Alexopoulos, C., Goldsman, D. and Serfozo, R.F. (2006) Stationary processes: Statistical estimation, in The Encyclopedia of Statistical Sciences, 2nd edn., Balakrishnan, N., Read, C. and Vidakovic, B. (eds.), Wiley, New York, NY. Alwan, L. C. and H. V. Roberts Time-series modeling for statistical process control. Journal of Business and Economic Statistics 6: Bagshaw, M. and R. A. Johnson The effect of serial correlation on the performance of CuSum tests II. Technometrics 17: Ben-Gal, I., Morag, G. and Shmilovici, A. (2003) Context-based statistical process control: A monitoring procedure for state-dependent processes. Technometrics 45(4), Billingsley, P Convergence of Probability Measures. John Wiley & Sons, New York. Brook, D. and D. A. Evans An approach to the probability distribution of CuSum run length. Biometrika 59: Carlstein, E The use of subseries for estimating the variance of a general statistic from a stationary sequence. Annals of Statistics 14: Chien, C., D. Goldsman, and B. Melamed Large-sample results for batch 21

22 means. Management Science 43: Damerdji, H Strong consistency of the variance estimator in steady-state simulation output analysis. Mathematics of Operations Research 19: Darling, D. A. and A. J. F. Siegert The first passage problem for a continuous Markov process. Annals of Mathematical Statistics 24: Glynn, P. W. and D. L. Iglehart Simulation output analysis using standardized time series. Mathematics of Operations Research 15:1 16. Goldsman, D., M. S. Meketon, and L. W. Schruben Properties of standardized time series weighted area variance estimators. Management Science 36: Goldsman, D., K. Kang, S.-H. Kim, A. F. Seila, and G. Tokol Combining Standardized Time Series Area and Cramer-von Mises Variance Estimators. Technical Report, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA. Jiang, W., K.-L. Tsui, and W. Woodall A new SPC monitoring method: The ARMA chart. Technometrics 42: Jiang, W., H. Wu, F. Tsung, V. N. Nair, and K.-L. Tsui Proportional integral derivative control charts for process monitoring. Technometrics 44: Johnson, R. A. and M. Bagshaw The effect of serial correlation on the performance of CuSum tests. Technometrics 16: Kim, S.-H., B. L. Nelson, and J. R. Wilson Some almost-sure convergence properties useful in sequential analysis. Sequential Analysis, forthcoming. Law, A. M. and D. Kelton Simulation Modeling and Analysis, 3rd ed. McGraw- Hill, New York. Lu, C.-W. and M. R. Reynolds EWMA control charts for monitoring the mean of autocorrelated processes. Journal of Quality Technology 31: Lu, C.-W. and M. R. Reynolds CuSum charts for monitoring an autocorrelated 22

23 process. Journal of Quality Technology 33: Meketon, M. S. and B. Schmeiser Overlapping batch means: Something for nothing? In Proceedings of the 1984 Winter Simulation Conference, , eds. S. Sheppard, U. Pooch, and D. Pedgen. Institute of Electrical and Electronics Engineers, Piscataway, NJ. Montgomery, D. C Introduction to Statistical Quality Control, 4th ed. John Wiley & Sons, New York. Montgomery, D. C. and C. M. Mastrangelo Some statistical process control methods for autocorrelated data. Journal of Quality Technology 23: Rowlands, R. J. and G. B. Wetherill Quality control. In Handbook of Sequential Analysis, , eds. B. K. Ghosh and P. K. Sen. Marcel Dekker, New York. Runger, G. C. and T. R. Willemain Model-based and model-free control of autocorrelated processes. Journal of Quality Technology 27: Runger, G. C., T. R. Willemain, and S. Prabhu Average run lengths for CUSUM control charts applied to residuals. Communication in Statistics Theory and Methods 24: Sargent, R. G., K. Kang and D. Goldsman An investigation of finite-sample behavior of confidence interval estimators. Operations Research, 40: Schruben, L.W Confidence interval estimation using standardized time series. Operations Research 31: Stoumbos, Z. G. and M. R. Reynolds, Jr Robustness to non-normality and autocorrelation of individuals control charts. Journal of Statistical Computation and Simulation 66: Siegmund, D Sequential Analysis: Tests and Confidence Intervals. Springer-Verlag, New York. 23

24 Song, W. T. and B. W. Schmeiser Optimal mean-squared-error batch sizes. Management Science 41: Steiger, N. M. and J. R. Wilson Convergence properties of the batch means method for simulation output analysis. INFORMS Journal on Computing 13: Timmer, D. H., J. Pignatiello, Jr., and M. Longnecker The development and evaluation of CUSUM-based control charts for an AR(1) process. IIE Transactions 30: Tseng, S. and B. M. Adams Monitoring autocorrelated processes with an exponentially weighted moving average forecast. Journal of Statistical Computation and Simulation 50: VanBrackle, L. N. and M. R. Reynolds, Jr EWMA and CUSUM control charts in the presence of correlation. Commnications in Statistics Simulation and Computation 26: