Control Charts Based on Alternative Hypotheses

Transcription

1 Control Charts Based on Alternative Hypotheses A. Di Bucchianico, M. Hušková (Prague), P. Klášterecky (Prague), W.R. van Zwet (Leiden) Dortmund, January 11, /48

2 Goals of this talk introduce hypothesis testing framework for control charts in SPC develop monitoring procedures for practical out-of-control situations 2/48

3 Contents of talk Background: statistical process control: control charts change-point problems sequential analysis Testing and control charts Some alternative hypotheses Some thoughts on performance measures of monitoring procedures Likelihood ratio tests Asymptotics for critical values Simulations Future work 3/48

4 Background 1. SPC (Statistical Process Control) background in industry (Shewhart 1924) uses control charts as monitoring tools (detection of out-of-control situations) emphasis on on-line monitoring (Phase II) 2. changepoint analysis (cf. Lai, J. Roy. Stat. Soc. B 57 (1995)) background in mathematical statistics aims at estimation of changepoint main emphasis on retrospective analysis 3. sequential analysis (cf. Lai, Stat. Sinica 11 (2001)) developed in military context (Wald, Wolfowitz 1940 s) initial emphasis on hypothesis testing 4/48

5 SPC: Shewhart charts introduced by Shewhart in 1924 practical tool without theoretical background specific terminology: in-control (common causes), out-of-control (special causes), rational subgroups,... chart signals if summary statistic of ith group is above or below 3σ T variants: X, R, S, MR, attribute control charts,... additions: VSR (Variable Sampling Rate), runs rules (Western Electric 1956), warning zones,... 5/48

6 Shewhart X-chart with control lines. µ + 3σ/ n µ UCL CL µ 3σ/ n LCL 6/48

7 SPC: CUSUM charts introduced in 1954 by Page cumulative sums enable to detect small changes of the mean (< 1.5σ ) recursive practical form: threshold on cumulative sums: Q i = max{0, Q i 1 + X i k)} optimality with respect to ARL proved by Moustakides and Ritov performance quickly deteriorates away from optimal alternative (finetuning of k and decision threshold) additions: FIR (Fast Initial Response) by Lucas monograph Hawkins and Olwell 7/48

8 SPC: EWMA chart based on ideas from Girshick, Rubin, Roberts and Shiryaev (early 1960 s) inspired by Bayesian analysis (prior distribution on changepoint) V i = λ(x i ) + (1 λ)v i 1, V 0 = 0 λ 0: CUSUM, λ = 1: Shewhart practical choice for λ: 0.1 < λ < 0.3 performance nearly as good as CUSUM, but less sensitive to nonnormality decision threshold changes with i 8/48

9 SPC: other charts control charts based on robust statistics combined control charts (e.g., Shewhart-CUSUM) cuscore charts (Box; based on Fisher s efficient score statistics) nonparametric control charts linear rank statistics (Wilcoxon,... ) precedence statistics (Chakraborti and V.d. Laan) regression-type control charts (monitoring linear profiles)... 9/48

10 SPC: Phase I and II Phase I retrospective (usually pilot study of new production process) determination of in-control parameter values Phase II on-line (full scale process) uses in-control parameter values from Phase I Detection performance of control charts in Phase II may heavily deteriorate when using estimated parameters in the control limits (see e.g., Chakraborti, Comm. Stat. Simul. 29 (2000)). Robust estimation of parameters is required in noisy environments (see e.g., Gather et al., Estadistica 53 (2001)). Hawkins et al. (J. Qual. Techn. 35 (2003)) argue that application likelihood ratio methods from changepoint analysis in SPC context makes Phase I/II distinction superfluous. / department of mathematics and computer science 10/48

11 sample X 1,..., X n H 0 : µ = µ 0 12 Neyman-Pearson Lemma H 1 : µ = µ 1 reject H 0 if P H 1 (x 1,..., x n ) P H0 (x 1,..., x n ) > c. This test has maximal power under all tests with the same type I error. Frisén and De Maré, Biometrika 78 (1991), have version of Neyman-Pearson for detection of critical events at given time point. Tests for composite hypotheses H 0 : µ µ 0 against H 1 : µ > µ 0 may be treated similarly if likelihood ratio is monotone. 11/48

12 Wald SPRT (Sequential Probability Ratio Test) sample X 1, X 2,... H 0 : µ = µ 0 against H 1 : µ = µ 1 accept H 0 accept H 1 if if P H1 (x 1,..., x n ) P H0 (x 1,..., x n ) < a P H1 (x 1,..., x n ) P H0 (x 1,..., x n ) > b continue testing if a P H 1 (x 1,..., x n ) P H0 (x 1,..., x n ) b This procedure simultaneously minimizes the ASN s (Average Sample Number) under H 0 and H 1, given P H0 (reject H 0 ) α and P H1 (reject H 0 ) β. 12/48

13 Summary of existing procedures Existing procedures either are not based on optimality criteria or are optimal w.r.t. to ARL (Average Run Length). However, run length distributions are highly skewed (often close to geometric distribution). Performance should be judged on other features of run length distributions (e.g., quantiles). Cf. PSD (Probability of Successful Detection) in medical applications (Frisén, Stat. Medicine 11 (1992) and Frisé and Wessmann, Comm. Stat. Simul. 28 (1999)). In case of active surveillance: predictive value of alarm. It would be nice to reconcile classical hypothesis testing with sequential detection. This would allow to develop optimal monitoring procedures for specific alternative hypotheses. 13/48

14 Examples of alternative hypotheses Usually theoretical studies of control charts focus on persistent changes of the mean (mathematical convenience?!). Examples of other alternative hypotheses (cf. Gitlow et al., Quality Management: Tools and Methods for Improvement, Chapter 8): persistent threshold crossing (sea and river levels) persistent monotone threshold crossing (tool wear, e.g., chisels) persistent shift or drift in variance (wear of bearing) epidemic alternatives (e.g., joint SPC APC scheme: feedback controller removes special cause )... One may also think of monitoring cycles (e.g., business cycles or medical cycles (Frisén)) or profiles (cf. Kim et al., J. Qual. Technology 35 (2003) ). 14/48

15 Our Model Sequential observations X 1,..., X k (k n) n fixed beforehand (batch industry, medical event,... ) X i = µ i + e i, i = 1,..., n, e 1,..., e n are i.i.d. error terms with density f symmetric around 0 µ 1,..., µ n are unknown parameters. Test statistic based on likelihood ratios (details later). Critical value of test statistics based on false alarm rate (details later). 15/48

16 Possible alternative hypotheses Persistent change of mean (µ 0 and δ known) H 0 : µ 1 =... = µ n { µ i = µ 0, i = 1,..., m, H 1 : µ i = µ 0 + δ, i = m + 1,..., n Epidemic alternative (µ 0 and δ known) H 0 : µ 1 =... = µ n µ i = µ 0, i = 1,..., l, H 1 : µ i = µ 0 + δ, i = l + 1,..., m, µ i = µ 0, i = m,..., n 16/48

17 Possible alternative hypotheses: continued Persistent non-monotone threshold crossing (δ known) H 0 : µ i δ, i = 1,..., n, { µ i δ, i = 1,..., m, H 1 : µ i > δ, i = m + 1,..., n, Persistent monotone threshold crossing (δ known) H 0 : µ 1... µ n δ { µ 1... µ m < δ i = 1,..., m H 1 : δ < µ m+1 µ m+2... µ n i = m + 1,..., n Similar alternative hypotheses for the variance are also important (cf. philosophy SPC). 17/48

18 Toy example: Shewhart chart + persistent threshold crossing alternative X i independent with densities f (x µ i ), where f is a symmetric density that is nonincreasing for x > 0. Hypotheses with known δ 0 : against H 0i : E X i = µ i δ 0 H 1i : E X i = µ i > δ 0 The ML-estimators µ i0 and µ i1 based on X i under H 0i and H 1i, respectively, are given by: { { X i if X i δ 0 δ 0 if X i δ 0 µ i0 = µ i1 = δ 0 if X i > δ 0 X i if X i > δ 0 18/48

19 The loglikelihood ratio based on X i thus equals log ( f (X i µ i1 )/f (X i µ 0i ) ) Under normality with common known variance σ 2 this reduces up to a multiplicative constant 1/2 to: Z i (δ 0 ) = (X i δ 0 ) 2 σ 2 sign (X i δ 0 ). Since (t δ 0 ) 2 sign (δ 0 t) is a monotone function of t, we restrict ourselves to the least favourable situation of the null hypothesis H 0 : the restricted null hypothesis H 0. H 0 : µ 1 = µ 2 =... = δ 0. N S = { min{1 k n; Z k (δ 0 ) c} if max 1 k n Z k c if max 1 k n Z k < c. 19/48

20 Critical values Z i Standard choice of 3σ control limits yields in-control ARL of 1/ E H 0 N S = 1 1 H(c), where H is distribution function of Z i. Note that E H 0 N S is not meaningful because there is no good way to incorporate the event {max 1 k n Z k < c}. Alternative 1: jointly considering H(c) n, the probability of no alarm at all, and the conditional ARL: E H 0 (N S N S n) = 1 1 H(c) n(h(c))n 1 H(c) n. Alternative 2: 100α% quantile of the distribution of N S. ( ) 1 H(c) ns(α) = P H 0 max Z i(δ 0 ) c = P H 1 i n S (α) 0 (N S n S (α)) α 20/48

21 Critical values Z i : continued Alternative 2: 100α% quantile of the distribution of N S. The lower bound n S (α) for the 100α% quantile of the distribution of N S satisfies ( ) P H 0 max Z i(δ 0 ) c = P H 1 i n S (α) 0 (N S n S (α)) α In other words, the test H 0 against H 1 based on n S (α) observations has significance level α. Hence, this choice of the critical value can be easily connected to the Neyman-Pearson framework. Thinking in terms of hypothesis testing with level α it will be typically chosen small, while thinking in terms of the median of the run length distribution we think more about α = 1/2. The next slides present tables with numerical values. 21/48

22 quantile of N S n E H 0 (N S N S n) P H 0 (N S > n) 1% 5% 50% 80% 90% Influence of n on the run length distribution: c = 9. 22/48

23 β = 0.8 β = 0.9 β = 1 n 90% 95% 99% 90% 95% 99% P H0 (N S > n) = Critical values for Shewhart charts with prescribed lower bound for quantiles of run length distribution and for prescribed false alarm rate: n s = β n. 23/48

24 Detection performance Z i distribution function H of Z i (δ 0 ) if E X i = δ has the form: ( ) x δ δ 0 σ, x < 0 H(x) = ( x ) δ δ 0 σ, x 0 In the next table, we assume that no false alarm takes place before the changepoint, which gives a good impression of the performance, because the Shewhart chart is memoryless. 24/48

25 H 1 -quantile of N S m N S > m n m c P H1 (N S > n N s > m) 1% 5% 50% 80% 90% Detection performance for Shewhart charts with prescribed false alarm probability 0.05 under jump alternative with δ 1 = δ 0 + σ : E X i = δ 0 + σ for i > m. 25/48

26 Detection performance Z i : uniform alternative Another interesting alternative is that the mean of X i after the changepoint is uniformly distributed on [δ 0, δ 0 + γ σ ] with γ > 0. We then have P H1 (Z i c) = 1 γ σ γ σ 0 P(Z i c µ i = δ 0 +x) dx = 1 γ σ γ σ 0 ( c x ) σ Writing as an integral and interchanging the order of integration, we obtain that b a (u) du = b (b) a (a) + ϕ(b) ϕ(a), where ϕ is the density of the standard normal distribution. Combining everything, we obtain P H1 (Z i c) = c ( c) ( c γ ) ( c γ ) + ϕ( c) ϕ( c γ ). dx 26/48

27 General approach: likelihood ratio tests max 1 m k max θ mi=1 f (x i ; θ) max η ki=m+1 f i (x i ; η) max θ ki=1 f (x i ; θ) Persistent non-monotone threshold crossing and Persistent monotone threshold crossing for normally distributed data with known variance: 1 k Q n = max m<k 2σ 2 (X i δ) 2 sign (X i δ), k = 2,..., n. i=m+1 Standardization necessary to obtain distributional results (cf. critical values). Q kn = 1 n Q k Q kk = 1 k Q k. Q kn may perform poorly for early changes. 27/48

28 Likelihood ratio: windowed versions (Willsky - Jones) Q k = max m<k 1 2σ 2 k i=m+1 (X i δ) 2 sign (X i δ), k = 2,..., n. The above expression (standardized or not) needs to be maximized once or twice with respect to m. Problems: maximization may be time consuming asymptotical distributional results may be hard to obtain Q k,g = 1 G max k G m<k 1 2σ 2 Q simp k,g = 1 1 k G 2σ 2 (X i δ) 2 sign (X i δ). i=k G+1 28/48 k i=m+1 (X i δ) 2 sign (X i δ),

29 Isotonic regression Chang and Fricker (J. Qual. Technology 31 (1999)) : isotonic regression method for Persistent monotone threshold crossing alternative M k = = k k (X i Z i ) 2 (X i Y i ) 2 i=1 i=1 0 for Y k δ k (X i δ) 2 k (X i Y i ) 2 for Y k > δ, i=j i=j where Z 1,..., Z k denotes an isotonic regression restricted to increase up to at most δ Y 1,..., Y k is the corresponding unrestricted isotonic regression J = min{i : Y i > δ}. 29/48

30 Intermezzo: scale alternatives X i = µ + σ i e i, i = 1,..., n, where µ is a known constant, σ i, i = 1,..., n are unknown positive parameters, e 1,..., e n are i.i.d. random variables with N(0, 1) distribution. Hypotheses: H 0 : σ i δ, i = 1,..., against H 1 : there is an m such that σ i δ, i = 1,..., σ i > δ i = m+1,..., n, The estimator σ i0 2 (H 0) of σi 2 under the restriction that σi 2 of the minimization problem: { min log σ σi 2 i δ2 σi 2 (X i µ) 2}. δ 2 is the solution 30/48

31 Scale alternatives: continued σ i0 2 = min((x i µ) 2, δ 2 ) The estimator σ i1 2 of σ i 2 under the restriction that σi 2 σ 2 i1 = max((x i µ) 2, δ 2 ). Hence, the loglikelihood ratio for the i-th observation is ( ( log δ 2 ) (X i µ) (X i µ) 2 δ 2 ) sign δ 2 is ( (Xi µ) 2 ) δ 2 1. q(t) = ( log(t 2 ) 1 + t 2 )sign (t 1), t > 0 is nondecreasing Q σ 2 k = max 0 j k k i= j+1 ( log ( (X i µ) 2 ) δ 2 1+ (X i µ) 2 ) δ 2 sign ((X i µ) 2 /δ 2 1). Clearly, the test statistics Q σ 2 k are similar to the Q k s. 31/48

32 Critical values The most common way to control false alarm in SPC is to use run lengths, in particular the in-control ARL (average run length). Sometimes also the SRL (standard deviation of the run length) is taken into account, or even better one uses quantiles. We reject H 0 as soon as for some k n T k > c n,α, where T k denotes any of the statistics presented in this talk and c N,α is chosen in such a way the significance level is α, i.e. ( ) P H0 max T k > c n,α = α. 1 k n Since the distribution of our statistics do not admit closed-form expression, these critical values have to be obtained by simulations or by asymptotics. 32/48

33 Asymptotics of critical values P H 0 (N C (c) > n C (α)) = ( )1 α for a prechosen α and a prechosen integer n C (α). We write c = c C (α) and rewrite the above relations as ( ) P H 0 max Q k < c C (α) 1 α, 1 k n C (α) so the probability of false alarm based on n C (α) observations is α. ( lim P H n 0 max Q k < c ) ( ) n(var H 1<k n 0 Z 1 (δ 0 )) 1/2 = P sup W (t) W (s) < c 0<s<t<1 where {W (t), t (0, 1)} is a Wiener process. This implies that lim n c C(α) n = c α ( VarH 0 Z 1 (δ 0 ) ) 1/2,, where c α is such that P ( ) sup 0<s<t<1 W (t) W (s) < c α = 1 α. 33/48

34 Asymptotics of critical values continued Hence, c C (α) can be approximated by c α n(varh 0 Z 1 (δ 0 )) 1/2 and as soon as n is large so does c C (α) and the procedure will perform poorly in detecting early changes. Alternative: weighted CUSUM N C,β (c n (β), n) = min{k 1; (k/n) β Q k nc n (β)}, β [0, 1/2], where c n (β) is determined in such a way that ( P H 0 max 1 k n (k/n) β Q k < ) nc n (β) 1 α. Hence, we reject H 0 as soon as there is a k such that Q k n c n (β)(k/n) β. Obviously, choosing β = 0 we have back the original procedure but for β (0, 1/2] larger weights are assigned to smaller k s. / department of mathematics and computer science 34/48

35 Asymptotics of critical values: weighted version lim c n(β) = d α (β) ( Var H n 0 Z 1 (δ 0 ) ) 1/2, β [0, 1/2), where d α (β) is determined as ( P sup 0<s<t<1 1 t β (W (t) W (s)) d α(β) ) = 1 α, where {W (t), t (0, 1)} is a Wiener process. It implies that c n (β)(var H 0 Z 1 (δ 0 )/n) 1/2 can be approximated by d β (α). For β = 1/2 we have for c n (1/2) the approximation d α,n (1/2)(Var H 0 Z 1 (δ 0 )) 1/2, where P ( sup 1 s t n ) 1 (W (t) W (s)) d α,n (1/2) = 1 α. t We can show that lim n c n (β) = O(1), β [0, 1/2), while c n (1/2) = O( log log n). / department of mathematics and computer science 35/48

36 Background of asymptotics I Komlós et al. (Z. Wahr. Verw. Gebiete 32 (1975) and 34 (1976)) Y 1,..., Y n i.i.d. with zero mean, unit variance and E X i 2+ < for some > 0. Then there exists a sequence of Wiener processes W n = {W n (t), t 0}, n n 0, such that, as n, ( ) and P max 1 k n k Y i W n (k) > x i=1 max 1 k n max 1 k n 1 k 1/(2+ ) C 1 nx (2+ ), x > 0 k Y i W n (k) = O P(1) i=1 k X i W n (k) = o P(n 1/(2+ ) ). i=1 36/48

37 Background of asymptotics II Erdös-Darling theorems and P(d 1 (log n) max 1<k n 1 k k Y i y + d 2 (log n)) exp{ exp{ y}}. i=1 P(d 1 (log n) max 1<k n 1 k k Y i y + d 2 (log n)) exp{ 2 exp{ y}}, i=1 where d 1 (t) = 2 log t, t > 1, d 2 (t) = 2 log t + 1 log log t log(π), log t > /48

38 Simulation of critical values (Ba- Sample quantiles ξ α are asymptotically normal with variance α(1 α) f 2 (ξ α ) hadur) We estimated f 2 (ξ α ) by applying a kernel density estimator to a pre-run of reasonable size. Example: for the 90% quantile the standard deviation equals ( ) /2, n because the the density at the quantile approximately equals Hence, if n = 10000, then the standard deviation of this quantile approximately equals /48

39 Performance simulation Simulation performed using R (see version on a Unix platform. Its new built-in procedure isoreg() is much faster in computing the isotonic regression (needed for simulations of the M n statistics) than the PAVA algorithm. Source codes of all procedures we used are available at klaster/compstat04 39/48

40 Simulation improvement Since Var N S E (N S ) 2, so increase in ARL leads to increase in variance of estimates. Variance reduction techniques have been developed for these simulations. Jun and Choi, Comm. Stat. A 22 (1993) present two techniques for CUSUM simulations: 1. hazard controlled estimator (following a suggestion of Ross to use the total hazard of a Markov chain) 2. ratio estimators using a cycle. 40/48

41 Simulation setup n = 100, α = 0.05, G = 0.1n = 10, σ = 1, 1000 repetitions. Two types of changes, each of which has a small and a large version: small version corresponds to change of mean of σ large version corresponds to change of mean of 3σ general change mean is uniformly distributed on [δ r, δ] before changepoint, and uniformly distributed on [δ, δ + r] after the change-point, where r = 1 (small change) or r = 3 (large change), respectively (so the expected mean increases by 1 and 3, respectively). gradual change mean increases on 5 equally spaced intervals before change point from µ to δ and on 5 equally spaced intervals after the change-point from δ to 2δ µ, where δ = 1 and µ = 0.5 (small change) or δ = 3 and µ = 1.5 (large change). 41/48

42 Simulation interpretation Detection delays must be interpreted with care. Suppose change at x and number of observations equals n, then in our simulations the number n x indicates either a detection at time point n or no detection at all Summary statistics of simulation include (run lengths are skewed!): mean standard deviation quantiles 42/48

43 Simulation results: general small change Q kn Q kk Q n,g Q simp n,g M n m mean; sd mean; sd mean; sd mean; sd mean; sd ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; /48

44 Gradual small change Q kn Q kk Q k,g Q simp k,g M k m mean; sd mean; sd mean; sd mean; sd mean; sd ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; /48

45 General large change Q kn Q kk Q k,g Q simp k,g M k m mean; sd mean; sd mean; sd mean; sd mean; sd ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; /48

46 Gradual large change Q kn Q kk Q k,g Q simp k,g M k m mean; sd mean; sd mean; sd mean; sd mean; sd ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; /48

47 Conclusions Chang and Fricker s M k has the smallest mean (both general and gradual case) Q kn has a considerably smaller standard deviation. M k has better low quantiles Q kn has better median and higher quantiles Summary: In most cases Q kn performs better, but M k sometimes detects extremely fast. 47/48

48 Future work other alternative hypotheses (in particular, changes of the variance) composite alternative hypotheses (asymptotical) distributional results combination of procedures application to real data... 48/48