Re-weighted Robust Control Charts for Individual Observations

Transcription

1 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 426 Re-weighted Robust Control Charts for Individual Observations Mandana Mohammadi 1, Habshah Midi 1,2 and Jayanthi Arasan 1,2 1 Laboratory of Applied and Computational Statistics, Institute for Mathematical Research, University Putra Malaysia,43400 Serdang, Selangor, Malaysia mandana@inspem.upm.edu.my, 2 Department of Mathematics, Faculty of Science, Universiti Putra Malaysia, UPM Serdang, Selangor, Malaysia Abstract. Control chart is the most valuable tool in statistical process control (SPC) and it is used to monitor the changes in a process. It presents a graphical display of process stability or instability over time. This enable us to instantly understand, whether the process is under control or not. Hotelling s T 2 chart is one of the most popular control charts for monitoring independently and identically distributed random vectors. In the computation of Hotelling s T 2, the classical mean and covariance are utilized, hence the results can be heavily influenced by outliers. In order to decrease the impact of outliers we propose to use the re-weighted robust estimator instead of the classical estimator in the computation of Hotelling s T 2. In this paper we construct a new statistic by substituting the classical estimators in Hotelling s T 2 by the reweighted MCD. Keywords: Hotelling s T 2 chart; Breakdown Point; Re-weighted Minimum Covariance Determinant Estimators; Multivariate Statistical Process Control. 1 Introduction The quality control and process management aim at creating computationally efficient algorithm to investigate and detect the signals. Many manufacturing and service businesses utilize univariate statistical control charts to monitor the performance of their processes. Recently, the monitoring production process has become more complex due to existence of more than one measurement process to check (Yang and Trewn, 2004 [21]). It is very difficult to determine the main cause of faults if multiple process variables exhibit signals or process deviations simultaneously. Under such condition the multivariate statistical process control (MSPC) methods overcome these limitations by monitoring the interactions of several process variables at the same time. In recent years, several multivariate statistical process control techniques have been proposed to analyze and monitor multivariate data (Sullivan and Woodall, 1996 [16]; Tracy, Young and Mason, 1992 [18]; Vargas, 2003 [20]). With the concept of multivariate quality control charts, it is possible to have well defined control limits, while taking into consideration the correlation between the variables. In literature, construction of control charts are carried out in two distinct phases. In Phase I operation, Hotelling s T 2 chart is often used to purge

2 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 427 outliers whereas signal detection is done in the Phase II operation. MSPC can be traced back to Hotelling s T 2 method (Hotelling, 1931 [6]) which is one of the most popular control charts for monitoring independently and identically distributed random vectors. In the computation of Hotelling s T 2, the classical mean and covariance are utilized, hence the results can be heavily influenced by outliers. This chart detects many types of out of control signals, but it is not sensitive to small shifts in the mean vector. In this paper, in order to decrease the impact of outliers we propose the use of re-weighted robust estimator instead of the classical estimator in the computation of Hotelling s T 2. This paper is organized as follows. In the next Section we give brief description of the Hotelling T 2. In Section 3 we introduce the new robust multivariate control charts, and in section 4 we estimate the quantile of robust T 2 via a simulation study. Following that, section 5 shows the comparison of the new robust control chart and experimental results. Finally, section 6 concludes the paper. 2 Hotelling T 2 Harold Hotelling (1947) [7], used the multivariate procedures to analyze bombsight data. In his earlier paper (1931) [6], he introduced the generalized Student t statistic and control procedure based on a new charting statistic. This statistic is defined as a generalized distance form a p dimensional sample point X = (x 1, x 2..., x p ) to its sample mean. In his honor the statistic was named as Hotelling s T 2. The T 2 statistic may be computed using a single observation from p components, or it may be computed using the mean from a sample of size n. In this paper, a subgroup of size 1 (i.e., a single observation) will be assumed for the T 2 computations. The T 2 statistic for a p 1 multivariate normal vector, X = (x 1, x 2..., x p ) is defined as, T 2 = (X i X) C 1 (X i X), (1) where X = 1 ni=1 X n i and C = 1 ni=1 (X n 1 i X)(X i X) are the common estimators of the mean vector and covariance matrix obtained from the historical data set. Usually, we assume that the X i s are independent multivariate normal, (MV N P (µ, Σ)) with mean µ and covariance matrix Σ. We compare the T 2 with the control limits to detect any possible signal, for each individual observation. In the computation of Hotelling s T 2 the classical estimators of location and dispersion have been used to estimate the population mean vector and covariance matrix. However, this classical estimators are affected by outliers in the Phase I data. Therefore, we construct an alternative estimator based on Re-weighted

3 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 428 minimum covariance determinant (RMCD) which is more efficient and has the ability to reduce the effect of outliers. Vargas [20] and Jensen et al.[9] recommended the use of Minimum Covariance Determinants (MCD) and Minimum Volume Ellipsoid (MVE) estimators of mean vector and covariance matrix in Hotelling s T 2 charts. (See also Wisnowski et al.[23]). The exact distribution of T 2 based on the RMCD is different with classical one, so we used Monte Carlo method to obtain the appropriate control limits. 3 Designing the Robust T 2 Control Limit In the literature, a variety of robust estimators have been proposed to overcome the outliers problem and minimize their impact. In this paper, we construct a new statistic by substituting the classical estimators in Hotelling s T 2 by the re-weighted MCD estimators. 3.1 Re-weighted MCD Based Hotelling T 2 Statistic Let us say that the phase I historical data set consists of n time-ordered vectors that are independent of each other. Each vector is of dimension p and there are no subsamples in the observations, so X is a vector containing p elements for the i th time period. The re-weighted MCD based Hotelling T 2 statistic for phase II observation X f / {X 1,..., X n }, is define as follows T 2 RMCD(f) = (X f X RMCD ) S 1 RMCD(X f X RMCD ), (2) The finite-sample distributions of the T 2 RMCD(f) is unknown, So we compute the control limits based on the empirical distributions of respective robust T 2 charts, Butler et al. (1993)[1], Croux and Haesbroeck (1999) [2] and Lopuhaa (1999)[11] have studied the asymptotic properties of these estimators. It should be noted that the X RMCD and S RMCD are a good approximation to the parameters µ and Σ and X f MV N P (µ, Σ). In this case, if we use Slutsky theorem (Serfling 1980), which says as n the asymptotic distribution of robust T 2 is χ 2. however this asymptotic distribution is only applicable when n is large. In the case of small sample size, we apply Monte Carlo simulations to estimate the quantiles of the T 2 RMCD(f), for several combinations of sample sizes and dimensions. 4 Simulation We generate 5000 samples of size n from a p-multivariate standard normal distribution (MV N P (0, I p )). In phase I we compute the re-weighted MCD mean vector and covariance matrix estimates ( X RMCD and S RMCD ) for each data set

4 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 429 of size n. In addition for each data set, in Phase II we randomly generate a new observation X f from MV N P (0, I p ) and calculate the respective T 2 RMCD value as given by equation 2. By inverting the empirical distribution function of T 2 RMCD(f), we obtain Monte Carlo estimates of the 99% quantiles of T 2 RMCD(f). The creation of control limits for each sample size is weariful and time consuming, so we fit a regression equation of the form f p,1 α,γ (n) = β 0,p,1 α,γ + β 1,p,1 α,γ n β 2,p,1 α,γ to smoothly predict the control limits for any Phase I sample of size n, dimensions p = 2,..., 10,breakdown points 1 γ = 0.25, 0.5 and confidence level 1 α = The result are shown in Tables 1 and 2. (3) Table 1. The least square estimates of the regression parameters β 0,p,1 α,γ and β 1,p,1 α,γ for p = 2,..., 10 confidence levels 1 α = 0.99 and breakdown points 1 γ = 0.5 for RMCD estimators. p ˆβ 0,p,1 α,γ ˆβ1,p,1 α,γ ˆβ2,p,1 α,γ Table 2. The least square estimates of the regression parameters β 0,p,1 α,γ and β 1,p,1 α,γ for p = 2,..., 10 confidence levels 1 α = 0.99 and breakdown points 1 γ = 0.25 for RMCD estimators. p ˆβ 0,p,1 α,γ ˆβ1,p,1 α,γ ˆβ2,p,1 α,γ

5 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia Robust Control Charts Performance In this section we designed a simulation study to assess the performance of our proposed methods. We considered the following charts parameters: number of variables (p = 2, 6, 10) number of observations (n = 50, 150) breakdown point (γ = 0.5, 0.75) proportion of outliers (π = 0, 0.1, 0.2) amount of the shift in the process mean (δ = 0, 3, 5). Our simulation studies include a no-outlier pattern and a pattern with multiple outliers. We can determine the performance of the control charts by computing the probability of changes detection based on the Phase II data. The efficiency of the control charts is determined by the probability of signal which is the proportion of the T 2 RMCD that is located over the control limit, using 1500 replications. We let ncp = (µ 1 µ 0 ) Σ 1 (µ 1 µ 0 ) be the non centrality parameter that measures the severity of a shift. The signal probability depends on the value of the non-centrality parameter but not on the in-control mean vector µ or the covariance matrix Σ, so we can, without loss of generality use a zero mean vector as µ and the identity covariance matrix as Σ due to affine equivariance property. In order to compare the performance of T 2 RMCD estimator with the ordinary MCD, MVE and the classical T 2 estimators, a simulation study using 1500 samples form the MV N P (0, I p ) for n = 50, 150 was conducted. We generate clean data set (no outlier) Phase I data from the standard multivariate normal distribution MV N P (0, I p ). π of them are random data points generated from the out-of-control distribution (MV N P (µ I, I p )), and the other 1 π observations were generated from the in-control distribution (MV N P (0, I p )). Phase II data are generated from MV N P (µ II, I p ) where δ 2 II = µ II δ I = 0 (Outlier Free Phase I ) In all cases, from Table 3 to Table 7, it is visible that, by increasing the phase II non-centrality parameter δ II, the probability of signal also increases. In Table 3, the probability of signal is showed when there are no outliers in the Phase I samples of sizes 50 and 150. In the case of small sample sizes (n = 50) and low dimension, where there are no outliers in the data set the mahalanobis distance based on the usual estimator is efficient in the detection of outliers. This result is consistent whith the previous experiments conducted by the Wisnowski et al.(2002) [23], Vargas(2003) [20] and Jensen et al.(2007) [9]. Conversely, as the sample size increases to 150, the performance of all robust

6 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 431 Table 3. Probability of signal in phase II, when Phase I data is outlier free, for p = 2; 6; 10 and sample size n = 50, 150 with different values of shift in phase II process mean vector (µ II). n p µ II T 2 TMCD 2 TMV 2 E TRMCD50 2 TRMCD control charts are similar to the standard T 2 satisfactory. chart and the performance is 5.2 δ I = 5 ( Small Proportion of Outlier in Phase I ) Table 4 shows the signal probabilities when n = 50. Phase I data are generated with π = 0.10, 0.20 and non-centrality parameter of δ I = 5. As shown in Table 4, for low dimension (p = 2), the re-weighted MCD for both amount of breakdown point γ = 0.5, 0.75 (Re-MCD50 and Re- MCD75) outperform the three other methods, but for p = 6, 10 and small sample size (n = 50) the performance of all methods are not satisfactory. The Re-MCD50 and Re-MCD75 did not perform as well as they did in low dimension. The Re-MCD50 and the Re-MCD75 methods work better than the other three methods for all dimensions, if we increase the sample size to n = 150.(See table (5).) 5.3 δ I = 30 ( High Proportion of Outlier in Phase I ) Based on Tables 6 and 7, when the amount of the Phase I process shift is slightly on the higher side (δ = 30), the Re-weighted robust control charts for the breakdown point of γ = 0.5 and 0.75 surpass the MCD, MVE and the classical T 2 charts.

7 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 432 Table 4. Probability of signal in phase II, when there is a slight shift in phase I process mean vector (µ I = 5), for p = 2; 6; 10 and sample size n = 50 with different values of shift in phase II process mean vector (µ II). n p µ II T 2 TMCD 2 TMV 2 E TRMCD50 2 TRMCD % % Table 5. Probability of signal in phase II, when there is a slight shift in phase I process mean vector (µ I = 5), for p = 2; 6; 10 and sample size n = 150 with different values of shift in phase II process mean vector (µ II). n p µ II T 2 TMCD 2 TMV 2 E TRMCD50 2 TRMCD % %

8 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 433 Table 6. Probability of signal in phase II, when there is a large amount of shift in phase I process mean vector (µ I = 30), for p = 2; 6; 10 and sample size n = 50 with different values of shift in phase II process mean vector (µ II). n p µ II T 2 TMCD 2 TMV 2 E TRMCD50 2 TRMCD % % Table 7. Probability of signal in phase II, when there is a large amount of shift in phase I process mean vector (µ I = 30), for p = 2; 6; 10 and sample size n = 150 with different values of shift in phase II process mean vector (µ II). n p µ II T 2 TMCD 2 TMV 2 E TRMCD50 2 TRMCD % %

9 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia 434 The T 2 does not work well even for small sample size. It is worth mentioning that in the case of small sample size, the Re-weighted robust control chart with γ = 0.75 works better than γ = 0.5 in high dimensions. On the other hand in the case of large sample size and small proportion of outliers in data set, the Re-weighted robust control charts for breakdown points γ = 0.5 and 0.75 works almost similarly. However, if the percentage of outlier in data set is increased the Re-weighted robust control chart with γ = 0.5 outperforms the RMCD75 and other methods. Based on this study it is recommended that, when the Phase I sample contains higher proportion of outliers, higher value of breakdown point is preferred. This is the main reason for the failure of robust control charts with γ = 0.75 when there are large numbers of outliers in Phase I with large δ II. We suggest that for breakdown points of 1 γ = 0.5, a Phase I sample size of 10 to 15 times the dimension (p) is sufficient. 6 Conclusion Standard control charts are widely used in industry to detect the special causes of variation. The common out-of-control status is the occurrence of several outliers in the process. It is well known that the usual parameter estimations are sensitive to the presence of outliers, so the T 2 chart based on these estimators performs poorly. In this paper we discuss and investigate some new methods in the case of individual multivariate control charts, which consist of ordinary robust estimators and re-weighted robust estimators. The Re-Weighting step that includes in the computation of new methods lead to increase the ability of outlier detection. These estimators have many advantages, because they are affine equivariance and highly robust with better efficiency than the ordinary MCD estimators used in Vargas[20], Hardin et al.[5] and Jensen et al.[9]. We also recommend generating the T 2 RMCD control limit for different combination of sample size and dimension via Monte Carlo simulation. Our simulation studies showed that when the process is in-control and the sample size is small, the best estimator is the standard T 2, as noted in the literatures. Nonetheless for large sample size, the T 2 RMCD performs similar to the classical T 2 chart. On the other hand, when there is outlier in phase I, the T 2 RMCD is more effective than the standard T 2 and the ordinary MCD charts. References 1. Butler, R. W., Davies, P. L. and Juhn, M.: Asymptotics for the minimum covariance determinant estimator. The Annals of Statistics. 21, (1993) 2. Croux, C., Haesbroeck, G.: Influence function and efficiency of the minimum covariance determinant scatter matrix estimator. Journal of Multivariate Analysis. 71, (1999)

10 Universiti Tunku Abdul Rahman, Kuala Lumpur, Malaysia Davies, P.L.: The asymptotics of Rousseeuw s minimum volume ellipsoid estimator. The Annals of Statistics. 20, (1992) 4. Donoho, D.L., Huber, P.J.: The notion of breakdown point. A Festschrift for Erich L. Lehmann in Honor of His Sixty-fifth Birthday (P. J. Bickel, K. A. Doksum and J. L. Hodges, Jr., eds.) Wadsworth, Belmont, CA, (1983) 5. Hardin, J., Rocke, D. M.: Outlier detection in the multiple cluster setting using the minimum covariance determinant estimator. Computational Statistics and Data Analysis. 44, (2004) 6. Hotelling, H.: The Generalization of Student s Ratio. Ann. Math. Statist, 2, pp (1931) 7. Hotelling, H., Eisenhart, C., Hastay,H. and Wallis,W.A.: Techniques of Statistical Analysis. McGraw-Hill, New York, pp (1947) 8. Hubert, M., Rousseeuw, P.J., Van Aelst, S.: High-breakdown robust multivariate methods. Statistical Science. 23, (2008) 9. Jensen, W.A., Birch, J.B. and Woodall, W.H.: High breakdown stimation methods for Phase I multi1 variate control charts. Quality and Reliability Engineering International. 23: (2007) 10. Lopuhaa, H.P., Rousseeuw, P.J.: Breakdown points of affine equivariant estimators of multivariate location and covariance matrices. The Annals of Statistics. 19, (1991) 11. Lopuhaa, H.P. : Asymptotics of reweighted estimators of multivariate location and scatter. Ann Stat, 27, (1999) 12. Rousseeuw, P.J.: Multivariate estimation with high breakdown point. Mathematical Statistics and Applications B (W. Grossmann, G. Pug, I. Vincze and W. Werz, eds.) Reidel,Dordrecht (1985) 13. Rousseeuw, P. J., Leroy, A. M.: Robust Regression and Outlier Detection. John Wiley & Sons, New York, NY (1987). 14. Rousseeuw, P.J.,Van Zomeren, B.C.: Unmasking multivariate outliers and leverage points. Journal of American Statistical Association. 85, (1990) 15. Rousseeuw, P. J., Van Driessen, K.: A fast algorithm for the minimum covariance determinant estimator. Technometrics. 41, (1999) 16. Sullivan, J. H., Woodall, W. H.: A comparison of multivariate control charts for individual observations. Journal of Quality Technology 28, (1996) 17. Pison, G., Van Alest, S., Willems, G.: Small sample corrections for LTS and MCD. Metrika. 55, (2002) 18. Tracy, N. D., Young, J. C. and Mason, R. L.: Multivariate control charts for individual observations. Journal of Quality Technology. 24, (1992) 19. Van Aelst, S., Rousseeuw, P.J.: Minimum Volume Ellipsoid. Wiley Interdisciplinary Reviews: Computational Statistics (E. Wegman, Y.H. Said, D.W. Scott, Eds.)(2009) 20. Vargas, J.A.: Robust estimation in multivariate control charts for individual observations. Journal of Quality Technology. 35, (2003) 21. Yang, K., Trewn, J.: Multivariate statistical methods in quality management. McGraw Hill Professional.(2004) 22. Willems, G., Pison, G., Rousseeuw, P.J., Van Alest, S.: A robust Hotelling test. Metrika. 55, (2002) 23. Wisnowski, J.W., Simpson, J.R., Montgomery, D.C.: A performance study for multivariate location and shape estimators. Quality and Reliability Engineering International 18, (2002)