Module B1: Multivariate Process Control

Transcription

1 Module B1: Multivariate Process Control Prof. Fugee Tsung Hong Kong University of Science and Technology Quality Lab:

2 I. Multivariate Shewhart chart

3 WHY MULTIVARIATE PROCESS CONTROL I. Need several variables to characterize a process II. Correlation between variables Dr. Fugee Tsung, HKUST 3

4 Dr. Fugee Tsung, HKUST 4

5 Multivariate control offers increased sensitivity to departure from in-control Multivariate control offers improved diagnostics Dr. Fugee Tsung, HKUST 5

6 Multivariate Shewhart chart: monitoring X1,, Xn ~ Np( μ, Σ ), individual identically independently distributed. To test whether there is a change in the data s distribution, i.e., to test hypothesis Dr. Fugee Tsung, HKUST 6

7 Multivariate Shewhart chart : monitoring T-squared statistics -Hoteling(1947) If and are known, the test statistic is given by Dr. Fugee Tsung, HKUST 7

8 Multivariate Shewhart chart : monitoring Otherwise, substitute their estimates in the mathematical expression. For individual observation Dr. Fugee Tsung, HKUST 8

9 Multivariate Shewhart chart : monitoring Dr. Fugee Tsung, HKUST 9

10 Multivariate Shewhart chart : monitoring Dr. Fugee Tsung, HKUST 10

11 Multivariate Shewhart chart : monitoring Dr. Fugee Tsung, HKUST 11

12 Multivariate Shewhart chart : Batch monitoring Batch process: I. Processing unit uses batches as input; II. The entire processing unit is changed at specified time intervals; III. Process that is purposely shifted to a different configuration to change production from one product to another inherently generates batches. Dr. Fugee Tsung, HKUST 12

13 Multivariate Shewhart chart : Batch monitoring For batch data in phase I, Mason Chou and Young (2001, JQT) propose ways to do more accurate analysis. They categorize the batch processed into two categories. a. Category 1: b. Category 2: Dr. Fugee Tsung, HKUST 13

14 Multivariate Shewhart chart : Batch monitoring Dr. Fugee Tsung, HKUST 14

15 Multivariate Shewhart chart : Batch monitoring Dr. Fugee Tsung, HKUST 15

16 Multivariate Shewhart chart : Batch monitoring Dr. Fugee Tsung, HKUST 16

17 Multivariate Shewhart chart : Batch monitoring Dr. Fugee Tsung, HKUST 17

18 Multivariate Shewhart chart : Regression charts T-square is optimal for a general shift in the mean. But when interest lies in just certain directions, it is not sufficiently efficient. Furthermore, it is not immediately interpretable. Dr. Fugee Tsung, HKUST 18

19 Multivariate Shewhart chart : Regression charts Dr. Fugee Tsung, HKUST 19

20 Multivariate Shewhart chart : Regression charts Dr. Fugee Tsung, HKUST 20

21 Multivariate Shewhart chart : Regression charts Dr. Fugee Tsung, HKUST 21

22 Multivariate Shewhart chart : Regression charts Dr. Fugee Tsung, HKUST 22

23 Multivariate Shewhart chart: Diagnosing Drawbacks of Hotelling s T-square statistic: when a T-square statistic shows that the process is out of control, a. It doesn t provide information on which variable or set of variables is out of control; b. Also, it s difficult to distinguish location shifts from scale shifts. Dr. Fugee Tsung, HKUST 23

24 Multivariate Shewhart chart: Diagnosing It would be helpful to be able to determine the net effect of each variable of the p variables on the statistic and the particular factors determining the effect. Principal component analysis is a good choice for interpretation of the out-ofcontrol signal. But, it is not always for the components to be meaningful. Dr. Fugee Tsung, HKUST 24

25 Multivariate Shewhart chart: Diagnosing Mason Young and Tracy (1995,JQT) propose a way to decompose T-square statistic into p independent components, each of which providing information on variables that significantly contribute to an out-of-control signal. Dr. Fugee Tsung, HKUST 25

26 Multivariate Shewhart chart: Diagnosing Dr. Fugee Tsung, HKUST 26

27 Multivariate Shewhart chart: Diagnosing Dr. Fugee Tsung, HKUST 27

28 Multivariate Shewhart chart: Diagnosing Dr. Fugee Tsung, HKUST 28

29 Multivariate Shewhart chart: Diagnosing Dr. Fugee Tsung, HKUST 29

30 Multivariate Shewhart chart: Diagnosing There are in fact p! different partitioning that will yield the same overall T-square statistic. The p terms within a particular decomposition are independent of one another although the terms across the p! decompositions are not all independent. Each of the terms are distributed as a constant times an F distribution having 1 and n-1 degrees of freedom. Dr. Fugee Tsung, HKUST 30

31 Multivariate Shewhart chart: Diagnosing Dr. Fugee Tsung, HKUST 31

32 Multivariate Shewhart chart: Diagnosing Illustrating Figure Dr. Fugee Tsung, HKUST 32

33 Multivariate Shewhart chart: Diagnosing For larger p, the number of possible decompositions is large, but appropriate computing can greatly reduce this computational effort (e.g., see Mason, Tracy and Young(1997)). Dr. Fugee Tsung, HKUST 33

34 Ⅱ. Multivariate CUSUM chart

35 Multivariate CUSUM chart As its univariate counterpart, T-square chart is not sensitive to small and moderate shifts. Method that can accumulate information across successive observations is needed. Dr. Fugee Tsung, HKUST 35

36 Multivariate CUSUM chart Crosier (1988, Technometrics ) proposed two multivariate CUSUM schemes. That is to test hypothesis: Dr. Fugee Tsung, HKUST 36

37 Multivariate CUSUM chart MC1(multi-variate CUSUM ) is given below: Initialize the CUSUM vector S to a zero vector, and then use the recursion S and where n 0; for Cn k = ( Sn 1+ Xn μ0)(1 k/ Cn); for Cn > k 1 1/2 C = [( S + X μ )' Σ ( S + X μ )]. n n 1 n 0 n n 1 n 0 1 1/2 The chart gives off signal if ( Sn' Σ Sn) > h, where his the scalar decision interval. Dr. Fugee Tsung, HKUST 37

38 Multivariate CUSUM chart MC2 is given below: Dr. Fugee Tsung, HKUST 38

39 Multivariate CUSUM chart Based on simulation study, Crosier pointed out that MC1 has a better performance than MC2. But the choice of k of MC1 remains an open-ended question. A reasonable choice is set k =δ/2, where δ is the shift size of interest. Dr. Fugee Tsung, HKUST 39

40 Multivariate CUSUM chart Pignatiello and Runger (1990, JQT) also proposed two multivariate CUSUM schemes, MC3 and MC4. MC3 is based on vectors of cumulative sums on the following page. Without loss of generality, let. Dr. Fugee Tsung, HKUST 40

41 Multivariate CUSUM chart Dr. Fugee Tsung, HKUST 41

42 Multivariate CUSUM chart MC4 is given by Dr. Fugee Tsung, HKUST 42

43 Comparison of two MCUSUM Pignatiello and Runger (1990, JQT) and Crosier (1988, Technometrics ) proposed four schemes of MCUSUM. MC1 and MC3 both accumulate the X vectors before producing the quadratic forms. While MC2 and MC4 do a quadratic form for each X first and then accumulate those quadratic forms. Dr. Fugee Tsung, HKUST 43

44 Comparison of two MCUSUM It s pointed out in Pignatiello and Runger (1990, JQT) that the MC1 and MC3 perform better than MC2 and MC4. MC1 and MC3 have similar ARL performance. MC1 is somewhat less complicated. Dr. Fugee Tsung, HKUST 44

45 Ⅲ. Multivariate EWMA chart

46 Introduction CUSUM and EWMA control charts are all used to detect small and moderate shifts. But, except the applicability for independent observations, EWMA statistic is the optimal predictor of the next observation from a first-order moving average process (MacGregor and Harris, 1990). Dr. Fugee Tsung, HKUST 46

47 Univariate EWMA statistic Dr. Fugee Tsung, HKUST 47

48 Multi-variate EWMA chart Suppose Xi ~ Np( μi, Σ ), i=1, 2, are independent observations. Σ is known and μ0 =0 is the in-control mean vector. To test Dr. Fugee Tsung, HKUST 48

49 Multivariate EWMA chart For this multivariate process, Lowry, Woodall, Champ, and Rigdon (1992, Technometrics) propose a multivariate EWMA chart. Dr. Fugee Tsung, HKUST 49

50 Multivariate EWMA chart R1,,rp will be set equally to be r for, a. with no prior knowledge, all variables are of equal importance; b. if not, a chart that is sensitive to a particular direction can be applied. Dr. Fugee Tsung, HKUST 50

51 Multivariate EWMA chart With r1= =rp=r, the statistic is simplified to be, Corresponding control limits can then be set. Dr. Fugee Tsung, HKUST 51

52 Ⅳ. A Kernel-distance-based Multivariate Control Chart Using Support Vector Methods Sun, R. and Tsung, F. (2003), A Kernel- Distance-Based Multivariate Control Chart using Support Vector Methods, International Journal of Production Research, 41,

53 Introduction Most quality variables to be controlled and monitored are not independent, but generally correlated with each other. How to monitor them simultaneously is crucial to success in multivariate SPC. Design of control limits in many MSPC schemes (eg. T- square, MCUSUM, MEWMA) is based on normality assumption while in practice, actual distribution is unknown and usually not easy to estimate. Sun and Tsung(2003, IJPR) propose a K-chart based on support vector method, which is distribution-free. Dr. Fugee Tsung, HKUST 53

54 Introduction Presentation of MSPC techniques from an evolutionary perspective: Dr. Fugee Tsung, HKUST 54

55 Dr. Fugee Tsung, HKUST 55

56 Support Vector Machine (SVM) SVM constructs a separating hyperplane that gives a maximal margin between two classes. Dr. Fugee Tsung, HKUST 56

57 SVM: two-dimensional illustration Dr. Fugee Tsung, HKUST 57

58 SVM: mathematical expression The hyperplane can be expressed as Def ( Margin between two classes ): The distance in the direction perpendicular to the seperating hyperplane between the nearest points of each class to the hyperplane. Dr. Fugee Tsung, HKUST 58

59 SVM: mathematical expression To find solution to is equivalent to solving the following quadratic programming problem: under constraints Dr. Fugee Tsung, HKUST 59

60 SVM: mathematical solution Solution to the optimization problem is unique and can be obtained by solving the following quadratic programming problem, Dr. Fugee Tsung, HKUST 60

61 SVM: kernel function The linear discrimination function in (*) is not flexible enough to deal with the real nonlinear boundary between two classes. The Hilbert-Schmidt theory provides one solution. Substitute the inner product with a nonlinear kernel function in (*), (Xi Xj) K(Xi, Xj) Dr. Fugee Tsung, HKUST 61

62 SVM: kernel function Commonly used kernels Dr. Fugee Tsung, HKUST 62

63 Support Vector Data Description (SVDD) SVM is for binary or multi-class classification; SVDD is for one-class classification. The main idea of SVDD is to envelop the samples within a high-dimensional space with the volume as small as possible. Dr. Fugee Tsung, HKUST 63

64 SVDD: mathematical expression The problem is to find the center (O) and the radius (R) of the hyperplane that has the minimum volume to contain all the sample data, i.e., Dr. Fugee Tsung, HKUST 64

65 SVDD: mathematical expression This is equivalent to solve Dr. Fugee Tsung, HKUST 65

66 SVDD: figure illustration Dr. Fugee Tsung, HKUST 66

67 K chart The distance between the sample points and the center is a useful measure in monitoring. The greater the distance, the greater the probability that the process may have gone out-of-control. Dr. Fugee Tsung, HKUST 67

68 K chart The distance between a new sample z and the center (O) is defined as With inner product replaced by kernel function, the kernel distance is obtained Dr. Fugee Tsung, HKUST 68

69 Comparison between K charts and T-square charts Dr. Fugee Tsung, HKUST 69

70 Journal of Quality Technology, V. High-Dimensional Process Monitoring and Fault Isolation via Variable Selection Journal of Quality Technology 41, , 2009 Journal of Quality Technology 44, 2012 Prof. Kaibo Wang Department of Industrial Engineering Tsinghua University Prof. Wei Jiang Antai College of Economics and Management Shanghai Jiaotong University Dr. Fugee Tsung, HKUST 70

71 Motivation - 1 Process Monitoring and Fault Isolation are equally important Challenges in High-Dimensional Process Monitoring Weak signals hard to be detected Fault isolation becomes difficult MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 71 71

72 Probability of Detection vs. Dimension Scenario: one variable, shift magnitude=1.0 Detection Probability Detection Probability MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 72 72

73 Shift Size vs. Dimension 5 Scenario: one variable, equal out-of-control ARL 4 NCP Variable MSN= MSN= MSN= Dimension MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 73 73

74 Motivation - 2 Scenario: 10 variables, each becomes out-of-control with a probability of 1% Probability % 0.4% Number Of Simultaneously Shifted Sources 10 MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 74 74

75 GLRT statistic for process monitoring Assume y ~ N ( μ, Σ) Statistical hypothesis Testing λ( y ) t t 0 1 p max L( y, μ) Ω = max L( y, μ) Ω t t H H : μ Ω 0 0 : μ Ω, 1 1 Λ = Σ + Σ < T 1 T 1 ( yt) min( yt yt ( yt μ) ( yt μ)) log μ Ω μ = yt 2 T 1 t 1 T = y Σ y > h t MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST S 2 c 1

76 A Penalized Likelihood Ordinary likelihood S Penalized likelihood for shift estimation S = min(( y μ) Σ ( y μ)) 2 T 1 t μ Ω 1 p 2 T 1 = yt Σ yt + p λ j j μ Ω1 j= 1 min(( μ) ( μ) ( μ )) L 0 -penality: t pλ ( μ ) = λi( μ 0) j j j L 1 -penality: p λ ( μ ) = λ μ LASSO j j j MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 76 76

77 L 0 Penalized Least Square Denote 2 T T S ( λ) = min(( y μ) x x( y μ) + λm Purpose μ Ω Obtain µ* M = Σ I( μ 0) 1 T = min(( xy xμ) ( xy xμ) + λm. μ Ω 1 j t t Use µ* to estimate process shifts Limited the number of nonzeros in µ* Solution Variable selection Keep M=s nonzero coefficients in µ* j MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST t t

78 Remarks Variable-selection Helps to screen out suspected out-of-control variables Helps to estimate shift magnitude Penalized likelihood Hotellint s T 2 estimates µ*=y Penalized likelihood gives more reliable estimate of µ* MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 78 78

79 Multivariate SPC via Variable Selection Charting statistics Λ = Σ Σ > T 1 * * T 1 * ( yt) 2 yt μ μ μ c ' Implementation procedures 1. Variable selection at each step 2. Process monitoring 3. Signal diagnosis MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 79 79

80 Performance Comparison MSPC ARL Comparison, p=50 2 δ T VS-MSPC M =1 M =2 M = Dr. Dr. Fugee Kaibo Tsung, WangHKUST 80

81 Performance Comparison Relative efficiency Hotelling s T 2 / VS-MSPC 3.5 Relative Efficiency Variable p=10 P=20 p=50 p= MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST δ 3 4 5

82 Diagnostic Capability Study δ y 1 y 2 y 3 y 4 y 5 y 6 y 7 y 8 y 9 y 10 Correctness % % % % % % % MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 82 82

83 An Real Example Vertical Density Profiles (VDP) Each profile consists of 314 points, correspond to 314 random variables Density Density Average Profile Depth Average Profile Depth MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 83 83

84 An Real Example Charting Statistics Although both show no out-of-control, VS- MSPC shows sample 13 is worse than sample 21 UCL 40 Charting Statistics UCL Sample Number Smallest p-value VS-MSPC: T2: Sample Number MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST

85 Individual Samples Density Density Variable Sample 13 Sample Variable Sample 13 Sample Depth Depth Williams et al. (2007) analyzed the same dataset but sample 13 is not suspected by any charts. In their analysis, the correlation structure among the variables was ignored. Williams et al. (2007) identified that sample 15 is the most profound outlier Even though sample 15 deviates far from zero near its high end, it stays closer to zero than sample 13 at most of the rest locations MSPC Dr. Dr. Fugee Kaibo Tsung, WangHKUST 85 85

86 EWMA Extensions time response covariate weight t t 1 t 2 M z z z t R R ω ω t 1 t 1 R ω t 2 t 2 M M M t t T min( ωi( zi Rμt) ( zt Rμt)) μt i= 0 st.. Σ ji( μt( j) 0) s VS-MEWMA Dr. Kaibo Wang 86

87 EWMA Extensions w t = (1 λ) w + t 1 λy t st.. Σ ji( μt( j) 0) s T 1 min( wt μt) Σ ( wt μt) μ t accumulate observations in an EWMA way. VS-MEWMA Dr. Kaibo Wang 87

88 Reference MShewhart Hawkins, D.M. (1991), Multivariate Quality Control Based on Regression-Adjusted Variables, Technometrics, 33, Hotelling, H. (1947), Multivariate Quality Control, Communications in Statistics- Theory and Methods, 14, Sullivan, J.H., and Woodall, W.H. (1996), A Comparison of multivariate control charts for individual observations, Journal of quality Technology, V. 28, Iss. 4, Mason, R.L, Chou, Y.M. and Young, J.C., (2001), Applying Hotelling s T(2) Statistic to Batch Process, Journal of quality Technology, V. 33, Iss. 4, Mason, R.L, Tracy, N.D, Young, J.C (1995), "Decomposition of T2 for multivariate control chart interpretation", Journal of Quality Technology, V. 27 No.2, Mason, R.L, Tracy, N.D, Young, J.C (1997), A Practical Approach for Interpreting Multivariate T(2) Control Chart Signals, Journal of Quality Technology, V. 29, Tracy, N.D, Young, J.C, Mason, R.L, (1992), Multivariate control Charts for Individual Observations, Journal of Quality Technology, V. 24 No.2, Dr. Fugee Tsung, HKUST 88

89 Reference MCUSUM Crosier, R.B.(1988), Multivariate Generalizations of Cumulative Sum Quality- Control Schemes, Technometrics, V.30, No. 3, Pignatiello, J.J.,Jr., and Runger, G.C. (1990), Comparison of Multivariate CUSUM Charts, Journal of Quality Technology, 22, MEWMA Lowry, C.A., Woodall, W. H., Champ, C.W. and Rigdon, S.E. (1992), A Multivariate Exponentially Weighted Moving Average Control Chart, Technometrics, 34, No. 1, Lucas, J.M., and Saccucci, M.S. (1990), Exponentially Weighted Moving Average Control Schemes: Property and Enhancements, Technometrics, 32, No. 1, MacGregor, J.F., and Harris, T.J. (1990), Discussion of Exponentially Weighted Moving Average Control Schemes: Property and Enhancements by J.M. Lucas, and M.S. Saccucci, Technometrics, 32, Roberts, S.W. (1959), Control Charts Tests Based on Geometrics Moving Average Schemes, Technometrics, 1, Dr. Fugee Tsung, HKUST 89

90 Reference Apley, D.W., and Tsung, F., (2002), The Autoregressive T-square Charts for Monitoring Univariate Autocorrelated Processes, Journal of Quality Technology, V.34, Ning, X. and Tsung, F. (2012), " A Density-based Statistical Process Control Scheme for High-dimensional and Mixed-type Observations," IIE Transactions, 44, Ning, X. and Tsung, F. (2012), " Improved Design of Kernel-distance-based Charts using Support Vector Methods," IIE Transactions, accepted. Sun, R. and Tsung, F. (2003), A Kernel-Distance-Based Multivariate Control Chart using Support Vector Methods, International Journal of Production Research, 41, Sun, R., Tsung, F. and Qu, L.(2007), Evolving Kernel Principal Component Analysis for Fraud Detection, Computers and Industrial Engineering, 53, Sukchotrat, T., Kim, S. B. and Tsung, F. (2010), " One-class classificationbased control charts for multivariate process monitoring," IIE Transactions, 42, Dr. Fugee Tsung, HKUST 90

91 Tsung, F., Zhou, Z.H. and Jiang, W. (2007), Applying Manufacturing Batch Techniques to Fraud Detection with Imcomplete Customer Information, IIE Transactions, 39, Wang, K. and Jiang, W. (2009). "High-Dimensional Process Monitoring and Fault Isolation via Variable Selection." Journal of Quality Technology 41, Wang, K., Jiang, W., and Tsung, F. (2012), " A Variable-Selectionbased Multivariate EWMA Chart for Process Monitoring and Diagnosis," Journal of Quality Technology, tentatively accepted. Wang, K. and Tsung, F. (2005), Using Profile Monitoring Techniques for a Data-rich Enviroment with Huge Sample Size, Quality and Reliability Engineering International, 21, Zou, C., Jiang, W., and Tsung, F. (2011), " LASSO-Based Diagnostic Framework for Multivariate Statistical Process Control," Technometrics, 53, Dr. Fugee Tsung, HKUST 91