Reconstruction-based Contribution for Process Monitoring with Kernel Principal Component Analysis.

Transcription

1 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July, FrC.6 Reconstruction-based Contribution for Process Monitoring with Kernel Principal Component Analysis. Carlos F. Alcala and S. Joe Qin Abstract This paper presents a new method for fault diagnosis based on kernel principal component analysis (KPCA). The proposed method uses reconstruction-based contributions (RBC) to diagnose simple and complex faults in nonlinear principal component models based on KPCA. Similar to linear PCA, a combined index, based on the weighted combination of the Hotelling s T and indices, is proposed. Control limits for these fault detection indices are proposed using second order moment approximation. The proposed fault detection and diagnosis scheme is tested with a simulated CSTR process where simple and complex faults are introduced. The simulation results show that the proposed fault detection and diagnosis methods are efective for KPCA. I. INTRODUCTION Principal component analysis (PCA) is a multivariate statistical tool widely used in industry for process monitoring ([], [4], [], [3]). It decomposes the measurement space into a principal component space that contains the commoncause variability, and a residual space that contains the process noise. The task of process monitoring is to detect and diagnose faults when they happen in a process. In process monitoring with PCA, fault detection is performed with fault detection indices. Popular indices are the Hotelling s T statistic for monitoring the principal component subspace; the index, which monitors the residual subspace; and a combination of both indices that monitors the whole measurement space. Once a fault is detected, it is necessary to diagnose its cause. A popular method for fault diagnosis is the contribution plots. They are based on the idea that variables with large contributions to a fault detection index are likely the cause of the fault. However, as was demonstrated by Alcala and Qin [], even for simple sensor faults, contribution plots fail to guarantee correct diagnosis. As an alternative to contribution plots, Alcala and Qin [] propose the use of reconstruction-based contributions (RBC) that overcomes the aforementioned shortcoming. The RBC method is related to fault identification by reconstruction ([6], [7]), but it does not require the knowledge of fault directions. The PCA-based process monitoring scheme mentioned above is based on the assumption that the process behaves linearly. However, when a process is nonlinear, the monitoring of a process using a linear PCA model might not C. F. Alcala is with the Mork Family Department of Chemical Engineering and Materials Science, University of Southern California, CA 989, USA, alcalape@usc.edu S. J. Qin is with the Mork Family Department of Chemical Engineering and Materials Science, and the Ming Hsieh Deparment of Electrical Engineering, University of Southern California, CA 989, USA, sqin@usc.edu perform properly. In order to address this issue, several nonlinear PCA methods have been proposed ([9], [5], [8], [3]). One of these methods is kernel principal component analysis (KPCA), developed by Schölkopf et al. [3], which maps measurements from their original space to a higher dimensional feature space where PCA is performed. KPCA has been applied successfully for process monitoring ([], [4], [7], [8]). Fault detection with KPCA is done in a similar way as with PCA using a T, and combined indices. However, fault diagnosis cannot be performed with contribution plots since there seems to be no way to calculate them with KPCA. Choi et al. [4] propose a method for fault diagnosis employing the reconstruction-method of Dunia et al. [7], which looks at a fault identification index obtained when a fault detection index has been reconstructed along the direction of a variable. While this method applies to sensor faults or a process fault with a known direction, it cannot be applied to diagnosing faults with unknown directions. In this paper, reconstruction-based contribution for kernel PCA (RBC-KPCA) is proposed for fault diagnosis. We present a matrix formulation for KPCA and the kernel trick that is simpler than the standard kernel PCA description, and is analogous to the linear PCA formulation familiar to the process monitoring community. The T and fault detection indices are expressed in terms of a kernel vector mapped by the kernel function, rather than the intangible vector in the feature space. Furthermore, the combined index approach of Yue and Qin [6] is proposed for fault detection in the feature space and new control limits for the fault detection indices are proposed. The proposed RBC diagnosis method is used just like the standard contribution plots for KPCA-based fault diagnosis, although it is difficult to extend the standard contribution plots for KPCA. It is shown that simple and complex faults can be diagnosed correctly using RBC-KPCA with a simulation of a CSTR process. A. KPCA Background II. KERNEL PCA In order to make the kernel PCA model of process data with n variables, m measurements, taken under normal operating conditions, form the training set X = [x x x m ] T. Each of the measurements x i is an n-dimensional vector and can be mapped into an h-dimensional space via a mapping function φ i = Φ (x i ). The h-dimensional space is called the feature space and the n-dimensional space is called the measurement space. An important property of the feature space is that the dot product of two vectors φ i and φ j can //$6. AACC 7

2 be calculated as a function of the corresponding vectors x i and x j, φ T i φ j = k (x i, x j ) () The function k (, ) is called the kernel function, and there exist several types of these functions. The gaussian kernel function, or radial basis function, is expressed as k (x i, x j ) = exp [ (x i x j ) T (x i x j ) c and will be used in this paper. For process monitoring with linear PCA, the model is obtained from the training set X. In kernel PCA, the covariance matrix of a set of m mapped vectors φ i is eigen-decomposed to obtain the principal loadings of the model. Assume that the vectors in the feature space are scaled to zero mean and form the training data as X = [φ φ φ m ] T. Let the sample covariance matrix of the data set in the feature space be S. We have, (m )S = X T X = ] () m φ i φ T i (3) i= KPCA in the feature space is equivalent to solving the following eigenvector equation, m X T X ν = φ i φ T i ν = λν (4) i= Note that φ i is not explicitly defined nor is φ i φ T j. The socalled kernel trick pre-multiplies Eq. 4 by X : X X T X ν = λx ν Defining φ T φ φ T φ m K = X X T =..... φ T mφ φ T mφ m k (x, x ) k (x, x m ) =..... (5) k (x m, x ) k (x m, x m ) and denoting we have α = X ν (6) Kα = λα (7) Eq. 7 shows that α and λ are an eigenvector and eigenvalue of K, respectively. In order to solve ν from Eq. 6, we premultiply it by X T and use Eq. 4, X T α = X T X ν = λν which shows that ν is given by ν = λ X T α. (8) Therefore, to calculate the PCA model (λ i and ν i ), we first perform eigen-decomposition of Eq. 7 to obtain λ i and α i. Then use Eq. 8 to find ν i. In order to guarantee that ν T i ν i =, Eqs. 6 and 4 are used to derive α T i α i = ν T i X T X ν i = ν T i λ i ν i = λ i. Therefore, α i needs to have a norm of λ i. Let α i unit norm eigenvector corresponding to λ i, α i = λ i α i. be the The matrix with the l leading eigenvectors are the KPCA principal loadings in the feature space, denoted as P f = [ν ν ν l ]. From Eq. 8, P f is related to the loadings in the measurement space as [ ] P f = X T α X T α l λ λ [ l ] = X T α λ X T α l λ l = X T PΛ (9) where P = [α α l ] and Λ = diag{λ λ l } are the l principal eigenvectors and eigenvalues of K, respectively, corresponding to the largest eigenvalues in descending order. For a given measurement x and its mapped vector φ = Φ (x), the scores are calculated as t = P T f φ, which, from Eq. 9, we can be expressed as where t = Λ P T X φ = Λ P T k(x). () k(x) = X φ = [ φ T φ φ T φ φ T mφ ] T B. Scaling = [k(x, x) k(x, x) k(x m, x)] T () The calculation of the covariance matrix in Eq. 3 holds if the vector φ in the feature space has zero mean. If this is not the case, the vectors in the feature space have to be scaled to zero mean using the sample mean of the training data. The scaled vector φ is φ = φ m m φ i = φ [φ φ m ] m i where m is an m-dimensional vector whose elements are /m. The kernel function of two scaled vectors φ i and φ j is k (x i, x j ) = φ T i φ j = k (x i, x j ) k (x i ) T m k (x j ) T m + T mk m () Similarly, the scaling of the kernel vector k (x) is where k (x) = [ φ φ φ m ]T φ = F [k (x) K m ] (3) F = I E (4) 73

3 In this equation, I is the identity matrix, and E is an m m matrix with elements /m. Finally, the scaled kernel matrix K, K, is calculated as K = [ φ φ φ m ] T [ φ φ φ m ] = FKF (5) Therefore, it is straightforward to transform vectors in the feature space to zero mean before KPCA. Without loss of generality, we assume the vectors in the feature space are zero mean for the rest of the paper. III. FAULT DETECTION WITH KPCA A. Fault Detection Indices Since the key idea of KPCA is to map the measurement space into the feature space, so that data in the feature space are linearly distributed, it is natural to perform fault detection by defining statistics in the feature space. The Hotelling s T index is calculated as T (x) = t T Λ t, where Λ = diag (λ,, λ l ) is the covariance of the scores t in the feature space. From Eq., T is calculated using kernel functions as T = k (x) T Dk (x) (6) with D = PΛ P T. The index is defined as the norm of the residual vector in the feature space and is calculated as φ T C f φ, where C f is the projection matrix of the residual space, which is orthogonal to the principal component space. Let t be the residual components and P f the corresponding loading matrix, t = P T f φ = [ν l+ ν l+ ] T φ The index is calculated as the squared norm of the residual components SP E = t T t = φ T P f P T f φ. Since we do not know the dimension of the feature space, it is not possible to know the number of residual components there. Thus, we cannot calculate explicitly the loading matrix P f. However, we can calculate the product P f P T f as the projection orthogonal to the principal component space, which is C f = P f P T f = I P f P T f and leads to SP E = φ T ( I P f P T ) f φ = φ T φ φ T P f P T f φ. From Eqs., 9 and we have SP E = k (x, x) k (x) T Ck (x) (7) where C = PΛ P T. Yue and Qin [6] proposed the use of a combined index for monitoring the principal and residual space simultaneously. Such an index is a combination of the T and indices weighted by their control limits. The same concept is used here to define a fault detection index to monitor the principal and residual spaces in the feature space. A combined index for fault detection in the feature space has been proposed by Choi et al. [4]; however, their definition is different to the one proposed here as they use an energy approximation concept to calculate the index. The combined index proposed for the feature space is defined as (x) = SP E (x) δ + T (x) τ (8) where δ and τ are the control limits for the and T indices, respectively. The calculation of these control limits is provided in the next subsection. The combined index can be calculated with kernel functions as where (x) = Ω = PΛ P T τ k (x, x) δ + k (x) T Ωk (x) (9) PΛ P T δ = D τ C δ. () The combined index has a control limit ζ and its calculation is provided in the next section. B. Control limits In the feature space, the fault detection indices have the quadratic form J = x T Ax () with A. If it is considered that x is zero mean and normally distributed, the results of Box [] can be applied to calculate control limits for the fault detection indices. These limits can be calculated as g Index χ α ( h Index ), with a confidence level of ( α) %, and the parameters g Index and h Index calculated as g Index = tr{sa} tr{sa} () h Index = [tr{sa}] tr{sa} (3) where tr{a} is the trace of matrix A. We use Index to represent either of the, T or indices. The parameters g Index and h Index for the detection indices are shown in Table I. In the table, the parameter λ i is the ith eigenvalue of the kernel matrix K. IV. RECONSTRUCTION-BASED CONTRIBUTION FOR FAULT DIAGNOSIS Reconstruction-based contribution (RBC), proposed by Alcala and Qin [], defines the reconstruction of a fault detection index along the direction of a variable as the variables contribution for fault diagnosis. The variable with the largest amount of reconstruction is likely a major contributor to the fault. This method is useful for it can not only obtain contributions along the direction of a variable, but along an arbitrary direction. Therefore, it is possible to obtain the contribution of simple faults, as well as complex faults provided that their directions are available in a fault library. 74

4 TABLE I CONTROL LIMITS FOR THE FAULT DETECTION INDICES. Limit = g Index χ α ( h Index ). Index Limit Parameters g T = T τ m h T = l SP E δ g SP E = h SP E = mi=l+ λ i (m ) m i=l+ λ i ( mi=l+ ) λ i mi=l+ λ i Solvent Flow FS CAS T 7 FA 9 CAA 8 Pure A Solute Flow SP M FC 3 SP TC TC 4 CA Cooling Water Flow Fig.. Diagram of the CSTR process [5] g = ζ (m ) h = l/τ 4 + m i=l+ λ i (l/τ /δ4 + m i=l+ λ i /δ ) (l/τ + m i=l+ λ i /δ ) l/τ 4 + m i=l+ λ i /δ4 The objective of RBC is to find the magnitude f i of a vector with direction ξ i such that the fault detection index of the reconstructed measurement z i = x ξ i f i (4) is minimized. This is, we want to find f i = arg min Index (x ξ i f i ). The same concept can be applied to KPCA and find f i such that f i = arg min Index (k (x ξ i f i )) (5) The RBC value for the direction ξ i is fi. If we want to do reconstruction along the direction of a variable, the direction can be written as ξ i = [ ], where is placed at the ith position. When complex faults are reconstructed, the direction of the fault can be any unitary vector ξ i. In order to find the RBC along a direction ξ i for a fault detection index, we have to perform a nonlinear search of the value of f i that minimizes Index (k (x ξ i f)). This is done by obtaining the first derivative of the fault detection index with respect to f i, then equate to zero and solve for f i. However, the resulting expression is not an explicit solution for f i and it has to be iterated until f i has converged. Employing the kernel function in Eq., and Eq. 6, the derived expression of f i for the T index is f i = ξt i BT FD k (z i ) k T (z i ) FD k (z i ) (6) In this equation, F is the scaling matrix defined in Eq. 4, k (z i ) is the vector with the unscaled values of k (z i, x i ), k (z i ) contains the scaled values and is calculated with Eq. 3. The matrix B is calculated as B = k (z i, x ) (x x ) T k (z i, x ) (x x ) T. k (z i, x m ) (x x m ) T (7) For, via Eq. 7, the derived expression for f i is f i = ξt i BT [ m + FC k (z i ) ] k T (z i ) [ m + FC k (z i ) ] (8) For the index, using Eq. [ 9, f i is calculated as f i = ξt i BT m δ FΩ k (z i ) ] k T (z i ) [ m δ FΩ k (z i ) ] (9) In the next section a simulation is performed to test the fault detection and diagnosis methods discussed so far. V. CSTR SIMULATION In this section, a nonisothermal continuous stirred tank reactor (CSTR) is simulated. The process model and simulation conditions are similar to the ones provided by Yoon and MacGregor [5], and they are given in Appendix A; a diagram of the proces is shown in Figure. The simulation is performed according to the following model V ρc p dt dt dc A dt = F V C A F V C A k e E RT CA (3) = ρc p F (T T ) af b+ C F C + af b C / (ρ Cc pc ) (T T C,in ) + ( H rxn ) V k e E RT CA (3) The process is monitored measuring the cooling water temperature T C, the inlet temperature T, the inlet concentrations C AA and C AS, the solvent flow F S, the cooling water flow F C, the outlet concentration C A, the temperature T, and the reactant flow F A. These nine variables form the measurement vector x = [T C T C AA C AS F S F C C A T F A ] T (3) The variables are sampled every minute and samples, taken under normal conditions, are used as the training set. Four different faults are introduced into the system. The first 3 faults are sensor faults similar to the ones shown in Choi et al. [4]. The first simulated fault, Fault, is a bias in the sensor of the output temperature T ; the bias magnitude is (K). Since T is a controlled variable, the effect of the fault will be removed by the PI controller and its effect will propagate to other variables. This fault is considered a complex fault 75

5 since it affects several variables. The second and third faults, Fault and Fault 3, are biases in the sensors of the inlet temperature T o and inlet reactant concentration C AA, respectively; the bias magnitude for T is.5(k) and for C AA is. (kmole/m 3 ). These faults are considered simple faults since they only affect one variable. The last fault, Fault 4, is a slow drift in the reaction kinetics; the fault has the form of an exponential degradation of the reaction rate caused by catalyst poisoning; in this case, the reaction rate coefficient will change with time as k (t + ) =.996 k (t). This process fault is a complex fault that affects several variables such as the output temperature T, concentration C A, and the cooling water flow F C. For each of the fault scenarios mentioned above, measurements are simulated and the fault is introduced at the 5st measurement. The KPCA model is built using 4 principal components and the parameter c in the kernel equation is set to.65. Figures to 5 show the fault detection and diagnosis results for the faults. The first column in each set of plots shows the, T and indices for the simulated measurements. The second colum shows the results of fault diagnosis given by RBC with KPCA for the three fault detection indices. In Figure, it is shown that the SP E and combined indices detect the fault as soon as it happens, not doing the same the T index. The first nine bars of the RBC plots, for the three fault detection indices, show the RBC values of all variables. Since the bias fault happens in the temperature sensor, Variable 8, this variable is supposed to have the largest RBC; however, due to the feedback control, the effect of the fault is transferred to the cooling water flow, Variable 6, as indicated by the RBCs of, T and. In this case, more information is needed to determine the cause of the fault. Once the cause of the fault has been determined and stored in a library of known faults, its direction can be used to calculate the RBC value along this fault direction. The tenth bar in the right column of Figure shows the RBC value for the direction of Fault. We can see that this fault has the largest RBC value with the three fault detection indices. The detection and diagnosis results for Fault are shown in Figure 3. Since this is a bias fault in T, Variable, it is expected that this variable has the largest RBC value, which actually happens for the SP E and indices. Again, the T index does not detect nor diagnose this fault correctly. Figure 4 shows the results for the fault in the sensor C AA, Variable 3. In this case all detection indices show C AA as the cause of the problem, however the SP E and combined indices make a better detection of the fault. In Figure 5 we can see the detection and diagnosis results for the slow drift on the reaction kinetics. In this case the and combined indices start growing up slowly until they become very large compared to their control limits; this is not the case of T since it does not change much. Looking only at the first nine bars of the diagnosis results, they would point to the cooling water flow, Variable 6, as the origin of the fault. However, if we have information of this fault from past experience, we can calculate RBC values along its direction; T value value value T value value value 5 T T Fig.. Fault detection indices and RBC values for Fault 5 T T Fig. 3. Fault detection indices and RBC values for Fault the average of these values is shown in the last bar of the plot. It can be seen that the RBC values are the largest ones along this direction; thus, we can diagnose this fault as a drift in the kinetics. VI. CONCLUSIONS It is shown in this paper that reconstruction-based contribution can be used with kernel PCA for the diagnosis of faults in nonlinear processes. The proposed RBC method works like the standard contribution plots in linear PCA, which does not require the fault direction to be known beforehand. Further, contributions along sensor directions as well as known process fault directions can be obtained with RBC. The combined index is proposed to monitor the whole feature space and new control limits are proposed for the three fault detection indices in the feature space. The detection and diagnosis power of the proposed methods has 76

6 T value value value T value value value 5 T T Fig. 4. Fault detection indices and RBC values for Fault 3 5 T T Fig. 5. Fault detection indices and RBC values for Fault 4 been tested with the simulation of a CSTR process with sensor and process faults. It was showed that, while the and T indices sometimes disagreed on their results, the combined index was effective to correctly diagnose simple and complex faults. Several open issues deserve further study. One is the high dimensionality of the feature space when the number of samples is very large. Another issue is the selection of the radius of the kernel function. This parameter, while being determined by trial and error at the moment, affects the nonlinearity of the mapping from the data space to the feature space. VII. ACKNOWLEDGEMENTS We appreciate the financial support from the Roberto Rocca Education Program and from the Texas-Wisconsin- California Control Consortium. REFERENCES [] C. F. Alcala and S. J. Qin. Reconstruction-based contribution for process monitoring. Automatica, 45(7):593 6, 9. [] G.E.P. Box. Some theorems on quadratic forms applied in the study of analysis of variance problems, I. effect of inequality of variance in the one-way classification. Ann. Math. Statistics, 5:9 3, 954. [3] L.H. Chiang, E.L. Russell, and R.D. Braatz. Fault diagnosis and fisher discriminant analysis, discriminant partial least squares, and principal component analysis. Chemometrics Intell. Lab. Syst., 5:43 5,. [4] S. W. Choi, C. Lee, J. Lee, J. H. Park, and I. Lee. Fault detection and identification of nonlinear processes based on kernel PCA. Chemometrics Intell. Lab. Syst., 75:55 67, 5. [5] D. Dong and T. J. McAvoy. Nonlinear principal component analysis - based on principal curves and neural networks. In Proceedings of American Control Conference, Baltimore, Maryland, June 994. [6] R. Dunia and S. J. Qin. A unified geometric approach to process and sensor fault identification: the unidimensional fault case. Comput. Chem. Eng., :97 943, 998. [7] R. Dunia, S. J. Qin, T. F. Edgar, and T. J. McAvoy. Identification of faulty sensors using principal component analysis. AIChE J., 4:797 8, 996. [8] H. G. Hiden, M. J. Willis, M. T. Tham, P. Turner, and G. A. Montague. Non-linear principal components analysis using genetic programming. In Genetic Algorithms in Engineering Systems: Innovations and Applications, Glasgow, UK, September 997. [9] Mark Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE J., 37():33 43, 99. [] J. Lee, C. Yoo, and I. Lee. Fault detection of batch processes using multiway kernel principal component analysis. Computer Chem. Eng., 8: , 4. [] P. Nomikos and J.F. MacGregor. Multivariate SPC charts for monitoring batch processes. Technometrics, 37():4 59, 995. [] S. J. Qin. Statistical process monitoring: Basics and beyond. J. Chemometrics, 7:48 5, 3. [3] B. Schölkopf, A. Smola, and K. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, :99 39, 998. [4] B.M. Wise and N.B. Gallagher. The process chemometrics approach to process monitoring and fault detection. J. Proc. Cont., 6:39 348, 996. [5] S. Yoon and J.F. MacGregor. Fault diagnosis with multivariate statistical models, part I: using steady state fault signatures. J. Proc. Cont., :387 4,. [6] H. Yue and S. Joe Qin. Reconstruction based fault identification using a combined index. Ind. Eng. Chem. Res., 4: ,. [7] Y. Zhang and S. J. Qin. Fault detection of nonlinear processes using multiway kernel independent analysis. Ind. Eng. Chem. Res., 46: , 7. [8] Y. Zhang and S. J. Qin. Improved nonlinear fault detection technique and statistical analysis. AIChE J., 54:37 3, 8. VIII. APPENDIX A. CSTR Simulation Parameters and Initial Conditions The CSTR simulation parameters are V = (m 3 ), ρ = 6 (g/m 3 ), ρ C = 6 (g/m 3 ), E/R = (K), c p = (cal/g K), c pc = (cal/gk), b =.5, k = (m 3 /kmole min), a = (cal/min K), H rxn =.3 7 (cal/kmole). The parameters of the temperature PI controller are K C =.5 and T I = 5.. The initial conditions for the simulation are T = 37. (K), T C = 365. (K), F C = 5 (m 3 /min), T = (K), F s =.9 (m 3 /min), F A =. (m 3 /min), C A =.8 (kmole/m 3 ), C A S =. (kmole/m 3 ), C AA = 9. (kmole/m 3 ). Gaussian noise with small variance was added to the measurements and disturbances. For more details of the simulation check [5]. 77