GEOMETRIC PROPERTIES OF KERNEL PARTIAL LEAST SQUARES FOR NON- LINEAR PROCESS MONITORING

Transcription

1 GEOMETIC POPETIES OF KEEL PATIAL LEAST SQUAES FO O- LIEA POCESS MOITOIG José L. Godo () Germán Bustos (b) Alejndro H. Gonzlez () nd Jinto L. Mrhetti (d) (bd) ITEC (COICET nd Universidd ionl del Litorl) Güemes 450 (000) Snt Fe Argentin () jlgodo (b) gbustos () lejgon (d) ABSTACT This work proposes new strteg for monitoring nonliner proesses bsed on Kernel Prtil Lest Squres (KPLS). When strongl non-liner proess re onsidered PLS regression model ould not be enough urte. So the first stge of the proposed method is to mp the input dt to high-dimension spe where liner regression model n be obtined. Then n impliit liner regression model relting the high-dimension spe with output spe (output dt) is obtined. This model impliitl indues deomposition of the high-dimension spe into the Model subspe nd the omplementr esidul subspe being the vetors in the first subspe the effetive domin of the liner regression model. Finll one the spe deomposition is understood new sttistis (metris) for eh subspe re proposed to monitor the proess nd detet possible bnorml behviours. The effetiveness of the method is tested b mens of sntheti simultion exmple from the literture. Kewords: KPLS spe deomposition multivrite proess monitoring fult detetion indexes.. ITODUCTIO KPLS is promising regression method for del with nonliner problems beuse it n effiientl ompute regression oeffiients in high-dimensionl spe b mens of the nonliner kernel funtion. It is lso n effiient method for estimting nd prediting qulit vribles in the strongl nonliner proess b mpping dt from the originl spe into highdimensionl spe. It onl requires the use of liner lgebr mking it s simple s liner multivrite projetion methods nd it n hndle wide rnge of nonlinerities beuse of its bilit to use different kernel funtions. Its pplition results from simple exmple show tht the proposed method n effetivel pture the nonliner reltionship mong vribles. The need for ssoiting input nd output dt sets obtined b online dt login of omplex proess vribles onstitutes problem tht requires inresing ttention. Ltel KPLS hs beome powerful pproh to find multivrible non-liner strutures minl beuse lrge non-liner orreltions impliit in the dt n be dequtel overome. Fult detetion mkes use of the so-lled fult detetion indies bsed on model. A fult is deteted when one of the fult detetion indies is beond its ontrol limit. One fult is deteted it is neessr to dignose its use. In this work the impliit spe deomposition (mde b KPLS) is obtined nd their min geometri properties re used to design non-liner monitoring strteg using new fult detetion indies tuting on eh subspe. This llows us to dignose the fult tpe. The min ontributions of this work re then the introdution of new fult detetion indies derived from the KPLS deomposition nd dignosis tool bsed on the sttistis tht triggers the lrm ondition.. KEEL PLS EGESSIO As usul in dt driven method we first should ollet set of smples of the preditor vetor m x with x i for i nd smples of the { i} i= response vetor { } i i= with p for i i whih re lled Identifition Dt Set (IDS). One these dt re olleted the Kernel Prtil Lest Squres (KPLS) method mps the preditor vetors x i m from to high-dimension spe (with m) where liner PLS regression model n be reted to relte vetors in with the response vetors i in p. In this w ltent model n be formulted in in order to extend liner PLS to non-liner kernel PLS (osipl nd Trejo 00). The non-liner m trnsformtion tht mps vetors from to is not mde b mens of n expliit non-liner funtion φ but b mens of kernel funtion k seleted to ompute the following inner produts: k ( xi x j) = φ( xi) φ( x j) for i = nd j =. otie tht trough the introdution of the kernel trik (osipl nd Trejo 00) φ( x ) i φ( x ) = k j ( xi x j ) one n void both performing expliit nonliner mpping nd omputing dot produts in the highdimensionl spe. Furthermore s it is known nd will be seen lter the PLS regression method need onl dot produts to perform the regression. ow from Eq. () it is obtined the Grm Kernel mtrix K defined s follows: K = ΦΦ Φ= φ( x ) φ( x ). () () 67

2 The entries of this lst mtrix re the ross inner { i } produts of ll mpped preditor vetors φ x. As in the PLS method it is ssumed here nonliner KPLS model with zero men. To enter the mpped dt in the high-dimensionl spe mtrix K should be substituted b the entered mtrix K given b: K = ΦΦ = KKE EK+ EKE () where E is mtrix with unitr entries nd K nd Φ re the entered versions of K nd Φ. ow from mtries K nd Y= [ ] p the trining KPLS lgorithm is derived s sequene of modified IPALS steps s follows (osipl nd Trejo 00): First set the index = K = K nd. Then:. Set the vetor of Y -sores u s the mximum-vrine olumn of Y.. Clulte the vetor of Φ -sores t = Ku t t t. egress olumns of Y on t : = Y t where is weight vetor. 4. Clulte the new sore vetor u for Y s: u = Y u u u 5. epet steps - until onvergene of t. 6. Deflte the mtries: K+ ( Itt ) K( Itt ) Y + Y tty 7. Set =+ nd return to step. Stop when >A where A is the number of ltent vribles seleted in the high-dimensionl spe. i= otie tht the seletion of A is determined b supervising the defltion of Y. Agin s in liner PLS the predition on trining dt hs the following from (osipl nd Trejo 00): Yˆ = ΦBPLS = ΦΦ U( TKU ) TY (4) = KU TKU TY = TTY = TC where the mtries T= [ t... ta] U = [ u... ua] nd C= [... A] re orthogonl b olumns. otie tht the regression oeffiient mtrix B PLS exists but is never omputed b the KPLS lgorithm sine the kernel substitution voids the neessit of n expliit omputtion. Equlit (4) shows tht the output response n be obtined from the inner produts of the mpped preditor vetors. So for new observtion x of the preditor vetor the response vetor will be given b: = φ x B = ku TKU TY PLS = kc = km where k is the vetor of entered kernel funtions evluted in the pirs (x x j ) with j=. From Eq. () this vetor is given b: k = k k x ( x ) = k Ke Ek EKe + (6) where e is n unitr vetor nd k = k k x ( x) is the kernel funtions vetor where the element j is given b the kernel funtion evluted t (x x j ) i.e. k x = k x x. j ( j ) Eh of the elements of vetor k is omputed s follows: kj( x) = k ( xj x) = k( x x) k( xj x) j= (7) + k ( xj xn) j= n= ow we fous our ttention on the ltent vetors A t. From Eq. (4) it follows tht for new observtion x this vetor will be given b: t = [ t t A ] = k (8) where t = kr =...A with = [ r ra ]. Therefore for new observtion x the predition n be omputed from t s (Eq. (5)): = tc. (9) ow given new mesurement preditor vetor x in originl units the response vetor (lso in originl units) is predited b: = DMk ( Dx ( x x) ) + (0) where the smple stndrd devitions D ( ˆ... ˆ x = dig σ x σ ) x m D ( ˆ... ˆ = dig σ σ ) p nd the mens x re determined in the trining proedure.. KPLS-BASED MODELIG FO POCESS MOITOIG The Kernel PLS lgorithm indues n externl model whih deomposes Φ nd Y in sore vetors t weight vetors (p nd ) nd residul error mtries ( Φ nd Y ) s follows: Φ = TP + Φ () Y= TC + Y () In greement with the stndrd liner PLS model (Godo et l 0) it is ssumed tht the sore vribles t re good preditor of Y. Furthermore it is lso ssumed liner inner reltion between the sores of t nd u tht is (5) U= TB+ H () 68

3 where B is the A A digonl mtrix nd H denotes the mtrix of residuls. Equivlentl we hve UB HB = T where C= BQ with Q orthonorml b olumns. Then the following regression kernel-bsed model is obtined: Y = KC + HB C + Y = Y+ Y+ Y Y = TC + Y + Y (4) where Y = Y + Y. We ll V the pseudo-inverse of P ( VP = PV = I); then the predition of T is diretl obtined from Eq. () s: where P = Φ T = Φ K with P nd V orthogonl b olumns. The oblique projetions tht pper in Eq. (6) deompose the high-dimensionl spe into two omplementr subspes W M nd W i.e. WM W (Godo et l 0). Mtrix PV is the projetor onto the model subspe W Spn P long the residul subspe M { } { } W Spn V diretion (Godo et l 0) where denotes the orthogonl omplement of the subspe (notie tht rnge of P is W M nd the nullspe is W ). Hene mpped preditor spe is deomposed in n underling form b KPLS into omplementr oblique subspes. T = ΦV (5) In ddition if we generlize the results of Godo et l (0) for the se KPLS response spe is lso deomposed (b KPLS) in omplementr oblique P i.e. ΦV = 0 (Godo et subspes whih is relted with the preditor modeled subspe ording to: beuse the row spe of Φ belongs to the null-spe of the liner trnsformtion l 0). If P= WAΣ AV A is the ompt singulr vlue deomposition (SVD) of P then V ( P ) = VAΣ AW A where denote generlized inverse (Meer 000). Equivlentl from Eq. (4) D is the pseudo-inverse of C ( CD = DC = I) nd sine YD= 0 then: T+ HB = YD. An interesting point of the propose proedure is tht the mtries V P B nd Q will never be estimted (otherwise it would be imprtil). The re onl defined to develop the proof tht follows in order to find metris bsed on kernel substitution trik... Underling KPLS deomposition of the input mpped nd output spes p = + = CD S Spn C MY { } ( ) { = ICD S Spn D} (7) Y = + = Ck ( x ) SMY ˆ ˆ = = CD Ck ( x) SMY (8) Figure illustrtes prt of underling spe deomposition developed in this work. Eh mpped mesurement vetor is deomposed nd their projetions re ompred with their ontrol limits. After snthesizing n in-ontrol KPLS model the p mesurement vetors φ( x ) nd re (underling) deomposed s desribed below. First the ltent vetor is omputes s k = φ( x) V = t (see Eq. 5) where V= [ v va] is the mtrix of PLS omponents given b v = α jφ( x j ) with α j. Then new mpped j= vetor φ( x ) (ssoited to the mesurements x) n be deomposed s: ˆ ( ) φ x = φ x + φ x ˆ φ x = PVφ x = Pt WM (6) φ x = IPV φ x W 4. POCESS MOITOIG BASED O KPLS 4.. Fult detetion indexes The multivrite proess monitoring strteg uses sttistil indexes ssoited to different subspes for fult detetion purposes. Bsed on the in-ontrol KPLS model it n be nlzed ever future behvior of the proess b mpping the new observtions x to the modeled subspe nd to the (omplementr) residul subspe x φˆ ( x) + φ ( x). In eh of these subspes it is possible to mesure distnes independentl. However sine no expliit formuls re vilble for the projetions of Eq. (6) it must be found new sttistis using the kernel substitution to obtin (estimted) norms for the projeted vetors. 69

4 on-liner funtion pproximtion i = f x i x Pired preditors { i} i= Model Subspe In-ontrol set { } i i= Input Spe High-dimension Spe ( m) Projetion onto W M WM Liner Trnsformtion C m p ˆ A W φ x = PVφ x = Pt t O Mpeo φ (. ) W M m x φ( x ) Projetion onto W esidul Subspe W Output Subspe = Bφ ˆ x = Ct W O = ( ) W φ x I PV φ x Figure : KPLS intrinsi rhiteture showing the input underling deomposition with their reltions nd ontrol limits. For instne to detet signifint hnge in W M the following Hotelling s T sttisti for t is defined: T x = tλ t = ( ) k k (9) t where Λ = ( ) TT = ( ) I. When new speil event (originll not onsidered b the in-ontrol KPLS model) ours the new mpped observtion φ( x ) will move out from W M into W. The squred predition error of φ ( x) (SPE X ) or distne to the φ( x) -model is defined s: SPE X ( x) = φ ( x) = φ( x) PVφ ( x) = φ x φ x φ x φ x + φ x φ x ˆ ˆ ˆ ( xx ) kkt tt = k + where ˆ (0) φ x φ x = φ x Pt = φ x Φ Kt = k Kt. Then SPE X n be used for deteting hnge in W. When the proess is in-ontrol the SPE X index represents the flututions tht n not be explined b the KPLS model. The distne from the regression model in S MY is defined s: SPE Y = = [ CD C ] () kx nd the distne from the -model in S Y is defined s: Y SPE = = ICD () Frequentl nd ˆφ ŷ re singulr. Then the generlized Mhlnobis distne for ˆφ nd ˆ re: D = φˆ φˆ () φˆ φˆ D = ˆ. (4) where the orreltion mtries re given b: φˆ ( ) = Yˆ Yˆ = C ( ) TT C = ( ) CC = ( ) ΦΦ ˆ ˆ = ( ) PT TP = ( ) PP (5) (6) Sine C is orthogonl b olumns the propert of the generlized inverse of SVD ields: ( ) ˆ = ( ) = ( ) C I C CC. Then repling this nd Eq. (9) into the Eq. (4) result the following equlit: Dˆ ˆ ˆ (( ) ) ( ) = = = = Tt ˆ tc CC Ct tt. (7) Similrl ˆ = ( ) PP (see Eq. 6). Then result: φ [ ] ˆ D ˆ T ˆ = φ ( ) PP φ = ( ) tt= t = D φ ˆ. (8) nd onsequentl the metris on φ ˆ ( x ) t or ŷ re equivlents. Hene the proess output (qulit vribles) n be monitored through PLS-bsed input sttisti. Then we propose monitoring using four non-overlpped metris (SPE X T t SPE Y nd SPE Y ) whih ompletel 70

5 over both mesurement spes eh one on different subspe. 4.. Fult dignosis b mens of lrmed subspes An noml is hnge in the mesurements following or not the orreltion struture ptured b the PLS model. If the hnge produes n out-ofontrol point the noml soure n be lssified ording to ) n exessivel lrge opertion hnge of the norml opertion; b) signifint inrese of vribilit; ) the ltertion of ross-orreltions nd d) sensor fults. Cses ) nd b) involve hnges in the mesurement vetor following the modeled orreltion struture; while ses ) nd d) involve hnges in some vribles ltering the orreltion pttern with the others. In ft n bnorml proess behvior involves devition of the modeled orreltions thus inresing the vlue of the proper metri being used. In order to lssif the bnormlities we nlze the effet on eh subspe (see Tble ). ows ) ) ) nd 6) feture omplex proess hnges; while rows 4) nd 5) represent lolized sensor fults. B nlzing the ontributions to n lrmed index (All nd Qin 0) suh s SPE X (or SPE Y ) it ould be possible to disriminte hnges in the X- / Y-outer prt ginst sensor fult in x / (see this mbiguit in Tble ). In summr the proposed monitoring strteg is bsed on n input nd output spe PLS deomposition whih lssifies the tpe of proess fult or noml ording to the sttisti tht triggers the lrm ondition. Tble. Fult dignosis bsed on lrmed index. Fult/Anoml in SPE X T t SPE Y SPE Y - Inner prt db - X outer prt dp o - Y outer prt dq o 4- x sensor o 5- sensor o 6- ltent spe upset : high vlue. : negligible vlue. o: high/low vlue. 5. APPLICATIO STUDY 5.. A on-liner umeril Simultion Exmple A simulted se stud is used to evlute the performne of the proposed monitoring tehnique for vriet of fult senrios. To understnd the impliit deompositions nd sttistis s monitoring tools we simulted different fults in sntheti sstem. We use the nonliner multidimensionl simultion exmple devised in efs. Zhng et l (008) nd Zho et l (006). It is defined s follows: x = t t+ + ε x = sin () t + ε x t t = x + x x + x + = + + ε os ε 4 where t is uniforml distributed over [- ] nd ε i i=4 re noise omponents uniforml distributed over [-0. 0.]. The generted dt of 00 smples re segmented into trining nd testing dt sets. The re illustrted in the Fig.. The first 00 smples re seleted for trining nd the subsequent 00 smples re used s testing dt set. It is pprent tht the input vribles re driven b one ltent vrible t onl in this se. From Fig. we n esil see tht the response vrible is nonlinerl orrelted with the input vribles. In this work we used the rdil bsis kernel ( i j) exp( i j ) k x x = x x h in our implementtion. When using this kernel funtion the vlue of prmeter h=σ =0.06 (σ=0.7) hs signifint influene on the KPLS predition performne (Zhng et l 008). The men squred error (MSE) is used to evlute the estimtor. Figure show the % MSE = 00 MSE (stndrd error) for eh omponent using trining dt or testing dt (or externl vlidtion dt). otie tht MSE() refers to MSE when the first ltent vribles re used. The seletion of n dequte number A of ltent vribles to be inluded in the KPLS model is ruil; if more thn neessr vribles re used n undesirble over-fitting might redue the preditive bilit. We use the Wold's 0.9 rule whih n be written s (+) = MSE(+))/MSE(). For suessive vlues the sequene is stopped when (+)>0.9; nd hene A=. Coneptull this riterion sttes tht n dditionl ltent vrible will not be inluded in the KPLS model unless it provides meningful predition improvement nd onsequentl it gives the mximum number on omponents to be inluded in the model. Figure b show the bove inditor vs. the number of omponents for trining dt nd testing dt. The rtio for trining dt is lrger thn 0.9 in += hene the most prsimonious model orresponds to A= (Zhng et l 008). The sme rtio with testing dt is for monitor overtrining when is seleting relible A vlue. Figures 4 show the predition results of the trining nd testing dt using the Eq. 0. The upper prt of Figure 4 shows the tul nd predited vlues nd the lower prt shows the errors between both vlues. Six fults were simulted whih re desribed in the Tble. Tble shows the dignosis expeted in eh simulted fult/noml. Figure 5 shows the time evolution of eh sttisti normlized b its ontrol limit whih lerl exhibit the six simulted bnormlities where it is possible to detet nd lssif eh tpe of simulted fult on the bsis of the informtion given in Tble. Sine (is not multivrible) then SPE Y 0 nd SPE ˆ Y =. Hene the -sensor fult will our in SPE Y (see Tble ). The simultion results show tht the developed strteg is ble to identif bnormlities ttributed to sensor fults proess hnges proess upsets nd disturbnes. The model supporting this monitoring pproh is bsed on norml operting dt. Proess 7

6 dt reolletion onstitutes ritil step when developing empiril models for monitoring. Figure 5: Time evolution of the ombined index nd eh normlized KPLS-sttisti. Figure. eltionship between inputs nd the response. Figure. KPLS Model order determintion. ) esidul error vrine vs. number of ltent vribles. b) tio of residul error vrines suessive. Figure 4. Predition results vi KPLS method. Tble : Simulted fult senrios. Tpe Smples Mgnitude Dignosis dx = -.5 / dx =.5 / dx =.5 / x = t t+ + ε / t =.0 (fixed) d = COCLUSIOS AD FUTUE WOKS Mn multivrite proess monitoring sstems ould be bsed on non-liner KPLS model tht represents inontrol onditions. As in more trditionl meningful devition of the vribles from their expeted trjetories serves for the detetion nd dignosis of bnorml proess behviors. The results of non-liner simultion exmple illustrtes tht the proposed strteg is effiient nd urte. However these results re preliminr more relisti pplitions re neessr for relible vlidting of the method nd to lern more bout the proposed non-liner strteg. ACKOWLEDGMETS The uthors re grteful for finnil support reeived from COICET MinCT Universidd ionl del Litorl nd Universidd Tenológi ionl (Argentin). EFEECES Godo J.L. Veg J.. Mrhetti J.L. 0. Geometri Properties of Prtil Lest Squres egression Applied to Proess Monitoring. III MACI 0. Bhi Bln (Argentin) 9 to m. Meer D Mtrix nlsis nd pplied liner lgebr. SIAM USA. Zho S. J.; Zhng J.; Xu Y. M.; Xiong Z. H onliner Projetion to Ltent Strutures Method nd its Applitions. Ind. Eng. Chem. es Zhng X. Yn W. nd Sho H onliner multivrite qulit estimtion nd predition bsed on kernel prtil lest squres. Ind. Eng. Chem. es. 47 (4) 0. osipl. nd Trejo L.J. 00. Kernel Prtil Lest Squres egression in eproduiing Kernel Hilbert Spe. J. of Mhine Lerning eserh