Comparison between MSE and MEE Based Component Extraction Approaches to Process Monitoring and Fault Diagnosis

Transcription

1 Coparso betwee MSE ad MEE ased Copoet Etracto pproaches to Process Motorg ad Fault Dagoss Zhehua Guo, ad Hog Wag $ School of Mechacal Scece ad Egeerg, Huazhog Uversty of Scece ad echology, Wuha, Hube, PC $ Departet of Electrcal Egeerg ad Electrocs, Uversty of Machester, UK bstract Copoet etracto s a techque for etractg the latet copoets that uderle the observato of a set of varables. I the paper both classcal Prcpal copoet aalyss (PC) ad autoassocatve prcpal copoet eural etwor (PC) ethods wth u ea square error (MSE) crtero are copared wth the correspodg eteded copoet etracto ethods wth Mu error etropy (MEE) crtero theory. Parze wdow estator based approatve coputato ethod for etropy s provded, ad the equvalece betwee MSE ad MEE crtera s also aalyzed. Key words: prcpal copoet aalyss; u square error; u error etropy; fault dagoss, eural etwor. IODUCIO Varous ultvarate statstcal techques have bee eployed for revealg the statstcal characterstcs of processes, cludg PC (Jollffe, 00), prcpal copoet eural etwors (PC) (Daataras ad Kug, 996), Fsher dscrat aalyss (He et al., 004) ad depedet copoet aalyss (IC) (Hyväre et al, 00) et al. Multvarate statstcal process cotrol (MSPC) techques based o PC ad ts etesos are the ost wdely appled process otorg ad fault dagoss (MacGregor ad Koutod, 995, oos ad MacGregor, 995). PC s a effectve statstcal techque for data copresso ad copoet etracto wth the optal perforace the sese of u ea square error (MSE), ad the etracted prcpal copoets are the cobatos of varables that descrbe ajor treds a data set. Low desoal features eable the process cotrol ad otorg to use the less coputatoal resources ad eory wthout coprosg the accuracy. Sce the Gaussa probablty desty fucto (PDF) s detered oly by ts frst- ad secod-order statstcs. Uder these learty ad Gaussaty assuptos, further supported by the cetral lt theore, MSE crtero would be able to etract all possble forato fro a sgal whose statstcs are solely defed by ts ea ad varace. herefore, MSE s a very sutable de for Gaussa dstrbuto systes, but ay be approprate for the o-gaussa dstrbuto systes. However, achevable perforace of PC s lted aly due to the assupto that the ultvarate data are orally dstrbuted. o further prove the aalytcal perforace for o-gaussa dataset, several MSPC ethods based o hgh order statstc ad eural etwor have bee proposed (Daataras ad Kug, 996; Grola, 999), cludg olear prcpal copoet eural etwor ad depedet copoet aalyss (IC). Etropy s a alteratve geeral crtero for the easure of the ucertaty o-gaussa syste, ad ca be used as a alteratve crtero to the MSE crtero (aylor ad Plubley, 993, Wag, 00).. PC OF MIIMUM SQUE EO CIEI Cosder a cotuous rado vector = [ L ] wth the ea E { } = 0 =.. If the E ad covarace atr egevalues of the covarace atr are ascedgly ordered as λλl λ > 0 ad the correspodg egevectors are e, e,l, e, the trasfor y e = s called th prcpal copoet (PC) or score ad the egevector e s called prcpal egevector or loadg (Daataras ad Kug, 996, Hyväre et al., 00). For ay th ad j th PCs, they have the varace var( y ) = λ ad the covarace cov( y, yj) = 0 ( j). herefore, the frst prcpal copoet has the largest varace ad the prcpal copoets are utually ucorrelated. PC ca also be preseted fro dscrete saple data. For a dscrete observato atr X= [, L, ] that cossts of

2 observatos wth varables, PC decoposes X to the su of the outer product of pars of vectors as follows X = t p + t p + L + t p = P () Let the egevalues of covarace atr = E{ XX } be arraged decreasg order λ λ L λ, ad let ther correspodg oralzed egevectors be deoted by e, e, L, e. ccordg to the aylegh-tz theore (Daataras ad Kug,996), these egevectors ad egevalues are equal to the loadgs ad the varaces of prcpal copoets (PCs) Equato ().. hat s e = p ; λ = var( t ) () Dfferet crtera had bee proposed the past for the selecto of the optal uber of PCs.. I ths paper the cuulatve percet varace (CPV) (Valle et al., 999) based o the cuulatve varace cotrbuto rate of the frst several PCs s used to choose the uber of PCs, whch s defed as follows CPV var( t ) = = var( t ) j= j j= = = λ λ j (3) where s the uber of PCs reserved. Whe PC s perfored o a data atr, t s ofte foud that oly the frst PCs are assocated wth syste varato the data ad the reag PCs are assocated wth the ose. Wth these reserved PCs, whch are descrptve of syste varato, the followg PC odel ca be forulated. ˆX = t p + t p + L + t p = P (4) E = t p + t p + L + t p (5) where ˆX s the estator of X, ad E s the correspodg resdual error atr. Multvarate cotrol charts are used to detect shfts the eas of varables or the relatoshp (covarace) betwee several related paraeters. he ost etesvely appled ultvarate cotrol charts are squared predcto error ( SPE ) chart, Hotellg s cotrol chart, cotrbuto charts ad loadg chart (MacGregor ad Koutod, 995, oos ad MacGregor, 995). SPE also called Q statstc ad Hotellg s statstc are ofte used to detect shfts the process. Q statstc offers a way to test f the process data has shfted outsde the oral operatg space, whle statstc provdes a dcato of uusual varace wth the oral subspace. For a ew observato, Q ad statstcs are defed as follows. Q = ( I P P ) (6) = PΛ P (7) Λ =, L, where dag{ λ λ}. he upper cotrol lts (UCLs) for SPE ad wth α cofdece ca be calculated respectvely as follows(macgregor ad Koutod, 995, oos ad MacGregor, 995, Jacso ad Mudholar, 979).. Q α Cα θ h0 θ h0( h0 ) = θ + + θ θ ( ) α = F α ( ),,,, j= + h0 (8) (9) θ = λj ( =,, 3) (0) h = θ θ 3 0 3θ () ad are the uber of varables or PCs ad the uber of saples respectvely. s the uber of the reserved PCs. λ s the egevalue of covarace atr = E { XX }. C α s the upper α th quatle of stadard oral dstrbuto. F,, α represets the F dstrbuto wth freedo paraeters ad. fter perforg PC o a rado vector wth the ea E { } = 0 ad covarace atr, ad reservg the frst prcpal copoets ad the correspodg orthogoal egevectors deoted by W, the followg prcpal copoet vector y ca be obtaed. y = W () Deote the -desoal subspace spaed by W as spa( W ). he projecto of oto subspace spa ( W ), whch s the recostructo of fro y, ca be epressed as follows (Daataras ad Kug, 996, Hyväre, et al., 00). ˆ = W y (3) PC sees to ze the ea square recostructo error (Daatara ad Kug, 996).. { } J = ˆ tr ( ˆ)( ˆ) MSE E = E (4) where tr( ) s the trace operator. ccordg to the fact that tr( ) = tr( ), ad, Equato (5) ca be further wrtte as J se = tr

3 J se = tr( ) tr(w W ) (5) { yy } E{ ˆˆ } tr( W W ) = E tr( ) = tr( ) (6) 3 PC OF MIIMUM SQUE EO eural etwor () provdes a ovel way for parallel ole coputato of PC, ad a uber of ectg advatages have bee obtaed. auto-assocatve eural etwor has the feature of the sae uber of euros the put ad output layers ad the less uber of euros the hdde layers, s traed usg the put vector tself as the desred output. hs trag leads to orgaze a copresso or ecodg etwor betwee the put layer ad the hdde layer, ad a decodg etwor betwee the hdde layer ad the output layer. It turs out that there s deed a close relatoshp betwee PC ad etwors of ths type. Deote the put of etwor as X ad the output of hdde layer as H. he output of etwors, whch s the recostructo of X, ca be wrte as ˆX = H + θ u (7) W = F ( W + θ ) (8) H X u where u = [,, L, ] ad F( ) s the olear actvato fucto operatg o each eleet of the vector. W ad W are the weght atrces of the hdde ad the output layers respectvely. θ ad θ are the bas vectors of the hdde layer ad the output layer respectvely. Usg the square error cost, the perforace de becoes (Daatara ad Kug, 996) ˆ J = X X = X W H θ u = = W h θ (9) he learg algorth for the etwor s to select the optal W, W, θ ad θ, so that the square error cost s zed. Settg J = ( W h θ ) = 0 (0) θ = he optcal θ ca be obtaed θ = ( W h ) = ( X W H ) u = whch subtted to the cost fucto yelds () J = X ( I uu / ) W H ( I uu / ) = X W H 3 X = X ( I uu / ), ad H = H ( I uu / ). () Sce W ad ra( W ), ra( W H ), the proble of zato of J s equvalet to the proble of the best ra- p approato of X. Usg sgular value decoposto (SVD), X ca be decoposed as X = U Σ V (3) σ σ L σ L, where Σ = dag[,,,, 0,, 0] ad σ σ L σ. σ s the sgular value of X. U ad V are the atres cotag the correspodg sgular vectors. ccordg to the results (chloptas ad McSherry, 00), f U ad V are fored by the frst colus fro U ad V respectvely, ad the dagoal atr Σ = dag[ σ, σ, L, σ ] s the partal atr cotag the largest sgular values, the followg epresso s the best ra- p approato of X. W H = U Σ V (4) W = U Λ; H = Λ Σ V (5) where Λ = s ay osgular atr.. 4. PC OF MIIMUM EO EOPY Etropy s a probablstc easure of ucertaty (Grola, 999). Let be a cotuous rado vector wth a probablty desty fucto p ( ). he dfferetal etropy of s defed as H( ) = p( )log p( ) d (6) If a -desoal Gaussa dstrbuto vector wth zero ea ad covarace atr. H ( ) = log( π e) (7) where deotes the deterat of. For two cotuous rado vectors ad y, the codtoal etropy H ( y) s defed as (8) H ( y) = p (, y)log p ( y) d d y where p ( y) s the codtoal probablty desty of gve y. he utual forato betwee two cotuous rado vectors ad y wth jot

4 probablty desty p (, y) s defed as p( y, ) I ( ; y) = ( y)log y p, d d p( ) p( y) (9) where p ( ) ad p ( y) are the argal probablty desty for ad y respectvely. For a = [ L ], the utual rado vector forato ca be wrote as I(,, L ) = H( ) H( ) (30) = Fro the deftos of etropy, codtoal etropy ad utual forato, the followg equato ca be got (Grola, 999). I( y ; ) = H( ) H( y) = I( y ; ) (3) Cosder a rado vector wth saple dataset, the Parze-wdow estator ca be wrtte as follows (Parze, 96). pˆ(, ) = κ( ) = E[ κ( )] (3) s a saple, where s the uber of saples used to estate the desty ad E [] s the ea values evaluated fro the saples. κ () s the erel fucto, whch usually uses Gaussa probablty desty fucto. ccordg to the PDF estator Equato (3), the etropy Equato (8) ca be rewrtte as follows. H( ) E[ log pˆ (, ) ] Substtutg the tegral E [ log pˆ ( ) ] (33), wth the ea value of the saples, the approatve etropy ca be further epressed as [ ˆ ] H ( ) E log p(, ) = log κ ( ) (34) where s the uber of saples used to estate desty, whle s the uber of saples used to approate etropy.. Let us cosder -desoal rado vector wth zero ea ad covarace atr = E{ }. Fro the PC odel gve by Equato (4) the followg prcpal copoet vector ca be obtaed. y = W (35) where W s a orthogoal atr whose. Uder the th colu s the th egevector of 4 -desoal Gaussa dstrbuto assupto for, y also s -desoal Gaussa dstrbuto wth covarace atr = W W. he etropy of y ca be calculated va Equato (7). pparetly, H ( y ) s proportoal to the deterat of covarace atr y. Usg the results (Jollffe, 00), t ca be see that deterat of y s the au. herefore, zg the ea square error or azg the PCs varace for Gaussa syste by PC s equvalet to the zato of the error etropy or the azato of the odel etropy. Sce etropy has ore geeral eag tha that of varace for arbtrary rado varables, t ca be used to easure the ucertaty of rado varables ad for a desg crtera for geeral stochastc syste subjected to arbtrary dstrbuto (Wag, 00). ased o ths dea a odfed PC techque wth u error etropy (MEE-PC) has bee proposed (Guo ad Wag, 004) MEE-PC perfors a covetoal PC o the data ad chooses the uber of PCs reserved va CPV, ad the optzes the correspodg loadg vector by a geetc algorth wth the u errors etropy. o be ore specfc, after PC wth the frst reserved PCs, the PC odel as Equato (4) ca be obtaed. I order to forulate the best odel wth u error etropy, the reserved loadg egevectors P shall be optzed va a geetc algorth. For the coveece of chroosoe codg geetc algorth, the reserved loadg egevectors P are rearraged as a value vector G for the optzato. P p g p g G M M p g = = (36) where there s a appg of eleets betwee p ad G as follows. p p g ( ) + p g ( ) + M L p g = (37). I order to further optze the error etropy, a optzato odel wth a rado vector pertet to the loadg egevectors Equato (36) shall be bult. hs rado vector ca also be uquely coverted bac to a rado loadg egevector atr. Deotg the rado vector as U ad ts relevat loadg egevector atr as L, the followg

5 optzato odel ca be forulated. u l l L l u l l l L U= L= L L L L L u l l l L (38) I order to optze the rado varable U va the geetc algorth, the doa of the rado varable shall be decded. For the reaso that the rado varables are used to deterate the drectos of loadg egevectors, ther values are proportoal to each other. he doa for the rado vector U Equato (38) ca be set to [0.5G, 5. G],.e. u [0. 5g,. 5 g] ( ( )). I the eate the loadg egevectors L shall also be subjected to ll j = 0 ( j; j, ( )) ll = ( = j; j, ( )) j (39) he ore detal troducto ad pleetato of ths MEE-PC ethod ca be refereed fro (Guo ad Wag, 004). 5. SED PC WIH MIIMUM EO EOPY I ths secto a auto-assocatve eural etwor wth a topology slar to autoassocatve MSE-based PC odel ad the correspodg learg algorth are preseted. Let ad ˆ be the put ad the output of the eural etwor, ad ( ) ad F( ) be the output ad the olear actvato fucto of hdde layer respectvely. he followg trastoal equatos ca be obtaed. h = F( + θ ) (40) ˆ W = h + W θ (4) ε = ˆ = W h + θ (4) where W ad W are the weght atr of the put-hdde layer ad the weght atr of the hdde-output layer. θ ad θ are the bas of the put-hdde layer ad the hdde-output layer. ε s the resdual error vector. If the Gaussa dstrbuto erel s used, the followg approato of error etropy ca be derved. H ( ε ) log GΨ ( ε ε ) ε ε h (43) G Ψ ( π ) det( Ψ ) ( ε Ψ ε ) ( ε ) = ep ( W ) ε = F + θ + θ (44) W ε ( + θ ) = W F W + θ (45) where ε ad ε are the saples belogg to saple dataset ad. Ψ s the covarace atr of Gaussa dstrbuto desty fucto whch ca be estated fro saples (ell ad Sejows, 995). y usg the gradet descedg algorth slar to coo bac propagato algorth, Equato (43) ca be derved by a pseudo arguet Λ as H ( ε) Λ ( ε ε ) GΨ ( ε ε) ε d Λ G ε ε Ψ G ( ε ε ) Ψ = DΨ ( ε ε) ε ( ) ε G ε ε ε Ψ Λ (46) where DΨ ( ε ε) = ( ε ε) Ψ ( ε ε). Sce there s the fact that ( ε Ψ ε ) ε = Ψ ε (47) herefore, f let Λ be the weght atr W, the dervatve of error etropy to W ca be epressed as W D Ψ ( ε ε ) = D ( ε ε ) ( ε ε ) Ψ ( ε ε ) W { ( W ) ( W ) } ( ε ε) F θ F θ = Ψ W + + Slarly, the dervatves of Equato (43) to W, θ ad θ ca also be obtaed. 6. COCLUSIOS PC s the ost etesvely appled copoet etracto techque wth the optal perforace the sese of MSE. However, MSE whch oly cosders the secod-order statstc s a sutable crtero for easurg the ucertaty of a lear ad Gaussa stochastc process. ut for the olear ad o-gaussa syste, etropy s a alteratve crtero for easurg the ucertaty. I ths paper MSE ad u error etropy (MEE) based copoet etracto techques are descrbed detal, ad the equvalece betwee MSE ad MEE crtera s also aalyzed fro forato theory. 5

6 cowledgeets: hs wor s supported by SFC of Cha ( ad ) ad the project (0805) wth ortheaster Uversty of Cha where the secod author s afflated to. hese are gratefully acowledged. EFEECES chloptas, D. ad McSherry, F. (00), Fast coputato of low ra atr approatos, Proceedgs of the thrty-thrd aual CM syposu o heory of coputg. (ew Yor: CM Press), ell,. ad Sejows, J. (995): forato-azato approach to bld separato ad bld decovoluto, eural Coputato 7, Daataras, K.I. ad Kug, S.Y. (996): Prcpal Copoet eural etwors: heory ad pplcatos. ew Yor : Joh Wley & Sos,. Grola, M. (999): Self-Orgasg eural etwors : Idepedet Copoet alyss ad ld Source Separato, Lodo: Sprger -Verlag,. Guo, Z. H. ad Wag, H. (004): odfed PC based o the u error etropy, Proc. CC 04, osto, US,. He, Q. Peter, Wag, J. ad Q, S. J. (004): ew fault dagoss ethod usg fault drectos Fsher dscrat aalyss, eas-wscos Modelg ad Cotrol Cosortu ech. eport Hyväre,., Karhue, J. ad Oja, E. (00): Idepedet Copoet alyss, ew Yor: Joh Wley & Sos. Jacso, J. E. ad Mudholar, G. S. (979): Cotrol procedures for resduals assocated wth prcpal copoet aalyss, echoetrcs, Johasso, K. H. (000): he quadruple-ta process: a ultvarable laboratory process wth a adjustable zero. IEEE rs. Cotrol Systes echology, 8, Jollffe, I.. (00): Prcpal Copoet alyss, d edto, ew Yor: Sprger-Verlag,. Lser,. (988): Self-Orgazato a Perceptual etwor, IEEE Coputer,, MacGregor, J. F. ad Koutod, M. (995): Statstcal process cotrol of ultvarate processes. Cotrol Egeerg Practce 3, oos, P. ad MacGregor, J. F. (995): Multvarate SPC charts for otorg batch process. echoetrcs, 37, Parze, E. (96): O the estato of a probablty desty fucto ad the ode, als of Matheatcal Statstcs, 33l, aylor J. ad Plubley M. (993), Iforato heory 6 ad eural etwors Matheatcal pproaches to eural etwors, J. aylor, (Ed.), Elsever Scece Publcato., sterda, Valle, S., L, W. ad Q, S. J. (999): Selecto of the uber of prcpal copoets: the varace of the recostructo error crtero wth a coparso to other ethods. Idustral ad Egeerg Chestry esearch, 38, Wag H. (00): Mu etropy cotrol of o-gaussa dyac stochastc systes. IEEE rs. utoatc Cotrol, 47,