Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 Sesors & Trasducers 4 by IFSA Publsh, S. L. http://www.sesorsportal.com Soft Ses of Key State Varables Fermetato Process Based o Relevace Vector Mache wth Hybrd Kerel Fucto Xal ZHU, Xaofu JI School of Electrcal ad Iformato Eeer, Jasu Uversty, 3, P. R. Cha Receved: 7 May 4 /Accepted: 6 Jue 4 /Publshed: 3 Jue 4 Abstract: To resolve the ole detecto dffculty of some mportat state varables fermetato process wth tradtoal strumets, a soft ses model method based o relevace vector mache (RVM wth a hybrd kerel fucto s preseted. Based o the characterstc aalyss of two commoly-used kerel fuctos, that s, local Gaussa kerel fucto ad lobal polyomal kerel fucto, a hybrd kerel fucto comb merts of Gaussa kerel fucto ad polyomal kerel fucto s costructed. To des optmal parameters of ths kerel fucto, the partcle swarm optmzato (PSO alorthm s appled. The proposed model method s used to predct the value of cell cocetrato the Lyse fermetato process. Smulato results show that the preseted hybrd-kerel RVM model has a better accuracy ad performace tha the sle kerel RVM model. Copyrht 4 IFSA Publsh, S. L. Keywords: Hybrd kerel fucto, Relevace vector mache (RVM, Partcle swarm optmzato (PSO, Fermetato procedure, Soft sesor.. Itroducto The fermetato process s a very complex bochemstry procedure that s characterzed by multvarable, system coupl, ad stro olearty, sce some complcated bolocal, thermodyamc ad physcal reactos are volved smultaeously. Especally, the varable of cell cocetrato mrror the fermetato procedure qualty s dffcult to detect ole, whch makes t dffcult to apply ole optmal cotrol stratey. The tradtoal method to obta ths varable s ole sample ad offle aalyss whch s characterzed by tme-cosumpto ad delayed measuremet. For ths reaso, the tradtoal method dd ot meet the requremet of ole cotrol. Recetly, a ewly developed method called soft ses ca effcetly resolve ths problem [, ]. The dea of soft ses orated from the feretal cotrol theory where prmary varables dffcult to detect ole s umercally estmated by some assstat varables easy to detect throuh measur strumets [, 3]. The commoly used soft ses methods are based of the artfcal eural etwork (ANN ad support vector mache (SVM respectvely. For examples, Gozaa et. al. used ANN to measure key process varables ole polyethylee terephthalate (PET maufactur processes [], ad Desa et. al. bult the soft ses model of fed-batch fermetato procedures [4], ad Lu et. al. ave a soft ses model erythromyc fermetato processes [5]. The relevace vector mache (RVM s a recetlydeveloped lear alorthm based o the sparse Bayesa lear theory, whose structure s smlar to SVM [6]. However, ulke the SVM, the RVM model has some excellet merts such as hh http://www.sesorsportal.com/html/digest/p_rp_5.htm 37
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 sparsty, ood predcto precso, stro eeralzato ablty, ablty to ve probablstc model output, ad so o [6, 7]. The kerel fucto s a mportat elemet of the RVM that vtally affects the RVM performace. Ulke kerel fuctos of SVM, kerel fuctos of RVM are ot requred to satsfy the Mercer codto, whch makes the costructo of RVM s kerel fuctos more flexble tha that of SVM. The frequetly appled kerel fuctos of RVM are local kerel fuctos preseted by the Gaussa kerel fucto ad the lobal kerel fuctos preseted by the polyomal kerel fucto. For these two classes of kerel fuctos, the local kerel fucto has stro local lear ablty ad poor eeralzato ablty, whle the lobal kerel fucto s wth poor lear stablty ad stro eeralzato ablty. To resolve the ole measuremet problem of mportat state varables fermetato processes wth tradtoal strumets, a ovel soft ses model method based o RVM wth a hybrd kerel fucto s preseted ths paper. Comb merts of the local Gaussa kerel fucto ad lobal polyomal kerel fucto, a ovel hybrd kerel fucto s costructed, whose key kerel parameters are optmally desed us the partcle swarm optmzato (PSO alorthm. To resolve the ole detecto of cell cocetrato the Lyse fermetato process, the proposed model method s appled to buld soft ses model. The smulato ad expermet show that the desed RVM model wth hybrd kerel fucto acheves a better ftt performace ad eeralzato ablty tha RVM models wth Gaussa kerel fucto ad polyomal kerel fucto respectvely. The rest of ths paper s orazed as follows. Secto ves some fudametals of RVM ad the eeral RVM model procedure. Secto 3 descrbes typcal characterstcs of local kerel fucto ad lobal kerel fucto, based o whch, the basc dea of hybrd kerel fucto s ve Secto 4. I the secto, PSO alorthm s also appled to des the optmal kerel parameters. I secto 5, the proposed RVM soft ses method s used to predct the value of cell cocetrato the Lyse fermetato process, whose result verfes the effectveess of the preseted method. Secto 6 ves the codto of ths paper.. Relevace Vector Mache For a ve tra set {,} xt =, the objectve reresso fucto t of RVM s supposed to be a olear fucto cotamated by a whte Gaussa ose, t = y(, x w + e, ( where the ose e follows the Gaussa dstrbuto wth mea ad varace s, that s, p( e s ~ (, s. By follow the same phlosophy as that the dervato of support vector maches, RVM model expresses the reresso fucto yxw (, as a combato of kerel fucto Kxx (,, å w ( = yx (, w = Kxx (, + w where w = [ w, w, ¼, w ] T s the weh vector, ad the the lkelhood fucto of tra samples ca be ve as (, w = ( (, w, s = - = ( ps exp - t -Fw yx t yx where t = [ t, t, ¼, t ] T, ad æ ö ç çè s ø F= [ f( x, f( x, ¼, f( x ] é Kx (, x Kx (, x Kx (, x ù Kx (, x Kx (, x Kx (, x = ê Kx (, x Kx (, x Kx (, x ë ú û (3 The pror dstrbuto of weh parameters w, ( =,, ¼, s assumed to be Gaussa oes depedet o the hyper-parameter a, that s ( (, = p w a = w a - (4 where a = [ a, a, ¼, a ] T s the hyper-parameter vector determ the pror dstrbuto of weh vector w, ad further determ the sparsty performace of the obtaed RVM model. Accord to the Bayesa rule, the posteror dstrbuto of w ca be ve as, pt ( w, pw ( pw ( t, as, = pt ( as, - - T - ( p exp ( w m ( w m s (5 æ ö = S - - S - ç çè ø, where the posteror covarace of ths dstrbuto s T S= ( s - F F+ A -, the posteror mea s m = s SF T t, ad A s a daoal matrx wth A = da( a, a, ¼, a. By terat the lkelhood fucto (3 wth weh vector, we ca obta the ede dstrbuto of t depedet o a ad s as a 38
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 æ t T t pt as p - - W ö ç (,, = ( W expç - çè ø, (6 - T where W= s I +FA F. To obta optmal parameters a ad s wth maxmz (6, we ca calculate the partal dervato of (6 ad apply the terato method. Wth ths phlosophy, the updat equato of hyperparameters a ad s ca be ve as ( s a ew ew = = (7 m t -Fm, (8 å = where = -as, m s the th posteror mea, S s the th daoal elemet of the posteror covarace matrx S. It ca be see that Î [,] ca measure the ftess performace of weh w to tra samples. Throuh the terato (7, (8, m ad S are updated accordly. I ths procedure, most a, ( =,, ¼, wll approach to ad the correspod bass fucto f ( x wll be deleted, whch way, the hh sparsty of RVM s obtaed. For a ew put x *, the predcated dstrbuto of correspod output t * s ve as ( t t, a, s = ò pt ( w, s p( w t, a, s dw ~ (, * MP MP * MP MP MP t y s * * * 3. Local ad Global Characterstcs of Typcal Sle Kerel Fuctos 3.. Local Gaussa Kerel Fucto The Gaussa kerel fucto æ x x ö - - Kxx (, = exp ç s çè ø (9 ( s a typcal local kerel fucto [7]. To verfy the local characterstc of ths fucto, we choose the test pot x = ad plot the characterstc curves wth dfferet s as show F.. It ca be show that Kxx (, s comparatvely b whe the put value x s close to x =., ad however, ths value oes dowhll quckly wth the dstace betwee x ad x creas. Especally, ths value approaches to whe the sample pot oes aloof. Ths pheomea verfes that the Gaussa kerel fucto has stro local terpolato ablty ad comparatvely poor eeralzato ablty, ad that the performace characterstc of Gaussa kerel fucto s closely related to s. Output K(x,x.9.8.7.6.5.4.3.. σ =. σ =.3 σ =.5 σ =.8 - -.5 - -.5.5.5 Iput of aussa kerel x F.. Characterstc curves of Gaussa kerel fucto. 3.. Global Polyomal Kerel Fucto The polyomal kerel fucto q æx x ö Kxx (, = + ( ç è s ø s the typcal lobal kerel fucto [8], where the order q determes the lobal eeralzato ablty. To obta comparso result wth the characterstc of Gaussa kerel fucto (, we ve the characterrstc curves wth test pot x =. ad dfferet q, s F.. K(x,x 3.5 3.5.5.5 q =, σ = q =, σ = q =, σ =.5 q = 3, σ = - -.5 - -.5.5.5 Iput of polyomal kerel x F.. Characterstc curves of polyomal kerel fucto. 39
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 It ca be show that ths fucto value chae etly wth the dsturbace betwee the put value x ad test value x creas. Ad thus, the polyomal kerel fucto has ood eeralzato ablty. It ca be also show that the parameter s determes the affecto speed of ths kerel fucto to sample value, ad thus the parameter s ca be used to ehace the local terpolato ablty of ( wthout creas q. 4. Hybrd Kerel Fucto ad PSO- Based Parameter Des 4.. Hybrd Kerel Fucto Des From above dscusso, we ca show that the Gaussa kerel fucto has a ood local lear ablty ad comparatvely poor eeralzato, ad that the polyomal kerel fucto has a stro eeralzato ablty ad comparatvely poor lear ablty. If a hybrd kerel fucto ca be desed comb merts of these two fuctos, ood abltes of local lear ad eeralzato ca be acheved. The hybrd kerel fucto of ths paper s desed as follows q æ x x ö æ x x ö - r a ç s s p ç Kxx (, = (- a + + exp ( è ø è ø where a Î [,] s the weh scalar. It ca be easly see that ths kerel fucto reduces to the tradtoal polyomal kerel fucto whe a =, ad that t reduces to the Gaussa kerel fucto whe a =. I ths sese, ths kerel fucto s more eeral tha tradtoal oes ad therefore ca acheve better performace. I hybrd kerel fucto (, the kerel wdth s ca adjust the local ftt ablty ad eeralzato ablty of Gaussa fucto, ad the order q ca adjust the computatoal complexty ad weht freedom of RVM model. To aalyze the characterstc of ths fucto, we also compares t wth the Gaussa kerel fucto ( ad polyomal kerel fucto ( respectvely. By choos the test pot x =., the characterstc curves are show F. 3, whch mples that ths hybrd kerel fucto combes both merts of the Gaussa kerel fucto ad the polyomal kerel fucto, that s, t acheves comparatvely stro lear ad eeralzato ablty. It ca be show from F. 3 that curves ( ad (3 mply the fluece of parameter r to the characterstc of hybrd kerel fucto, ad that curves (4 ad (6 mply the lobal adjustmet ofq to hybrd kerel fucto, ad that curves (4 ad (5 mply the fluece of weh scalar a to hybrd kerel fucto. It ca be also see from F. 3 that the kerel parameters a, s, r, s, ad q are very mportat to determe the ftt ad eeralzato ablty of RVM model. Output of hybrd kerel.8.6.4..8.6.4. - - Iput of hybrd kerel x ( α =., σ =., r=, =, q= ( α =.3, σ =., r=.5, =.5, q= (3 α =.3, σ =., r=, =.5, q= (4 α =.6, σ =.4, r=, =, q= (5 α =.9, σ =.7, r=, =, q= (6 α =.6, σ =.4, r=, =, q=3 (7 α =.6, σ =.4, r=, =, q= F. 3. Characterstc curves of hybrd kerel fucto. 4.. Parameter Des of Hybrd Kerel Fucto us PSO Alorthm It ca be see from secto 4. that a stro lobal eeralzato ablty ad local terpolato ablty ca be acheved f parameters a, s, r, s, ad q re sutably desed. However, t s ot a easy work sce the relatoshp betwee these parameters s somewhat complex, whch s mpossble to be expressed explctly. Wth ths cosderato, PSO alorthm s appled to des optmal parameters, whch may comparatvely mprove the RVM model performace. PSO alorthm, whch s kow as a hh-qualty tellet evolutoary computato alorthm, was frst preseted 995 [9]. Compar wth the tradtoal mesh search method ad radet descet method, PSO alorthm s characterzed by rapd search rate, lobal coverece ad hh stablty. PSO alorthm smulates the socal behavor of dvduals (partcles fly a multdmesoal search space. Each partcle the swarm looks for ts optmal value the search space by utlz both ts ow experece ad ts ehbors experece, whch way, PSO alorthm ca solve some dfferet optmzato problems. 4
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 I a D -dmesoed complex search space, the th partcle updates ts posto ad velocty us the follow equatos, v = w( t v + cr( p - x + c r ( -x (3 x x v k+ k k k k k d d best d best d k+ k k+ = + d d d where v = ( v, v, ¼, v ad x = ( x, x, ¼, x are D D the speed ad posto vector of the th partcle, k s the terato step dex, r ad r are radom varables betwee ad, c ad c are lear factors, p s the best soluto amo the partcles best foud the curret terato, best s the lobal beat soluto acheved so far, wt ( s the erta weht whch ca be descrbed as 5. Expermets ad Smulato of Lyse Fermetato I ths secto, the hybrd-kerelled RVM model s bult to predct the cell cocetrato the Lyse fermetato process. The fermetato expermets are carred out based o WKT-3L fermetato equpmet show F. 5 ad the correspod dtal automatc cotrol system. w - w = - (4 ed wt ( w t Tmax where w s the oral erta weht, w ed s the fal erta weht, T max s the whole teratve tme, t s the curret teratve tme. It ca be show that wt ( wll learly decrease from w to w wth ed terato proceed. The speed vector v (3 s costraed by d v ìï v, v ³ v k + k + max d max = ï d í ï k + - v, v -v max d max ïî (5 The detaled procedure of des parameters of hybrd kerel fucto s show F. 4. PSO N Model parameter selecto PSO ed? Y Best parameters of hybrd kerel Soft Sesor Model of RVM wth hybrd kerel Tra of RVM wth hybrd kerel Predcto of RVM wth hybrd kerel Italzato RVM Hybrd kerel fucto Results of Tra Results of Predcto RVM Tra RVM Predcto Calculate the ftess value F. 4. The flow chart of soft sesor model based o RVM of hybrd kerel. F. 5. WKT-3L fermetato expermet system. Accord to the evrometal requremet of the Lyse fermetato process, the pressure fermetato procedure s cotrolled to be. MPa, temperature 3 C, rotat speed of the mx motor r/m. Accord to our early research, the supplemetary varables are chose as the varables of ph level (ph, dssolved oxye (Do, ar flux F ad the prmary varable s chose as cell cocetrato y. I ths way, the soft sesor model ca be desed as y = f(ph, Do, F (6 where f ( deote the complex olear relatoshp amo these varables. To obta eouh test samples, WKT-3L expermet system samples ph, Do, ad F every mute. At the same tme, the reacto lqud s sampled every 4 hours that ves exact cell cocetrato va 7-type spectrophotomete. Oe batch sample ca be obtaed a fermetato perod. Totally, 5 expermets are carred out ad thus 84 samples are obtaed, amo whch, the former 38 samples are used as tra data ad the rest 6 samples as test data. To obta a better model performace, samples are ormalzed to the terval [, ] before RVM tra. The speed costrat PSO alorthm s chose as % of v max. Ths mples, accord to (5, that 4
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 wt ( wll decrease from.9 to.4 the terato procedure. The rest parameters are ve as c = c =, N = 4, T =. Follow the max des flow show F. 4, the optmal parameters of hybrd kerel fucto are obtaed as a =.476, s = 6.9969, r =.8358, s =.886, q =. The RVM models wth hybrd kerel fucto, polyomal kerel fucto, Gaussa kerel fucto, respectvely are show F. 6 (I ths Fure, the kerel parameters of RVM models wth sle kerel fucto are also desed us PSO alorthm, ad the correspod relatve error curves are show F. 7. bacteral cocetrato(/l 9 8 7 6 5 4 3 True value RVM wth poly kerel RVM wth aussa kerel RVM wth hybrd kerel 3 4 5 6 7 tme(t/h F. 6. Comparso of predcto us soft sesor model based o sle kerel fucto ad hybrd kerel fucto. * y -y ARE = å % = y (7 MaxE = max( y - y * (8 * MSE = å ( y -y (9 = where y s the exact value, y * of RVM. s the predcted value The umber of relatve vectors (RVS, MSE, ARE ad MaxE of three RVM models are ve respectvely Table. It ca be show that the predcto performace of RVM model wth hybrd kerel fucto s much better tha the oes wth polyomal kerel fucto ad Gaussa kerel fucto respectvely. Furthermore, the RVM model wth hybrd kerel fucto has less relatve vectors tha the oes wth polyomal kerel fucto ad Gaussa kerel fucto. Ths mples that the RVM model wth hybrd kerel fucto s smpler ad less tme-cosumpto whe t s traed. Table. RVs, MSE, ARE ad MaxE of three RVM models. RVs MSE ARE (% MaxE Gaussa 7.46 8.53.79 Polyomal 7.74 7.57.9 Hybrd 6.43.59.366 Relatve error (%.6.4. -. RVM wth poly kerel RVM wth aussa kerel RVM wth hybrd kerel -.4 3 4 5 6 7 tme (h F. 7. Comparso of relatve error us soft sesor model based o sle kerel fucto ad hybrd kerel fucto RVM. 6. Cocluso A soft ses model method based o RVM wth a hybrd kerel fucto s preseted to resolve the ole detecto dffculty of some mportat state varables fermetato processes wth tradtoal strumets. Based o the characterstc aalyss of two tradtoal frequetly-used kerel fuctos, that s, local Gaussa kerel fucto ad lobal polyomal kerel fucto, a hybrd kerel fucto comb merts of Gaussa fucto ad polyomal fucto s costructed. PSO alorthm s also appled to obta optmal parameters of kerel fuctos. The proposed model method s used to predct the cell cocetrato the Lyse fermetato process. Smulato results show that the hybrd-kerel RVM model has a better accuracy ad performace tha sle kerel RVM models. To further aalyze the predcto ablty of three models, the follow averae relatve error (ARE, maxmum absolute error (MaxE, ad root mea squared error (MSE are defed to evaluate the model performace Ackowledemets Ths work s supported by the atural scece foudato of Jasu Provce, Cha uder rat BK465. 4
Sesors & Trasducers, Vol. 73, Issue 6, Jue 4, pp. 37-43 Refereces []. J. Zha, Z. Xo, D. Gullaume, A. Lamade, Batch to batch teratve lear cotrol of a fed-batch fermetato process us learsed models, Proceeds of the Iteratoal Coferece o Cotrol, Automato, Robotcs ad Vso (ICARAV'8, Mela Hao Hao, Vetam, 7- December 8, pp. 745-75. []. J. C. B. Gozaa, L. A. C. Melero, C. Ka, R. M. Flho, ANN-based soft-sesor for real-tme process motor ad cotrol of a dustral polymerzato process, Computers ad Chemcal Eeer, Vol. 5, Issue, 9, pp. 43-49. [3]. C. Brosllow, Iferetal cotrol of process, Amerca Isttute of Chemcal Eeers Joural, Vol. 4, Issue, 978, pp. 485-59. [4]. K. Desa, Y. Badhe, S. S. Tambe, B. D. Kulkar, Soft-sesor developmet for fed-batch boreactors us support vector reresso, Joural of Bochemcal Eeer, Vol. 7, Issue 3, 6, 5-39. [5]. G. Lu, D. Zhou, H. Xu, C. Me, Model optmzato of SVM for a fermetato soft sesor, Expert Systems wth Applcatos, Vol. 37, Issue 4,, 78-73. [6]. M. E. Tpp, Sparse Bayesa lear ad the relevace vector mache, Joural of Mache Lear Research, Vol.,, -44. [7]. M. E. Tpp, C. A. Faul, Fast maral lkelhood maxmsato for sparse Bayesa models, Proceeds of the Iteratoal Workshop o Artfcal Itellece ad Statstcs (AISTATS'6, Key west, Florda, 3-6 Jauary 6, pp. -6. [8]. K. R. Müller, S. Mka, G. Rätsch, K. Tsuda, A troducto to Kerel-Based Lear Alorthms, IEEE Trasactos o Neural Networks, Vol., Issue,, pp. 8-. [9]. J. Keedy, R. Eberhart, Partcle swarm optmzato, Proceed of the Iteratoal Coferece o Neural Network (INNS'95, Perth, Wester Australa, 995, pp. 94-948. 4 Copyrht, Iteratoal Frequecy Sesor Assocato (IFSA Publsh, S. L. All rhts reserved. (http://www.sesorsportal.com 43