RADIAL BASIS FUNCTION PROCESS NEURAL NETWORK TRAINING BASED ON GENERALIZED FRÉCHET DISTANCE AND GA-SA HYBRID STRATEGY

Transcription

1 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 RADIAL BASIS FUNCTION PROCESS NEURAL NETWORK TRAINING BASED ON GENERALIZED FRÉCHET DISTANCE AND GA-SA HYBRID STRATEGY Wang Bng, Meng Yao-hua, Yu Xao-hong 3 School of Compuer & Informaon Technology, Norheas Peroleum Unversy, Daqng, Chna Communcaon Research Cener, Harbn Insue of Technology, Harbn, Chna 3 Exploraon and Developmen Research Insue, Daqng Olfeld Company, Daqng, Chna ABSTRACT For learnng problem of Radal Bass Funcon Process Neural Newor (RBF-PNN), an opmzaon ranng mehod based on GA combned wh SA s proposed n hs paper. Through buldng generalzed Fréche dsance o measure smlary beween me-varyng funcon samples, he learnng problem of radal bass cenre funcons and connecon weghs s convered no he ranng on correspondng dscree sequence coeffcens. Newor ranng obecve funcon s consruced accordng o he leas square error creron, and global opmzaon solvng of newor parameers s mplemened n feasble soluon space by use of global opmzaon feaure of GA and probablsc umpng propery of SA. The expermen resuls llusrae ha he ranng algorhm mproves he newor ranng effcency and sably. KEYWORDS Radal Bass Funcon Process Neural Newor, Tranng Algorhm, Generalzed Fréche Dsance, GA-SA Hybrd Opmzaon. INTRODUCTION Radal bass funcon neural newor (RBFNN) s a hree-layer feedforward neural newor model and was pu forward by Powell, Broomhead and Lowe[,] n he md-980s. I mplemens nonlnear mappng by change propery parameers of a neuron and mproves learnng rae by lnearzaon of connecon wegh adusmen. RBFNN has been appled n many felds such as funcon approxmaon, splne nerpolaon and paern recognon [3,4]. However, n praccal engneerng, many sysems npus are dependen on me; he npus of exsng RBFNN model are usually consan. They have nohng o do wh he me, ha s, he npu/oupu s nsananeous relaonshp beween geomerc pons. Is nformaon processng mechansm canno drecly reflec he characerscs of me-varyng process npu sgnals and he dynamc effec relaonshp beween varables. Therefore, a radal bass funcon process neural newor (RBF-PNN) s esablshed n leraure [5], and s npus can be me-varyng process sgnal DOI : 0.5/cse

2 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 drecly. Through space-me aggregaon of ernel funcon neurons o fnd sample paern process characerscs, he RBF-PNN has been appled n dynamc sample cluserng and mevaryng sgnal denfcaon effecvely. In pracce, me-varyng sample daa s usually dscree me seres obaned by samplng, he exsng RBF-PNN s unable o deal wh dscree npu drecly. I need o f dscree samples, hus fng error exss. In subsequen orhogonal bass expanson, he fne erms of bass funcon also brng runcaon error. In addon, learnng effcency of radonal RBF-PNN algorhm based on graden descen combned wh bass expanson s low, and a he same me affecs sably and generalzaon ably of he newor ha here are greaer freedom consrans among newor parameers. In recen years, evoluonary compuaon-based opmzaon echnology s wdely used o solve opmal soluon n varous engneerng problems, and has been effecvely appled n complex funcon calculaon, process opmzaon, sysem denfcaon and conrol, and so on [6-8]. Genec algorhm (GA) [9] s a bonc global opmzaon mehod ha smulaes Darwn s naural selecon and naural bologcal evoluon process. I s smple, general, robus, suable for parallel processng, as well as effcen and praccal, and has been wdely appled n varous felds. However, GA has some shorcomngs, e.g. mmaure convergence and slow erave rae. If probablsc umpng propery of smulaed annealng (SA) s made use of, he GA combned wh SA are appled no he ranng of RBF-PNN, hs wll mprove RBF-PNN learnng propery grealy. Fréche dsance [0] s a udgmen o measure he smlary beween polygonal curves. In applcaons, dynamc samples normally conss of a sequence of dscree samplng pons, herefore, Eer and Mannla [] pu forward dscree Fréche dsance on he bass of connuous Fréche dsance, and acheved good applcaon effec n proen srucure predcon[], onlne sgnaure verfcaon [3], ec. Tme-varyng process sgnal can be seen as a onedmensonal curve abou he me; herefore, dscree Fréche dsance can be exended o mevaryng funcon space o measure naure dfference beween npu samples of RBF-PNN. For he RBF-PNN ranng problem, we propose a ranng mehod based on generalzed dscree Fréche dsance combned wh GA-SA opmzaon n hs paper. By consrucng a generalzed Fréche dsance norm o measure he dsance beween dynamc samples n me-varyng funcon space, hen he learnng problem of wegh funcons s ransformed no dscree coeffcen ranng. Accordng o he leas square error creron, he newor ranng obecve funcon s consruced, and GA-SA algorhm s adoped for global opmzaon solvng. I s appled no EEG eye sae recognon and acheves good resuls. RBF-PNN opmzaon ranng seps are also gven n he paper.. DISTANCE MEASUREMENT BETWEEN TIME-VARYING SAMPLES BASED ON GENERALIZED FRÉCHET DISTANCE.. Dscree Fréche Dsance Dscree Fréche dsance s defned as follows: (a) Gven a polygonal chan P =< p, p,, pn > wh n verces, a -wal along P parons he pah no dson non-empy subses { P } =,, such ha P,, =< pn p + n > and = n < n < < n = n. 0 0

3 x ( ) Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 (b) Gven wo polygonal chans A =< a, a,, am > and B =< b, b,, bn >, a pared wal along A and B s a -wal { A } =,, along A and a -wal { B } =,, along B for, eher A = or B = (namely, A or B conans exacly one verex). (c) The cos of a pared wal W = {( A,)} B along A and B s W d ( A,) B max = max(,) ds a b () F ( a,) b A B The dscree Fréche dsance beween wo polygonal chans A and B s d ( A,) B mn( = d,) W A B () The pared wal s also called he Fréche algnmen of A and B. F.. Generalzed Fréche Dsance Beween Tme-Varyng Samples W The dscree Fréche dsance can effecvely measure he dfference beween me-varyng funcons (whch can be regarded as he polygonal chans), whle he npu of he RBF-PNN s a vecor composed of me-varyng funcons. Thus, usng he propery ha Eucldean dsance can mplemen pon arge machng, combnng he dscree Fréche dsance and Eucldean dsance, a generalzed Fréche dsance s esablshed, whch can measure he dsance beween mevaryng funcon vecor samples. Gven wo me-varyng funcon samples X()((),(),...,()) = x x xn and X ()((),(),...,()) = x x x n, her generalzed Fréche dsance d((),()) X X s defned as F n = d((),())(((),())) X X = df x x (3) where df ((),()) x x denoes he dscree Fréche dsance beween dscree samplng pons correspondng o x () and x (). 3. RBF-PNN MODEL 3.. Radal Bass Funcon Process Neuron The srucure and nformaon ransfer flow of a radal bass funcon process neuron are shown n Fgure. x () x () x () n C() () ds K ( ) y 3

4 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 Fgure. Radal bass funcon process neuron In Fgure, X()((),(),...,()) = x x xn s an npu funcon vecor; C()((),(),...,()) = c c cn s a cenre funcon vecor of a radal bass funcon neuron; ds() s a dsance measuremen funcon beween he npu funcon vecor and he cenre funcon vecor, and here we adop he generalzed Fréche dsance defned n Eq.(3). K( ) s a ernel funcon of a radal bass process neuron. I could be a Gauss funcon, a mulquadrc funcon and a hn-plae splne funcon ec. The npu/oupu relaonshp of a radal bass funcon process neuron s defned as: y = K((),()) ds X C (4) 3.. Radal Bass Funcon Process Neural Newor Model A RBF-PNN model s composed of radal bass funcon process neurons and oher ypes of neurons accordng o ceran srucure relaons and nformaon ransmsson process. For convenence, consder a mulple npu/sngle oupu feedforward newor model only wh one radal bass process hdden layer defned by Eq. (4). Is srucure s shown n Fgure. x () C() (),() ds K x (). x () n C() (),() ds K. Cm() (),() ds K y Fgure. RBF-PNN model In Fgure, he npu layer has n neurons, he radal bass process hdden layer has m neurons, and ( =,,,) m are he weghed coeffcens o be adused. The npu/oupu relaonshp of RBF-PNN s: m y = K((),()) ds X C (5) = 3.3. Newor Obecve Funcon Gven K learnng sample funcons ((),) X d where X ()(),(),...,() = x x xn, =,,..., K. Suppose ha y s a real oupu correspondng o he h sample npu. Accordng o he leas square error creron, he newor obecve funcon s defned as below: K ( ) (6) = E = y d 4

5 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 The ernel funcon of he radal bass process neuron adops Gauss funcon: K() d = exp() d (7) where s he wdh of Gauss funcon, and needs o be deermned va newor ranng, hen we have K E = ( y d = ) ds X (),() C = d K m ( exp() = = ) (8) In Eq.(8), snce ds X (),() C s based on generalzed dscree Fréche dsance, f dscree samplng pon number of RBF-PNN npu s S, hen { } s= c s S s he dscree pon sequence correspondng o c (), =,,, n, whch s he h componen of he h radal bass cenre funcon vecor. In concluson, he learnng obecve funcon E of RBF-PNN s a s funcon abou varable parameers { },, S c. s= 4. RBF-PNN TRAINING BASED ON GA-SA ALGORITHM In GA [9], selecon, crossover and muaon consue s man operaons. Parameer encodng, nal populaon, fness funcon choce, genec operaon desgn, conrol parameer seng form he core conen of GA. The obvous advanage of GA s o search many pons n search space a he same me, and s opmal soluon searchng process s nsrucve so as o avod he dmenson dsaser problem exsng n some oher opmzaon algorhms. Smulaed annealng (SA)[4] algorhm s a heursc random search algorhm. Under a ceran nal emperaure, accompaned by emperaure fall, by use of probablsc umpng propery, he global opmal soluon of he obecve funcon s obaned n soluon space. In vew of he respecve feaures of GA and SA, consderng ha SA can effecvely preven GA from rappng n local opmal soluon, GA and SA wll be combned for RBF-PNN ranng o oban he opmal soluon of he problem. The ranng problem of RBF-PNN s o adus parameers o mnmze he obecve funcon s S { } s= E=( E c,,). The concree RBF-PNN ranng seps based on GA-SA s descrbed below: Sep Gven nal smulaed annealng emperaure, and se =. Sep Deermne he populaon sze N, and randomly generae nal populaon. Decmal number s used for chromosome encodng, and he gene number on each chromosome s he number of varables o be opmzed. Se he maxmum erave number M and error precson. Sep 3 Evaluae chromosomes n he populaon. Snce RBF-PNN ranng s he mnmum opmzaon problem for he obecve funcon, he fness funcon s aen as 5

6 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 f = + E (9) Sep 4 Perform genec operaons for chromosomes n he populaon. () Selecon operaon. Adop proporonal selecon operaor, and he probably ha he chromosome X wh fness value f s seleced no he nex generaon s M p = f f (0) = () Crossover operaon. For decmal number encodng, choose arhmec crossover operaor. Chromosomes X and X n he paren perform crossover operaon wh crossover probably p c, he offsprng chromosomes generaed are X = ax + () a X () ' where a s a parameer and a (0,). X = () a X + ax () ' (3) Muaon operaon. Use unform muaon operaor. Each gene n a chromosome X performs muaon operaon wh muaon probably p m, namely unformly dsrbued random number whn he scope of gene value s used o nsead of he orgnal value wh probably p m. Sep 5 Inroduce he opmal reenon sraegy. Sep 6 Each chromosome n he populaon performs smulaed annealng operaon. () Creae new gene g ' ()() = g + by use of SA sae generaon funcon where ( mn, max) are he random dsurbance and (mn,max) are he scope of correspondng 0 0 gene value. () Calculae obecve funcon dfference value C beween (3) Calculae accepance probably p = mn[,exp( C /)]. r g ' () and g(). ' (4) If pr > random[0,] hen g()() = g ; oherwse g() remans unchanged. (5) Inroduce he opmal reenon sraegy. (6) Anneal emperaure by usng he annealng emperaure funcon + = v where v (0,) s he annealng emperaure rae, and se = +. Sep 7 If he obecve funcon value correspondng o he opmal chromosome mees error requremens or he eraon process acheves he maxmum number M, urn o Sep 8; oherwse reurn o Sep 3. Sep 8 Decode he opmal chromosome searched by GA and oban he opmal parameers of RBF-PNN. 5. APPLICATIONS IN EEG EYE STATE CLASSIFICATION 6

7 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 The smulaon expermens use EEG Eye Sae Daa Se from UCI. The daa se was colleced for examnng a classfer for sequenal me seres. The daa se consss of 4 EEG values and a value ndcang he eye sae, and '' ndcaes he eye-closed and '0' he eye-open sae. All daa s from one connuous EEG measuremen wh he Emov EEG Neuroheadse. The par of samples wh he eye-closed sae and he eye-open sae are shown n Fgure (a) Fgure 3. (a) The eye-closed sae samples (b) The eye-open sae samples In he expermens, he ranng se ncludes 60 samples wh each 30 samples separaely from wo saes, and he es se consss of 60 samples. The RBF-PNN srucure parameers are : mevaryng npu node n =, 8 RBF hdden nodes m = 8 and common oupu node. The dscree samplng number s S = 4, he populaon sze s 5,opmzaon eraon number s 000, and he error precson s As he comparson, we also use graden descen (GD) algorhm [5] and GA o ran he RBF-PNN. Each algorhm runs 0 mes, and hen hree algorhms are separaely used o denfy he es samples. The average denfcaon accuracy s shown n Fgure 4. I can be seen ha he SA s on no GA prevens no local mnmum effecvely and mproves he denfcaon resul of he newors. (b) 00 Accuracy (%) GD GA GA-SA 7

8 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 Fgure 4. Accuracy of hree denfcaon algorhms In addon, he ranng algorhms based on generalzed Fréche dsance and on orhogonal bass expanson are respecvely adoped o ran he RBF-PNN. The accuracy based on generalzed Fréche dsance s superor o on orhogonal bass expanson. Ths s because he algorhm based on orhogonal bass expanson has fng error and runcaon error nevably. 6. CONCLUSIONS A RBF-PNN ranng mehod based on GA and SA hybrd opmzaon sraegy s pu forward n hs paper. Accordng o he newor ranng obecve funcon, by buldng a generalzed Fréche dsance used o measure smlary beween me-varyng funcon samples, he funconal opmzaon problem of RBF-PNN ranng s convered no exremal opmzaon solvng of mulvarae funcon, and he global opmzaon soluon of he newor parameers s obaned n he feasble soluon space. I also has ceran reference value for oher machne learnng and complex funcon opmzaon problems. REFERENCES [] Powel M.J. D. (985) Radcal bass funcons for mulvarable nerpolaon: A revew, IMA conference on Algorhms for he Approxmaon of Funcons and Daa, RMCS, Shrvenham, UK: pp [] Broomhead D. S, Lowe D. (988) Mulvarab le funcon nerpolaon and adapve newors, Complex Sysems, No., pp [3] He S, Zou Y, Quan D, e al. (0) Applcaon of RBF Neural Newor and ANFIS on he Predcon of Corroson Rae of Ppelne Seel n Sol, Recen Advances n Compuer Scence and Informaon Engneerng. Sprnger Berln Hedelberg, pp [4] Kang L, Jan-Xun Peng e al. (009) Two -Sage Mxed Dscree Connuous Idenfcaon of Radal Bass Funcon (RBF) Neural Models for Nonlnear Sysems, IEEE Transacons on Crcus and Sysems, Vol. 56, No.3, pp [5] Xu Shao-hua, He Xn-gu. (004) Research and applcaons of radal bass process neural newors,journal of Beng Unversy of Aeronaucs and Asronaucs, Vol. 30, No., pp4-7. [6] De Jong K. (0) Evoluonary compuaon: a unfed approach[c] Proceedngs of he foureenh nernaonal conference on Genec and evoluonary compuaon conference companon. ACM, pp [7] Smon D. (0) A probablsc analyss of a s mplfed bogeography-based opmzaon algorhm, Evoluonary compuaon, Vol. 9, No., pp [8] Vdal T, Cranc T G, Gendreau M, e al. (0) A hybrd genec algorhm for muldepo and perodc vehcle roung problems, Operaons Research, Vol. 60, No. 3, pp [9] De Govann L, Pezzella F. (00) An mproved genec algorhm for he dsrbued and flexble ob-shop schedulng problem, European ournal of operaonal research, 00, 00(): [0] Al H, Godau M, (99) Measurng he Resemblance of Polygonal Curves, In Proceedngs of he 8h Annual Symposum on Compuaonal Geomery, pp [] Eer T, Mannla H, (994) Compung Dscree Fréche Dsance, Vena: Informaon Sysems Deparmen, Techncal Unversy of Vena. [] Mnghu Jang, Yng Xu, Bnha Zhu, (007) Proen Srucure -srucure Algnmen wh Dscree Fréche Dsance, In Proceedngs of he 5h Asa-Pacfc Bonformacs Conference, APBC, pp3-4. [3] Janbn Zheng, Xaole Gao, Enq Zhan, Zhangcan Huang, (008) Algorhm of On-lne Handwrng Sgnaure Verfcaon Based on Dscree Fréche Dsance, In Proceedngs of he 3rd Inernaonal Symposum on Inellgence Compuaon and Applcaons, ISICA 008, pp

9 Compuer Scence & Engneerng: An Inernaonal Journal (CSEIJ), Vol. 3, No. 6, December 03 [4] Avnaash M R, Kumar G R, Bhargav K A, e al. (03) Smulaed annealng approach o soluon of mul-obecve opmal economc dspach, Inellgen Sysems and Conrol (ISCO), 03 7h Inernaonal Conference on. IEEE, pp 7-3. Auhors Wang Bng, born n 98, s a Ph.D. canddae n Norheas Peroleum Unversy. She receved her Bachelor and Maser degrees n Norheas Peroleum Unversy, DaQng provnce Chna n 7/004, 4/007 respecvely. From 007 o now she s a lecurer a Deparmen of Compuer & Informaon Technology, Norheas Peroleum Unversy. She maors n process neural newors, nellgence nformaon processng and daa mnng ec. She has publshed more han 5 refereed ournal and conference papers. Meng Yao-hua, born n 98, s a Ph.D. canddae n he Harbn Insue of Technology, Communcaon Research Cener. He receved hs Bachelor and Maser degrees n Norheas Peroleum Unversy, DaQng provnce Chna n 7/003, 8/007 respecvely. From 007 o 00 he was a lecurer a Deparmen of Elecrcal and Elecronc Engneerng, Helongang Bay Agrculural Unversy. Durng he year 0, he was a vsng scholar a Harbn Insue of Technology. In Augus 0, he on Cenre for massve daase mnng a Harbn Insue of Technology as a researcher. Hs research of neress ncludes process neural newors, sgnal and nformaon processng ec. Yu Xao-hong, born n 98, s an engneer n Exploraon and Developmen Research Insue of Daqng Olfeld Company, Chna. She receved her Bachelor and Maser degrees n Norheas Peroleum Unversy, DaQng provnce Chna n 7/005, 4/008 respecvely. From 008 o now she wors n Daqng Olfeld Company. Her neress nclude ol feld daa denfcaon, arfcal neural newors, nellgen nformaon processng ec. 9