SVM Parameters Tuning with Quantum Particles Swarm Optimization

Size: px

Start display at page:

Download "SVM Parameters Tuning with Quantum Particles Swarm Optimization"

Jack Gardner
6 years ago
Views:

1 SVM Paamees uning wih Quanum Paices Swam Opimizaion Zhiyong Luo Wenfeng Zhang Yuxia Li 3 and Min Xiang. Schoo of Auomaion Chonging Univesiy of Poss and eecommunicaions Chonging 465 China. Schoo of Auomaion Nohwesen Poyechnica Univesiy Xi an 77 China 3. Schoo of Auomaion Engineeing Univesiy of Eecic Science and echnoogy of China Chengdu 654 China uozhyong@63.com Absac Common used paamees seecion mehod fo suppo veco machines (SVM) is coss-vaidaion which is compicaed cacuaion and aes a vey ong ime. In his pape a nove eguaizaion paamee and ene paamee uning appoach of SVM is pesened based on uanum paice swam opimizaion agoihm (QPSO). QPSO is a paice swam opimizaion (PSO) wih uanum individua ha has bee goba seach capaciy. he paamees of eas suaes suppo veco machines (LS-SVM) can be adused using QPSO. Cassificaion and funcion esimaion ae sudied using LS-SVM wih wavee ene and Gaussian ene. he simuaion esus show ha he poposed appoach can effecivey une he paamees of LS-SVM and impoved LS-SVM wih wavee ene can povide bee pecision. Keywods uanum paice swam opimizaion agoihm (QPSO) paamees uning suppo veco machines (SVM) eas suaes suppo veco machines (LS-SVM) I. INRODUCION As a machine eaning mehod suppo veco machines (SVM) oiginay inoduced by Vapni [] wihin he aea of he saisica heoy and sucua is minimizaion has emeged as one powefu oo fo daa anaysis. I has been widey used fo many appicaions such as egession and paen ecogniion [3] [4]. I is we nown ha SVM geneaizaion pefomance depends on a good seing of eguaizaion paamee and he ene paamee. o minimize he geneaizaion eo hese paamees shoud be popey seeced. hee ae oughy sevea mehods fo SVM o seec paamee: coss-vaidaion Vapni-Chevonenis (VC) bound and he Bayesian evidence famewo mehod [5] [6] [7]. Coss-vaidaion is he mos commony used appoach due o is high accuacy bu i is compex cacuaion and consumes moe compuaion ime. Paice swam opimizaion agoihm (PSO) [8] [9] pu fowad by Kenney and Ebeha is a swam ineigen opimizaion mehod. PSO begins wih a andom popuaion and seaches fo opima by updaing he popuaion. he uanum paice swam opimizaion agoihm (QPSO) was pesened in [] [] as an agoihm wih good pefomancecompexiy ade-off. Based on he uanum bi he bes chomosome's guidance is used o daw cose o he opimum sep by sep QPSO can find souion uicy in goba space. In his pape QPSO is appied o une paamees of eas suaes suppo veco machines (LS-SVM) [] which ae used in paen ecogniion and funcion esimao. Simuaions conduced on benchma daa se and sandad appoximaed funcions demonsae he effeciveness and efficiency of he poposed mehod. he bee pefomances and easie of impemening paamees seecion wih simpe cacuaion is compaed wih coss-vaidaion mehod o une LS-SVM wih wavee ene and Gaussian ene. he es of his pape is oganized as foows: In Secion II we eview LS-SVM. he poposed appoach is given in Secion III. Expeimena esus ae shown in Secion IV and some concusions wi be given in Secion V. II. LEAS SQUARES SUPPOR VECOR MACHINES Le D {( x y)( x y ) ( x y )} be a aining se d wih inpu daa x R and coesponding oupu daa y R. Leaning fom he aining daa can be viewed as a muivaiae funcion f appoximaion ha epesens he eaion beween he inpu daa and oupu daa [] []. In he genea case he inpu daa ae mapped ino a feaue space by noninea funcion Φ(x ) he SVM funcion f (x) can be expessed as: f ( x) ω Φ( x) + b () whee ω is a m -dimensiona veco and b is a scaa. A. LS-SVM fo funcion esimaion [] In LS-SVM fo funcion esimaion one defines he foowing opimizaion pobem: s.. min J ( ω b e) ω ω + γ e ω b e whee [ ] y () ω Φ( x ) + b + e (3) e e e e e R denoes he eo veco γ is eguaizaion paamee /8 /$5. 8 IEEE CIS 8

2 Using he opimizaion heoy can sove his pobem. One can define he Lagangian fo his pobem as foows L( ω b e; α) J ( ω b e) α ω { Φ( x ) + b + e y } whee α ( ) ae Lagange muipies. he condiions fo opimaiy ead o a se of inea euaions: b + Ω γ I α y whee y [ y ] y y [ ] α [ α α α ] and Ω Φ( x ) Φ( x ) (4) (5) mn m n fo m n. Accoding o Mece s condiion hee exiss K ( ) ( ) ene funcion x m x ) Φ x Φ ( n m x n he esuing LS-SVM mode fo funcion esimaion becomes: whee f ( x) α K( x x ) + b (6) α b ae he souion o he inea sysem (5). B. LS-SVM fo cassificaion [] Consideing y ( ω Φ( x ) + b) e y { + } fo cassificaion simia o funcion esimaion he souions ead o a se of inea euaions: y b + y Ω γ I α whee y [ y ] y y [ ] α [ α α α ] and Ω y y Φ( x ) Φ( x ) whee mn m. (7). he esuing LS-SVM mode fo cassificaion is: III. f ( x) sgn( α y K( x x ) + b) (8) α b ae he souion o he inea sysem (7). QUANUM PARICLES SWARM OPIMIZAION UNING Paamees seecion fo SVM is vey compex and uie had o sove by conveniona opimizaion echniues. Hee uanum paices swam opimizaion agoihm (QPSO) is adoped o une he paamees of LS-SVM mode. he QPSO was pesened in [] [] as an agoihm wih good pefomance-compexiy ade-off. Based on he uanum bi he uanum bi has he advanage ha i can epesen a inea supeposiion of saes in seach space pobabiisicay and he bes chomosome's guidance is used o daw cose o he n m n opimum sep by sep QPSO can find souion uicy in goba space. In he uanum heoy he minimum uni ha caies infomaion is a ubi which can be in any supeposiion of sae and. We define such a uanum paice veco Q ( ) n a geneaion whee n is he size of he popuaion and is he paice wih uanum enegy which is defined as: [ () ( ) ( m) ] (9) whee () i ( i m) uanum paice m is he paice's engh. () i which epesen a bi of gives he appeaing pobabiiies of he sae. hen he QPSO agoihm can be descibed as: begin iniiaize Q () obseve Q () o ge P () evauae P () soe he bes souion among P () whie (no eminaion-condiion) do begin + obseve Q ( ) o ge P () evauae P () updae Q () soe he bes souion among P () end end whee P () ae ae paice veco P () [ p pn ] is he h paice. In he iniiaize Q () a () i ( i m) uanum chomosome ( n) p of ae iniiaized as which means ha a he possibe inea supeposiion of saes appea in he same pobabiiy. Paice veco P () is fomed by obseving Q () in he nex sep he pocedue can be descibed as: [ ] [ ] < and p () i () > and whee p (i) epesens he i h bi of evauae P () and he bes souion among iniia () p ( n). hen sep is pefomed wih evauaion funcion P is soed.

3 In whie ( ) do cyces Paice veco () by obseving Q ( ) and evauae P () P is fomed sep is impemened wih above-menioned mehod. A updae Q () sep is added ie he PSO he QPSO agoihm has memoy in ode o soe he bes posiion vaues aeady found fo each paice ( p bes ) and he bes goba posiion ( p gbes ). Fom hese posiions he bes goba and individua uanum enegy vaues ae cacuaed in ode o geneae changes in he paice posiions. gbes bes α p + β ( p ) () gbes gbes α p + β ( p ) () bes whee α + β < α β < ae caed he cono paamees which epesen he cono degee of Q (). he smae of α he bigge of he appea pobabiiy of he desied iem + bes bes c + c + c 3 gbes (3) whee c + c + c3 < c c c3 < epesen he degee of he beief on onesef oca maximum and goba maximum especivey. Afe updae Q () sep in he cyces he bes paice souion among P () is seeced if i is bee han he soed bes paice souion hen i wi be soed. When he cyces ae compeed he soed bes paice souion is he expeced paice souion wih uanum paices swam opimizaion. In ode o speed up uanum paices swam opimizaion we adop wo-sep scheme o appy QPSO on he paamees seecion. Because he ange of possibe paamees is wide educing he seach ange can impove efficiency of opimizaion so he fis sep is appied o ough adus he paamees and he second sep o une he paamees wihin he seeced ange which is confimed by he fis sep s obained paamees ha is shown as: whee ( ) v u * ( u v ) ( u v ) λ η (4) λ + η (5) * v ae oigina minimum of eguaizaion paamee γ ange and ene paamee σ ange and u ( ) ae oigina maximum of paamees ange. * λ ( ) epesen he fis sep opimized vaue of γ and σ v and u ae he second sep minimum and maximum of paamees ange especivey. he coefficien η [.5] is used o adus he second sep paamees seach ange accoding o he opimized esu of he fis sep. Fo pevening he new ange o ge acoss oigina ange we mae he foowing ues: if v < hen u. u v hen v v and if u > u o appy QPSO on he paamees uning we uiize he vaue ange of he eguaizaion paamee and he ene paamee o decide he bi numbe of a paice souion p hen he coesponding bi numbe of a uanum chomosome can be confimed. Fo Gaussian ene of LS-SVM mode he iniia vaue ange of he eguaizaion paamee γ and he ene paamee σ ae [. ] and [. ] especivey. he evauaion funcion which defines he seecion cieia affecs he pefomance of he mode paamees seecion. Hee QPSO shows is fexibiiy o impemen vaious cieia accoding o appicaions. One choice of he evauaion is o use he ovea cassificaion ae on a es se. he evauaion funcion of cassificaion is defined as: Evauaion s Accuacy s (6) whee Accuacy s is he cassifie s s h geneaion esing accuacy. Anohe choice of he evauaion is o use he funcion appoximaion pecision on a es se. he evauaion funcion of funcion esimaion is defined as: Evauaion z ( y z f ) (7) whee y and f denoe he desied oupu and he appoximaion oupu fo he es se s h inpu daa especivey z is he inpu daa numbe of he es se and Evauaion is h geneaion evauaion o pesen funcion appoximaion accuacy. Accoding o LS-SVM mode he evauaion funcion is chosen o opimize he paamees by using QPSO. In genea uanum paice swam opimizaion he size of popuaion is. geneaions opimizaion is impemened in he fis sep he pogam is eminaed when he uanum paices swam opimizaion has been compeed. In second sep he pogam is eminaed when he bes evauaion has 4 no changed moe han a vey sma vaue i.e. fo cassificaion and funcion appoximaion ove he as geneaion. When uanum paice swam opimizaion is accompished he opimized bes paice souion is conveed o ea vaue he bes vaue of eguaizaion paamee γ and ene paamee σ can be obained. IV. EXPERIMENS In his secion we evauae he poposed SVM paamees uning mehod wih hee numeica expeimens he cassificaion of wo-cass benchma pobem appoximaion of a singe-vaiabe funcion and wo-vaiabe funcion. Fo compaison he coss-vaidaion (CV) and QPSO mehod ae pefomed o une he paamees of LS-SVM mode wih wavee ene and Gaussian ene especivey. he ene funcion of wavee ene and Gaussian ene ae

4 d K ( x ) i i i ( x x a )exp( x x x i i and K( x x ) exp( x (σ )) whee he x a ) denoes he i h componen of he h aining sampes a is wavee diaion coefficien [3]. A. Cassificaion on benchma pobem Fo cassificaion we use he UCI binay cassificaion benchma eposioy [4]: he Johns Hopins univesiy ionosphee (ion) and he sona (sn) wih inpu dimension n eua o 33 and 6 and oa numbe of paens 35 and 8 especivey. Each componen of he inpu daa is nomaized o zeo mean and uni sandad deviaion. We exac andomy /3 of he daa as ain se and he es as es se. he paamees of LS-SVM cassifies wih wavee ene and Gaussian ene ae ( γ a) and ( γ σ ) which ae adused by he coss-vaidaion (CV) and QPSO opimizaion especivey. he paamees of PSO ae c c. c. 3 6 α. and β. 9. he uned paamees and esus of cassificaion ae shown in abe I. i x pocedue. abe II iss he uned paamees and appoximaion eos. he appoximaion esus ae poed in Figs. and especivey. Mehod CV QPSO ABLE II. UNED PARAMEERS AND RESULS OF APPROXIMAION Kene γ a o σ NRMSE ( ain ) ( es ) Wavee Gaussian Wavee Gaussian ABLE I. UNED PARAMEERS AND RESULS OF CLASSIFICAION Mehod Daa Kene γ a o σ ain (%) es (%) CV ion sn Wavee Gaussian Wavee Gaussian Figue. Oigina funcion (soid ine) and esuing appoximaion by LS- SVM wih Gaussian ene (doed ine) QPSO ion sn Wavee Gaussian Wavee Gaussian B. Appoximaion of singe-vaiabe funcion In his expeimen we appoximae he foowing singevaiabe funcion[3] f.86x x.5x. e sin ( x) 5 [(.3x +.7) x] x < x < x We have unifomy samped exampes of poins poins of which ae aen as aining exampes and ohes esing exampes. he paamees of LS-SVM esimao wih wavee ene and Gaussian ene ae adused wih above wo uning paamees mehods. he paamees of QPSO ae c c. c. 3 8 α. and β. 9. he nomaized oo of mean-suae-eo (NRMSE) is seved as one cieia fo assessing he exapoaion abiiy of ou Figue. Oigina funcion (soid ine) and esuing appoximaion by LS- SVM wih Wavee ene (doed ine) C. Appoximaion of wo-vaiabe funcion his expeimen is o appoximae a wo-vaiabe funcion[3] f ( x y) ( x y ) sin(.5x) We ae unifomy 8 samped poins as he aining exampes and 6 poins as he esing exampes. he

5 paamees of QPSO ae c c. c. 3 8 α. 5 and β. 95. abe III iss he uned paamees and appoximaion esus of LS-SVM wih wavee ene and Gaussian ene especivey. Fig. 3 shows he oigina funcion f ( x y) and Figs. 4 and 5 show he appoximaion esus. ABLE III. UNED PARAMEERS AND RESULS OF APPROXIMAION Mehod Kene γ a o σ NRMSE ( ain ) ( es ) CV QPSO Wavee Gaussian Wavee Gaussian Figue 3. Oigina funcion Fig. 5. Resuing appoximaion by LS-SVM wih Wavee ene We have compaed he cassificaion and funcion appoximaion esus obained by wavee ene and Gaussian ene whose paamees ae uned wih he coss-vaidaion and QPSO especivey. o summaize he QPSO mehod is bee han he coss-vaidaion o adus he paamees of LS- SVM modes in hese hee expeimens and he wavee ene has bee pefomance han Gaussian ene. V. CONCLUSION In his pape we discuss a pacica way o une he eguaizaion paamee and he ene paamee wih uanum paice swam opimizaion (QPSO) which aes fu advanage of paice swam opimizaion (PSO) and updae wih uanum individua. his wo povides a new adusing paamees of SVM appoach. hee simuaions of LS-SVM mode wih wavee ene and Gaussian ene show ha he poposed mehod is effecive and efficien and enhanced LS- SVM wih wavee ene shows good geneaizaion abiiy on cassificaion pobem and gives bee appoximaion on funcion esimaion. he main aacion of he poposed uning mehod is ha i is simpe cacuaion of impemenaion wih bee pefomances in compaison wih he coss-vaidae mehod. ACKNOWLEDGMEN he fis auho woud ie o han Pof. Ping Wang and D. Min Xiang of Chonging Univesiy of Poss and eecommunicaions of China fo hei hep. his wo was Suppoed by Naiona High-ech R&D Pogam of China (No. 6AA43-) and Scienific Reseach Foundaion fo PhD of Chonging Univesiy of Poss and eecommunicaions (No. A7-5). Figue 4. Resuing appoximaion by LS-SVM wih Gaussian ene REFERENCES [] V.N. Vapni he Naue of Saisica Leaning heoy (he second ediion) New Yo: Spinge-Veag 998. []. Poggio R. Rifin S. Muheee P. Niyogi Genea Condiions fo Pediciviy in Leaning heoy Naue vo. 48 pp Ma.4. [3] M. Caozza S. Rampone owads an incemena SVM fo egession Poc. Inenaiona Join Confeence on Neua Newos Como Iay Ju. pp [4] Z. Y. Luo Z.. Shi Fau diagnosis of anaog cicuis using SVM wih Bayesian Famewo Jouna of Sysem Simuaion vo. 9 pp Juy. 7.

6 [5] O. Chapee V. Vapni O. Bousue and S. Muheee Choosing muipe paamees fo suppo veco machines Machine Leaning vo. 46 pp Jan.. [6] O. Chapee V. Vapni Mode seecion fo suppo veco machines in Handboo of Advances in Neua infomaion Pocessing Sysems Cambidge: Mass MI Pess. [7] J.. Kow he evidence famewo appied o suppo veco machines IEEE ansacion on Neua Newo vo. pp Sep./Oc.. [8] J. Kennedy R. C. Ebeha Paice Swam Opimizaion Poc. IEEE Inenaiona Confeence on neua newos Nagoya Japan 995 pp [9] R. C. Ebeha J. Kennedy A new opimize using paice swam heoy Poc. of he sixh inenaiona symposium on mico machine and human science Nagoya Japan 995 pp [] S. Y. Yang M. Wang L. C. Jiao A Quanum Paice Swam Opimizaion Poc. IEEE Congess on Evouionay Compuaion CEC 4 Poand Oegon USA June 4 pp [] L. D. Oiveia F. Ciiaco. Ab ao P. J. E. Jeszensy Paice Swam and Quanum Paice Swam Opimizaion Appied o DS/CDMA Muiuse Deecion in Fa Rayeigh Channes Poc. IEEE Ninh Inenaiona Symposium on Spead Specum echniues and Appicaions Manaus Amazon Bazi Aug. 6 p [] J. A. K. Suyens. Van Gese J. De Babane B. De Moo and J. Vandewae Leas suaes suppo veco machines Singapoe: Wod Scienific pess. [3] L. Zhang W. D. Zhou L. C. Jiao Wavee Suppo Veco Machine IEEE ansacion on Sysems Man and Cybeneics-Pa B: Cybeneics vo. 34 pp Jan./Feb. 4. [4] C. L. Bae C. J. Mez UCI Reposioy of machine eaning daabases Ivine CA: Univesiy of Caifonia Dep. of Infomaion and Compue Science 998.

Parameter Dynamic Estimation in Near-Field Bistatic MIMO Radar System

Parameter Dynamic Estimation in Near-Field Bistatic MIMO Radar System Jouna of Communicaions Vo. No. Novembe 5 Paamee Dynamic Esimaion in Nea-Fied Bisaic MIMO Rada Sysem i i and ianshuang Qiu Infomaion Engineeing Coege Daian Univesiy Daian China Daian Univesiy of echnoogy