The Particle Swarm: Explosion, Stability, and Convergence in a Multi-Dimensional Complex Space

Transcription

1 Th Paril Swarm: Explosion Sabiliy and Conrgn in a Muli-Dimnsional Complx Spa Mauri Clr Fran Téléom Fran Mauri.Clr@WriM.om Jams Knndy Burau of Labor Saisis Washingon D.C. Knndy_Jim@bls.go Absra Th paril swarm is an algorihm for finding opimal rgions of omplx sarh spas hrough inraion of indiiduals in a populaion of parils. Though h algorihm whih is basd on a maphor of soial inraion has bn shown o prform wll rsarhrs ha no adqualy xplaind how i works. Furhr radiional rsions of h algorihm ha had som dynamial propris ha wr no onsidrd o b dsirabl noably h parils loiis ndd o b limid in ordr o onrol hir rajoris. Th prsn papr analyzs h paril s rajory as i mos in disr im h algbrai iw hn progrsss o h iw of i in oninuous im h analyial iw. A 5-dimnsional dpiion is dlopd whih omplly dsribs h sysm. Ths analyss lad o a gnralizd modl of h algorihm onaining a s of offiins o onrol h sysm s onrgn ndnis. Som rsuls of h paril swarm opimizr implmning modifiaions drid from h analysis suggs mhods for alring h original algorihm in ways ha limina problms and inras h opimizaion powr of h paril swarm.

2 . INTRODUCTION Paril swarm adapaion has bn shown o sussfully opimiz a wid rang of oninuous funions Anglin 998; Knndy and Ebrhar 995; Knndy 997; Knndy 998; Shi and Ebrhar 998. Th algorihm whih is basd on a maphor of soial inraion sarhs a spa by adjusing h rajoris of indiidual ors alld parils as hy ar onpualizd as moing poins in mulidimnsional spa. Th indiidual parils ar drawn sohasially oward h posiions of hir own prious bs prforman and h bs prious prforman of hir nighbors. Whil mpirial idn has aumulad ha h algorihm works.g. i is a usful ool for opimizaion hr has hus far bn lil insigh ino how i works. Th prsn analysis bgins wih a highly simplifid drminisi rsion of h paril swarm in ordr o proid undrsanding abou how i sarhs h problm spa Knndy 998 hn oninus on o analyz h full sohasi sysm. A gnralizd modl is proposd inluding mhods for onrolling h onrgn propris of h paril sysm. Finally som mpirial rsuls ar gin showing h prforman of arious implmnaions of h algorihm on a sui of s funions.. Th paril swarm r r A populaion of parils is iniializd wih random posiions x i and loiis i and a funion f is aluad using h paril s posiional oordinas as inpu alus. Posiions and loiis ar adjusd and h funion aluad wih h nw oordinas a ah im-sp. Whn a paril disors a parn ha is br han any i r r has found priously i sors h oordinas in a or p i. Th diffrn bwn pi h bs poin found by i so far and h indiidual s urrn posiion is sohasially addd o h urrn loiy ausing h rajory o osilla around ha poin. Furhr ah paril is dfind wihin h onx of a opologial nighborhood omprising islf and som ohr parils in h populaion. Th sohasially wighd diffrn bwn h r nighborhood s bs posiion p g and h indiidual s urrn posiion is also addd o is loiy adjusing i for h nx im-sp. Ths adjusmns o h paril s momn hrough h spa aus i o sarh around h wo bs posiions. Th algorihm in psudood follows: Iniializ populaion Do For i o Populaion Siz r r r r if f x i <f pi hn p i x i r r min p nighbors p g For d o Dimnsion p x p x id id id id id sign id min abs id Vmax x id xid id Nx d Nx i Unil rminaion ririon is m gd id Th ariabls and ar random posii numbrs drawn from a uniform disribuion and dfind by an uppr limi max whih is a paramr of h sysm. In his rsion h rm ariabl id is limid o h rang ± Vmax r for rasons whih will b xplaind blow. Th alus of h lmns in p g ar drmind by omparing h bs prformans of all h mmbrs of i s opologial nighborhood dfind by indxs of som ohr populaion

3 r mmbrs and assigning h bs prformr s indx o h ariabl g. Thus p g by any mmbr of h nighborhood. p id rprsns h bs posiion found Th random wighing of h onrol paramrs in h algorihm rsuls in a kind of xplosion or a drunkards walk as parils loiis and posiional oordinas arn oward infiniy. Th xplosion has radiionally bn onaind hrough implmnaion of a V max paramr whih limis sp-siz or loiy. Th urrn papr howr dmonsras ha h implmnaion of proprly dfind onsriion offiins an prn xplosion; furhr hs offiins an indu parils o onrg on loal opima. An imporan sour of h swarm s sarh apabiliy is h inraions among parils as hy ra o on anohr s findings. Analysis of inrparil ffs is byond h sop of his papr whih fouss on h rajoris of singl parils.. Simplifiaion of h sysm W bgin h analysis by sripping h algorihm down o a mos simpl form; w will add hings bak in lar. Th r paril swarm formula adjuss h loiy i by adding wo rms o i. Th wo rms ar of h sam form ha r is p x r r i whr p is h bs posiion found so far by h indiidual paril in h firs rm or by any nighbor in h sond rm. Th formula an b shornd by rdfining as follows: pid pid pgd Thus w an simplify our iniial insigaion by looking a h bhaior of a paril whos loiy is adjusd by only on rm: whr id p x id. This is algbraially idnial o h sandard wo-rm form. id r Whn h paril swarm opras on an opimizaion problm h alu of p i is onsanly updad as h sysm r ols oward an opimum. In ordr o furhr simplify h sysm and mak i undrsandabl w s p i o a onsan alu in h following analysis. Th sysm will also b mor undrsandabl if w mak a onsan as wll; whr normally i is dfind as a random numbr bwn zro and a onsan uppr limi w will rmo h sohasi omponn iniially and rinrodu i in lar sions. Th ff of on h sysm is ry imporan and muh of h prsn papr is inold in analyzing is ff on h rajory of a paril. Th sysm an b simplifid n furhr by onsidring a -dimnsional problm spa and again furhr by rduing h populaion o on paril. Thus w will bgin by looking a a srippd-down paril by islf.g. a populaion of on on-dimnsional drminisi paril wih a onsan p. id Thus w bgin by onsidring h rdud sysm: p x x x Equ.. whr p and ar onsans. No or noaion is nssary and hr is no randomnss. Knndy 998 found ha h simplifid paril s rajory is dpndn on h alu of h onrol paramr and rognizd ha randomnss was rsponsibl for h xplosion of h sysm hough h mhanism whih ausd h xplosion was no undrsood. Ozan and Mohan 998a; 998b furhr analyzd h sysm and onludd ha h paril as sn in disr im surfs on an undrlying oninuous foundaion of sin was.

4 Th prsn papr analyzs h paril swarm as i mos in disr im h algbrai iw hn progrsss o h iw of i in oninuous im h analyial iw. A 5-dimnsional dpiion is dlopd whih omplly dsribs h sysm. Ths analyss lad o a gnralizd modl of h algorihm onaining a s of offiins o onrol h sysm s onrgn ndnis. Whn randomnss is r-inrodud o h full modl wih onsriion offiins h dlrious ffs of randomnss ar sn o b onrolld. Som rsuls of h paril swarm opimizr using modifiaions drid from h analysis ar prsnd; hs rsuls suggs mhods for alring h original algorihm in ways ha limina som problms and inras h opimizaion powr of h paril swarm.. ALGEBRAIC POINT OF VIEW Th basi simplifid dynami sysm is dfind by y y y Equ.. whr. x p y L b h urrn poin in and h marix of h sysm. In his as w ha and mor gnrally. Thus h sysm is omplly dfind by M. y P MP R M M P P P Th ignalus of M ar: Equ.. W an immdialy s ha h alu is spial; blow w will s wha his implis. For w an dfin a marix A so ha Equ.. L AMA no ha A - dosn xis whn For xampl from h anonial form w find a A a Equ.. In ordr o simplify h formulas w muliply by o produ a marix A:

5 A Equ..5 So if w dfin Q AP w an now wri P A LAP AP LAP Q LQ Equ..6 and finally Q L Q Bu L is a diagonal marix so w ha simply L Equ..7 In pariular hr is yli bhaior in h sysm if and only if Q Q or mor gnrally if Q k Q. This jus mans ha w ha a sysm of wo quaions:. Cas < Equ..8 For << h ignalus ar omplx and hr is always a las on ral soluion for. Mor prisly w an wri os θ isin θ os θ isin θ Equ..9 wih os θ and Thn sin θ. os θ isin θ os θ isin θ kπ and yls ar gin by any θ suh ha θ Equ.. So for ah h soluions for ar gin by kπ os for k {... } Equ..

6 Tabl. gis som nonriial alus of for whih h sysm is yli. Tabl.. Som alus for whih h sysm is yli. Cyl priod s Figur. 5 ± 5 5 s Figur. for h sum and Figur. for h diffrn 6 ± 6 Figurs. hrough. show h rajoris of a paril in phas spa for arious alus of. Whn aks on on of h alus from Tabl. h rajory is ylial for any ohr alu h sysm is jus quasi-yli as in Figur.. W an b a lil bi mor pris. Blow For xampl for and y w ha Q AP is h -norm h Eulidan on for a or Q Q. Equ.. A Q P A max ± P Equ.. max ± Cas > Figurs If > hn and ar ral numbrs and so w ha ihr for n whih implis no onsisn wih h hypohsis > or - whih is impossibl ha is o say no onsisn wih h hypohsis > So and his is h poin hr is no yli bhaior for >. And in fa h disan from h poin nr is srily monooni inrasing wih whih mans ha So Q AP L Q AP L Q AP Equ.. P o h

7 L Q A P L Q A P Equ..5 Bu on an also wri P A P Q A Q Equ..6 P A L Q So finally P inrass lik L Q. In Sion his rsul is usd o prn h xplosion of h sysm whih an our whn paril loiis inras wihou onrol.. Cas In his siuaion M In his pariular as h ignalus ar boh qual o - and hr is jus on family of ignors gnrad by V. So w ha MV V. Thus if P is an ignor proporional o V ha is o say if y hr ar jus wo symmrial poins for y P ± P y Equ..7 In h as whr P is no an ignor w an dirly ompu how P drass and/or inrass. L us dfin P P. By rurrn h following form is drid: a b y y Equ..8 whr a b ar ingrs so ha for y. Th ingrs an b ngai zro or posii. Supposing for a pariular w ha > on an asily ompu y y. This quaniy is posii if and only if is no bwn or qual o h roos { y y } y Now if is ompud hn w ha 5 y 6y and h roos ar y. As < his rsul mans ha is also posii. So as soon as P bgins o inras i dos so infinily. Bu i an dras a h bginning. Th qusion o b answrd nx is how long an i dras bfor i bgins inrasing?

8 Now ak h as of <. This mans ha is bwn -y and -y. For insan in h as whr y > y ε wih ε ] y[. Equ..9 By rurrn h following is drid: Finally y ε ε y ε ε y ε ε ε k y y ε wih k k k Equ.. ε ε Equ.. as long as ε yε whih mans ha P drass as long as y Ingr _ par Equ.. ε Afr ha P inrass. Th sam analysis an b prformd for y <. In his as ε< as wll so h formula is h sam. In fa o b n mor pris if hn w ha y ε ε β ε β β P P Equ.. Thus i an b onludd ha P drass/inrass almos linarly whn is big nough. In pariular n if i bgins o dras afr ha i nds o inras almos lik 5 y.. ANALYTIC POINT OF VIEW. Basi xplii rprsnaion From h basi irai implii rprsnaion h following is drid: y Equ.. No ha h prsn papr uss h Bourbaki onnion of rprsning opn inrals wih rrsd braks. Thus ]ab[ is quialn o parnhial noaion ab.

9 Assuming a oninuous pross his boms a lassial sond ordr diffrnial quaion: ln ln ln whr and ar h roos of As a rsul Equ.. λ λ. Equ.. Th gnral soluion is Equ.. Equ..5 A similar kind of xprssion for y is now produd whr y Equ..6 Th offiins and dpnd on and y. If w ha y y Equ..7 In h as whr Equ..5 and Equ..6 gi so w mus ha y Equ..8 in ordr o prn a disoninuiy. y Equ..9 Rgarding h xprssions and ignalus of h marix M as in Sion abo h sam disussion abou an b mad pariularly abou h non xisn of yls. h sign of Th abo rsuls proid a guidlin for prning h xplosion of h sysm for w an immdialy s ha i dpnds on whhr w ha

10 max >. Equ... A posriori proof On an dirly rify ha and y ar indd soluions of h iniial sysm. On on hand from hir xprssions y Equ.. and on h ohr hand y Equ.. and also [ ] [ ] y y Equ... Gnral implii and xplii rprsnaions IR and ER A mor gnral implii rprsnaion IR is produd by adding fi offiins { } η β whih will allow us o idnify how h offiins an b hosn in ordr o nsur onrgn. Wih hs offiins h sysm boms { } * R y N R y y y η β Equ.. Th marix of h sysm is now. L b is ignalus. η β M and Th xplii analyi rprsnaion ER boms

11 wih y * R N β { y } R Equ..5 βy βy Equ..6 Now h onsriion offiins s Sion for dails and ar dfind by. Equ..7 wih Equ..8 whih ar h ignalus of h basi sysm. By ompuing h ignalus dirly and using Equ..7 ar: and η η η η η β η η η β Equ..9 Th final ompl xplii rprsnaion an hn b wrin from.5 and.6 by rplaing and rspily by and and hn by hir xprssions as sn in Equ..8 and.9. I is immdialy worh noing an imporan diffrn bwn IR and ER. In h IR is always an ingr and and y ar ral numbrs. In h ER ral numbrs ar obaind if and only if is an ingr; nohing howr prns h assignmn of any ral posii alu o in whih as and y bom ru omplx numbrs. This fa will proid an lgan way of xplaining h sysm s bhaior by onpualizing i in a 5-dimnsional spa as disussd in Sion.

12 No. If and ar o b ral numbrs for a gin alu hr mus b som rlaions among h fi ral offiins { } η β. If h imaginary pars of and ar s qual o zro h following is obaind: A B E sign E C E sign E A B E sign E C E sign E η η Equ.. wih β η η η E sign C D sign C sign C B sign A Equ.. Th wo qualiis of Equ.. an b ombind and simplifid as follows: A E Bsign E A B E sign E η Equ.. Th soluions ar usually no omplly indpndn of. In ordr o saisfy hs quaions a s of possibl ondiions is < > η A E Equ.. Bu hs ondiions ar no nssary. For xampl an inrsing pariular siuaion sudid blow xiss whr. In his as * R η β for any alu and Equ.. is always saisfid.. From ER o IR Th xplii rprsnaion will b usful o find onrgn ondiions. Nrhlss in prai h irai form obaind from Equ..9 is ry usful: β η η η η Equ..

13 Alhough hr ar an infiniy of soluions in rms of h fi paramrs { } η β i is inrsing o idnify som pariular lasss of soluions. This will b don in h nx sion..5 Pariular Classs of Soluions.5. Class modl Th firs modl implmning h fi-paramr gnralizaion is dfind by h following rlaions: η β Equ..5 In his pariular as and η ar η Equ..6 An asy way o nsur ral offiins is o ha R. Undr his addiional ondiion a lass of soluion is simply gin by η β Equ Class modl A rlad lass of modl is dfind by h following rlaion: η β Equ..8 Th following xprssions for and ar drid from Equ..: 8 m Equ..9 If h ondiion is addd hn

14 or Equ.. Wihou his ondiion on an hoos a alu for for xampl and a orrsponding alu whih gi a onrgn sysm..5. Class " modl A sond modl rlad o h Class formula is dfind by η β Equ.. Equ.. For hisorial rasons and for is simpliiy h as has bn wll sudid. S Sion. for furhr disussion..5. Class modl A sond lass of modls is dfind by h rlaions η β Equ.. Undr hs onsrains i is lar ha Equ.. whih gis us and rspily. Again an asy way o obain ral offiins for ry alu is o ha. In his as Equ..5 In h as whr h following is obaind: Equ..6 From h sandpoin of onrgn i is inrsing o no ha w ha

15 for h Class modls wih h ondiion Equ..7 for h Class modls wih h ondiions and Equ..8 and for h h Class modls wih lass lass Equ..9 This mans ha w will jus ha o hoos < < and lass < rspily o ha a onrgn sysm. This will b disussd furhr in Sion..6 Rmoing h disoninuiy Dpnding on h paramrs { } η β h sysm may ha a disoninuiy in du o h prsn of h rm η β η in h ignalus. Thus in ordr o ha a omplly oninuous sysm h alus for { } η β mus b hosn suh ha { } 5 η β η η β R R Equ.. By ompuing h disriminan h las ondiion is found o b quialn o > η β β Equ.. In ordr o b physially plausibl h paramrs { } η β mus b posii. So h ondiion boms η β < Equ.. Th s of ondiions akn oghr spify a olum in for h admissibl alus of h paramrs. 5 R

16 .7 Rmoing h imaginary par Whn h ondiion spifid in Equ.. is m h rajory is usually sill parly in a omplx spa whnr on of h ignalus is ngai du o h fa ha is a omplx numbr whn is no an ingr. In ordr o prn his w mus find som srongr ondiions in ordr o mainain posii ignalus. Sin > > > > Equ.. h following ondiions an b usd o nsur posii ignalus: η β > η > Equ.. No. From an algbrai poin of iw h ondiions dsribd in Equ.. an b wrin as d M > Equ..5 ra M > Bu now hs ondiions dpnd on. Nrhlss if h maximum alu is known hy an b rwrin as: > η β > η max max Equ..6 Undr hs ondiions all sysm ariabls ar ral numbrs. in onjunion wih h ondiions in Equ s.. and. h paramrs an b sld so ha h sysm is omplly oninuous and ral..8 Exampl As an xampl suppos ha β and η. Now h ondiions bom For xampl whn < max max max < < Equ..7

17 max y β max max.99 η.99 max.895 Equ..8 h sysm onrgs qui quikly afr abou 5 im sps and a ah im sp h alus of y and ar almos h sam or a larg rang of alus. Figur. shows an xampl of onrgn and y for a oninuous ral-alud sysm wih..9 Raliy and onrgn Th quik onrgn sn in h abo xampl suggss an inrsing qusion. Dos raliy using ral-alud ariabls imply onrgn? In ohr words dos h following hold for ral-alud sysm paramrs? > max < η β > < max η Equ..9 Th answr is no. I an b dmonsrad ha onrgn is no always guarand for ral-alud ariabls. For xampl gin h following paramrizaion: max y β η.99 Equ..5 h rlaions ar.5 > max η β. > max η Equ..5 whih will produ sysm dirgn whn. for insan sin.9 >. This is sn in Figur Figurs. and

18 . CONVERGENCE AND SPACE OF STATES From h gnral xplii rprsnaion w find h ririon of onrgn: < < Equ.. whr and y ar usually ru omplx numbrs. Thus h whol sysm an b rprsnd in a 5-dimnsion spa R y Im y R Im. In his sion w sudy som xampls of h mos simpl lass of onsrid ass: h ons wih jus on onsriion offiin. Ths will allow us o dis mhods for onrolling h bhaior of h swarm in ways ha ar dsirabl for opimizaion.. Consriion For Modl Typ Modl Typ is dsribd as follows: y y y W ha sn ha h onrgn ririon is saisfid whn offiin blow is produd: ] [ Equ.. < min. Sin h onsriion Equ... Consriion for modl Typ Jus as a onsriion offiin was found for h Typ modl h following implii rprsnaion wih insad of is usd for Typ : y y y Equ.. Th offiin boms ] [ for ] [. Equ..5 Bu as sn abo his formula is a priori alid only whn < so i is inrsing o find anohr onsriion offiin ha has dsirabl onrgn propris. W ha hr Equ..6 Th xprssion undr h squar roo is ngai for ] [ ru omplx numbr and ha ] [. In his as h ignalu is a. Thus if < ha is o say if < a nds o b sld suh in ordr o saisfy h onrgn ririon. So for xampl dfin as

19 for ] [ Equ..7 Now an anohr formula for grar alus b found? Th answr is No. For in his as is a ral numbr and is absolu alu is and h minimal alu is grar han srily drasing on ] ] srily drasing on [ [ wih a limi of. For simpliiy h formula an b h sam as for Typ no only for < bu also for <. This is indd also possibl bu hn an no b oo small dpnding on. Mor prisly h onsrain > mus b saisfid. Bu as for < w ha his jus mans ha h urs in Figurs. and. an hn b inrprd as h man and minimally apabl alus for sur onrgn. For xampl for h onsrain >.56 mus hold. Bu hr is no suh rsriion on ] [ if No. Figurs. and Th abo analysis is for onsan. If is random i is nrhlss possibl o ha onrgn n wih a small onsriion offiin whn a las on alu is srily insid h inral of ariaion.. Consriion yp " Rfrring o h Class " modl in h pariular as whr w us h following implii rprsnaion wih insad of y Equ..8 y y In fa his sysm is hardly diffrn from h lassial paril swarm as dsribd in h Sion p x Equ..9 x x so i may b inrsing o dail how in prai h onsriion offiin is found and is onrgn propris pron. Sp. Marix of h sysm W ha immdialy M. Equ.. Sp. Eignalus Thy ar h wo soluions for h quaion or Thus ra M Z drminan M Z Equ.. Z Z Equ..

20 wih ra M drminan M Equ.. Equ.. Sp. Complx and ral aras on Th disriminan is ngai for h omplx numbrs and hir absolu alu i.. modul is simply. Sp. Exnsion of h omplx rgion and onsriion offiin alus in. In his ara h ignalus ar ru In h omplx rgion aording o h onrgn ririon < in ordr o g onrgn. So h ida is o find a onsriion offiin dpnding on so ha h ignalus ar ru omplx numbrs for a larg fild of alus. This is gnrally h mos diffiul sp and somims nds som inuiion. Thr pis of informaion hlp us hr: Th drminan of h marix is qual o This is h sam as in Consriion Typ and W know from h algbrai poin of iw h sysm is nually onrgn lik M So i appars ry probabl ha h sam onsriion offiin usd for Typ will work. Firs w ry ] [ Equ..5 T for > ha is o say ls In his as h ommon absolu alu of h ignalus is Equ..6 ls for > Equ..7 whih is smallr han for all alus as soon as is islf smallr han.

21 I is asy o s ha is ngai only bwn max is qui ompliad polynomial in asir o ompu i indirly for som alus. If min and max dpnding on. Th gnral algbrai form of 6 wih som offiins bing roos of an quaion in min is smallr han hn find ha min. This rlaion is alid as soon as dpnds on for wo alus. I is ngai bwn h alus gin blow: min max Modra onsriion Figur so i is muh and by soling w. Figur. shows how h disriminan 9 Whil i is dsirabl for h paril s rajory o onrg by rlaxing h onsriion h paril is allowd o osilla hrough h problm spa iniially sarhing for impromn. Thrfor i is dsirabl o onsri h sysm modraly prning xplosion whil sill allowing for xploraion. To dmonsra how o produ modra onsriion h following xplii rprsnaion is usd: y ] [ Equ..8 ha is o say. From h rlaions bwn ER and IR h following is obaind: η Equ..9 η η η β Thr is sill an infiniy of possibiliis for sling h paramrs..η. In ohr words hr ar many diffrn implii rprsnaions ha produ h sam xplii on. For xampl: or β η Equ..

22 β η Equ.. From a mahmaial poin of iw his as is rihr han h prious ons. Thr is no mor xplosion bu hr is no always onrgn ihr. This sysm is sabilizd in h sns ha h rprsnai poin in h sa spa nds o mo along an araor whih is no always rdud o a singl poin as in lassial onrgn.. Araors and onrgn Figurs. hrough.7 show h ral rsriions R R an larly s h hr ass: Spiral asy onrgn owards a nonriial araor for < Figur.5 Diffiul onrgn for Figur.6 and Quik almos linar onrgn for > Figur.7. y of h parils ha ar ypially sudid. W Figur Figur Nrhlss i is inrsing o ha a look a h ru sysm inluding h omplx dimnsions. Figurs.8 hrough. show som ohr sions of h whol surfa in R Figur.. Global araor for and. Axis R Im No. Thr is a disoninuiy for h radius is qual o zro for > Figur Thus wha i sms o b an osillaion in h ral spa is in fa a oninuous spirali momn in a omplx spa. Mor imporanly h araor is ry asy o dfin: i is h irl nr and radius ρ. Whn < ρ and whn > hn ρ lim wih < for h onsriion offiin has bn prisly hosn so ha h par of nds o zro. This proids an inuii way o ransform

23 his sabilizaion ino a ru onrgn. W jus ha o us a sond offiin in ordr o rdu h araor in h as < so ha x ] [ Equ.. Th modls sudid hr ha only on onsriion offiin. If on ss h Typ onsriion is produd. Bu now w undrsand br why i works. 5. GENERALIZATION OF THE PARTICLE SWARM SYSTEM Thus far h fous has bn on a spial rsion of h paril swarm sysm a sysm rdud o salars ollapsd rms and nonprobabilisi bhaior. Th analyi findings hough an asily b gnralizd o h mor usual as whr is random and wo or rms ar addd o h loiy. In his sion h rsuls ar gnralizd bak o h original sysm as dfind by x x x p x p Equ. 5. Now p and y ar dfind o b x p y p p p Equ. 5. o obain xaly h original nonrandom sysm dsribd in Sion. For insan if hr is a yl for hn hr is an infiniy of yls for h alus { } so ha. Upon ompuing h onsriion offiin h following form is obaind: ] [ ls if > Equ. 5. Coming bak o h x sysm and x ar p p x x p x p Equ. 5.

24 Th us of h onsriion offiin an b iwd as a rommndaion o h paril o ak smallr sps. p p Th onrgn is owards h poin x. Rmmbr is in fa h loiy of h paril so i will indd b qual o zro in a onrgn poin. Exampl x.5 p p max. max 5 and ar uniform random ariabls bwn and max and max rspily. This xampl is shown in Figur Figur RUNNING THE PARTICLE SWARM WITH CONSTRICTION COEFFICIENTS As a rsul of h abo analysis h paril swarm algorihm an b onid of in suh a way ha h sysm s xplosion an b onrolld wihou rsoring o h dfiniion of any arbirary or problm-spifi paramrs. No only an xplosion b prnd bu h modl an b paramrizd in suh a way ha h paril sysm onsisnly onrgs on loal opima. Exp for a spial lass of funions onrgn on global opima anno b pron. Th paril swarm algorihm an now b xndd o inlud many yps of onsriion offiins. Th mos gnral modifiaion of h algorihm for minimizaion is prsnd in h following psudood: Assign max Calula β η Iniializ populaion: random x i i Do For i o populaion siz if fx i <fp i hn p i x i For d o dimnsion rand max / rand max / p*p id *p gd / xx id id id * β**p - x x id p * - η* * p x Nx d Conrgn implis loiy bu h onrgn poin is no nssarily h on w wan pariularly if h sysm is oo onsrid. W hop o show in a lar papr how o op wih his problm by dfining h opimal paramrs.

25 Nx i Unil rminaion ririon is m In his gnralizd rsion of h algorihm h usr sls h rsion and hooss alus for and ha ar onsisn wih i. Thn h wo ignalus ar ompud and h grar on is akn. This opraion an b prformd as follows: η β disrim µ a if disrim> hn nprim abs a disrim nprim abs a disrim ls nprim a abs disrim nprimnprim maxig.maxnprim nprim η Ths sps ar akn only on in ah program and hus do no slow i down. For h rsions sd in his papr h onsriion offiin is alulad simply as. For insan h Typ rsion is maxig. dfind by h ruls β η. Th gnralizd dsripion allows h usr o onrol h dgr of onrgn by sing o arious alus. For insan in h Typ rsion. rsuls in slow onrgn maning ha h spa is horoughly sarhd bfor h populaion ollapss ino a poin. In fa h Typ onsriion paril swarm an b programmd as a ry simpl modifiaion o h sandard rsion prsnd in h Inroduion. Th onsriion offiin is alulad as shown in Equaion.6: for > ls Th offiin is hn applid o h righ sid of h loiy adjusmn: Calula Iniializ populaion Do For i o Populaion Siz r r r r if f x i <f pi hn pi x i r r min p nighbors p g For d o Dimnsion p x id x x id Nx d Nx i Unil rminaion ririon is m p x id id id gd id id id

26 No ha h algorihm now rquirs no xplii limi V max. Th onsriion offiin maks i unnssary. Ebrhar and Shi ha rommndd basd on hir xprimns ha a libral V max for insan on ha is qual o h dynami rang of h ariabl b usd in onjunion wih h Typ onsriion offiin. Though his xra paramr may nhan prforman h algorihm will sill run o onrgn n if i is omid. 7. EMPIRICAL RESULTS Sral yps of paril swarms wr usd o opimiz a s of unonsraind ral-alud bnhmark funions namly sral of D Jong s 975 funions Shaffr s f6 and h Griwank Rosnbrok and Rasrigin funions. A populaion of parils was run for wny rials pr funion wih h bs prforman aluaion rordd afr iraions. Som rsuls from Anglin s 998 runs using an oluionary algorihm ar shown for omparison. Though hs funions ar ommonly usd as bnhmark funions for omparing algorihms diffrn rsions of hm ha appard in h liraur. Th formulas usd hr for D Jong s f f f wihou nois f5 and Rasrigin funions ar akn from Rynolds and Chung 997. Shaffr s f6 funion is akn from Dais 99 rd diion; no ha arlir diions gi a somwha diffrn formula. Th Griwank funion gin hr is h on usd in h Firs Inrnaional Cons on Eoluionary Opimizaion hld a ICEC 96 and h Rosnbrok funion is akn from Anglin 998. Tabl 7.. Funions usd o s h ffs of h onsriion offiins. Sphr funion D Jong s f f x x Rosnbrok arian D Jong s f D Jong s f no nois Foxhols D Jong s f5 Shaffr s f6 Griwank funion Rosnbrok funion f f f f n i i x x x x n x i 5 6 x i x i. 5 6 j j i x i aij sin x y.5 x.5.. x y n n f os 7 x i i x x i i i n f 9 x xi x xi Rasrigin funion [ ] n f x i x osπ xi 7.. Algorihm ariaions usd i i Thr ariaions of h gnralizd paril swarm wr usd on h problm sui. Typ. Th firs rsion applid h onsriion offiin o all rms of h formula: β η using. 8. Typ. Th sond rsion sd was a simpl onsriion whih was no dsignd o onrg bu no o xplod ihr as was assignd a alu of.. Th modl was dfind as: β η.

27 Exprimnal rsion. Th hird rsion sd was mor xprimnal in naur. Th onsriion offiin was iniially dfind as. If > hn i was muliplid by.9 iraily unil. On a max saisfaory alu was found h following modl was implmnd: β η As in h firs rsion a gnri alu of. 8 was usd. Tabl 7. displays h problm-spifi paramrs implmnd in h xprimnal rials. 7.. Rsuls Tabl 7. Funion paramrs for h s problms. Funion Dimnsion Iniial Rang ± ±5 ± 5 ±5 Shaffr s f6 ± Griwank ± Akly ± Rasrigin ±5. Rosnbrok ± Tabl 7. blow ompars arious onsrid paril swarms prforman o ha of h radiional V max paril swarm and oluionary opimizaion EO rsuls rpord by Anglin 998. All paril swarm populaions omprisd indiiduals. Funions wr implmnd in dimnsions xp for f f5 and f6 whih ar gin for wo dimnsions. In all ass xp f5 h globally opimal funion rsul is.. For f5 h bs known rsul is.998. Th limi of h onrol paramr was s o. for h onsrid rsions and. for h V max rsions of h paril swarm. Th olumn labld E&S was programmd aording o h rommndaions of Ebrhar and Shi. This ondiion inludd boh Typ onsriion and V max wih V max s o h rang of h iniial domain for h funion. Funion rsuls wr sad wih six dimal plas of prision. Tabl 7.. Empirial rsuls. Man bs aluaions a h nd of iraions for arious rsions of paril swarm and Anglin s 998 oluionary algorihm. Funion V max V max Typ " Typ Exp. E&S Anglin 998 Vrsion Shaffr s f Griwank Akly Rasrigin Rosnbrok

28 As an b sn h Typ " and Typ onsrid rsions ouprformd h V max rsions in almos ry as; h xprimnal rsion was somims br somims no. Furhr h Typ " and Typ onsrid paril swarms prformd br han h omparison oluionary mhod on hr of h four funions; wih som auion w an a las onsidr h prformans o b omparabl. Ebrhar and Shi s suggsion o hdg h sarh by raining V max wih Typ " onsriion dos sm o rsul in good prforman on all funions. I is h bs on all xp h Rosnbrok funion whr prforman was sill rspabl. An analysis of arian was prformd omparing h E&S rsion wih Typ " sandardizing daa wihin funions. I was found ha h algorihm had a signifian main ff F. p<.6 bu ha hr was a signifian inraion of algorihm wih funion F8.68 p<. suggsing ha h gain may no b robus aross all problms. Ths rsuls suppor hos of Ebrhar and Shi. Any omparison wih Anglin s oluionary mhod should b onsidrd auiously. Th omparison is offrd only as a prima fai sandard by whih o assss prformans on hs funions afr his numbr of iraions. Thr ar numrous rsions of h funions rpord in h liraur and i is xrmly likly ha faurs of h implmnaion ar rsponsibl for som arian in h obsrd rsuls. Th omparison hough dos allow h radr o onfirm ha onsrid paril swarms ar omparabl in prforman o a las on oluionary algorihm on hs s funions. As has long bn nod h V max paril swarm suds a finding opimal rgions of h sarh spa bu has no faur ha nabls i o onrg on opima.g. Anglin 998. Th onsriion hniqus rpord in his papr sol his problm hy do for onrgn. Th daa larly india an inras in h abiliy of h algorihm o find opimal poins in h sarh spa for hs problms as a rsul. No algorihmi paramrs wr adjusd for any of h paril swarm rials. Paramrs suh as V max populaion siz. wr hld onsan aross funions. Furhr i should b mphasizd ha h populaion siz of is onsidrably smallr han wha is usually sn in oluionary mhods rsuling in fwr funion aluaions and onsqunly fasr lok im in ordr o ahi a similar rsul. For insan Anglin s rsuls id for omparison ar basd on populaions of CONCLUSIONS This papr xplors how h paril swarm algorihm works from h insid ha is from h indiidual paril s poin of iw. How a paril sarhs a omplx problm spa is analyzd and impromns o h original algorihm basd on his analysis ar proposd and sd. Spifially h appliaion of onsriion offiins allows onrol or h dynamial hararisis of h paril swarm inluding is xploraion rsus xploiaion propnsiis. Though h psudood in Sion 6 may look diffrn from prious paril swarm programs i is ssnially h sam algorihm rarrangd o nabl h judiious appliaion of analyially-hosn offiins. Th aual implmnaion may b as simpl as ompuing on onsan offiin and using i o wigh on rm in h formula. Th Typ " mhod in fa rquirs only h addiion of a singl offiin alulad on a h sar of h program wih almos no inras in im or mmory rsours. In h urrn analysis h sin was idnifid by Ozan and Mohan 998a; 998b urn ou o b h ral pars of h 5-dimnsional araor. In omplx numbr spa.g. in oninuous im h paril is sn o spiral oward an araor whih urns ou o b qui simpl in form: a irl. Th ral-numbr sion by whih his is obsrd whn im is rad disrly is a sin wa. Th 5-dimnsional prspi omplly summarizs h bhaior of a paril and prmis h dlopmn of mhods for onrolling h xplosion ha rsuls from randomnss in h sysm. Coffiins an b applid o arious pars of h formula in ordr o guaran onrgn whil nouraging xploraion. Sral kinds of offiin adjusmns ar suggsd in h prsn papr bu w ha barly srahd h surfa and plny of xprimns should b prompd by hs findings. Simpl modifiaions basd on h prsn analysis rsuld in

29 an opimizr whih appars from hs prliminary rsuls o b abl o find h minima of som xrmly omplx bnhmark funions. Ths modifiaions an guaran onrgn whih h radiional V max paril swarm dos no. I is possibl o modify h algorihm wih an inras in ffiiny and wihou inrasing is omplxiy in fa i suggss ha no problm-spifi paramrs may nd o b spifid. W rmind h radr ha h ral srngh of h paril swarm dris from h inraions among parils as hy sarh h spa ollaboraily. Th sond rm addd o h loiy is drid from h susss of ohrs i is onsidrd a soial influn rm; whn his ff is rmod from h algorihm prforman is r abysmal Knndy 997. Effily h ariabl p g kps moing as nighbors find br and br poins in r h sarh spa and is wighing rlai o p i aris wih randomly wih ah iraion. As a paril swarm populaion sarhs or im indiiduals ar drawn oward on anohr s susss wih h usual rsul bing lusring of indiiduals in opimal rgions of h spa. Th analysis of h soial-influn asp of h algorihm is a opi for a fuur papr.

30 REFERENCES Anglin P Eoluionary opimizaion rsus paril swarm opimizaion: Philosophy and prforman diffrns. In V. W. Poro Saraanan N. Waagn D. and Eibn A. E. Eds. Eoluionary Programming VII 6-6. Brlin: Springr. Dais L. Ed. 99. Handbook of Gni Algorihms. Nw York NY: Van Nosrand Rinhold. D Jong K An analysis of h bhaior of a lass of gni adapi sysms. PhD hsis Unirsiy of Mihigan. Ebrhar R. C. and Shi Y.. Comparing inria wighs and onsriion faors in paril swarm opimizaion. Prodings of h Inrnaional Congrss on Eoluionary Compuaion San Digo California IEEE Sri Cnr Pisaaway NJ Knndy J Th paril swarm: Soial adapaion of knowldg. Prodings of h 997 Inrnaional Confrn on Eoluionary Compuaion Indianapolis Indiana IEEE Sri Cnr Pisaaway NJ -8. Knndy J. and Ebrhar R. C Paril swarm opimizaion. Prodings of h IEEE Inrnaional Confrn on Nural Nworks Prh Ausralia IEEE Sri Cnr Pisaaway NJ IV: Knndy J Mhods of agrmn: Infrn among h lmnals. Prodings of h 998 IEEE Inrnaional Symposium on Inllign Conrol ISIC Hld Joinly wih h Inrnaional Symposium on Compuaional Inllign in Robois and Auomaion and h Inllign Sysms and Smiois ISAS A Join Confrn on h Sin and Thnology of Inllign Sysms. Ozan E. and Mohan C. K. 998a. Trajory analyss for a simpl paril swarm opimizaion sysm. Foundaions of Gni Algorihms 998. Ozan E. and Mohan C. K. 998b. Th bhaior of parils in a simpl PSO sysm. Prodings of ANNIE. Rynolds R. G. and Chung C-J 997. Knowldg-basd slf-adapaion in oluionary programming using ulural algorihms. Prodings of h 997 Inrnaional Confrn on Eoluionary Compuaion Indianapolis Indiana IEEE Sri Cnr Pisaaway NJ Shi Y. and Ebrhar R. C Paramr slion in paril swarm adapaion. In V. W. Poro Saraanan N. Waagn D. and Eibn A. E. Eds. Eoluionary Programming VII Brlin: Springr.