Fast Multi-swarm Optimization with Cauchy Mutation and Crossover operation

Transcription

1 Fast Mult-swarm Optmzato wth Cauchy Mutato ad Crossover operato Qg Zhag,, Chaghe L, Y. Lu 3, Lsha Kag Cha Uversty of Geosceces, School of Computer, Wuha, P.R.Cha, 4374 Huaggag Normal Uversty 3 The Uversty of Azu,Azu-Wakamatsu, Fukushma, Japa, zhagqg@hgc.et,lch_wfx@yahoo.com.c,ylu@u-azu.ac.jp, kag_whu@yahoo.com Abstract. The stadard Partcle Swarm Optmzato () algorthm s a ovel evolutoary algorthm whch each partcle studes ts ow prevous best soluto ad the group s prevous best to optmze problems. Oe problem exsts s ts tedecy of trappg to local optma. I ths paper, a multple swarms techque() based o fast partcle swarm optmzato(f) algorthm s proposed by brgg crossover operato. F s a global search algorthm wtch ca prevet from trappg to local optma by troducg Cauchy mutato. Though t ca get hgh optmzg precso, the covergece rate s ot satsfed, ot oly ca fd satsfed solutos,but also speeds up the search. By proposg a ew formato exchagg ad sharg mechasm amog swarms. By comparg the results o a set of bechmark test fuctos, shows a compettve performace wth the mproved covergece speed ad hgh optmzg precso. Keywords: Partcle swarm optmzato, Cauchy mutato, swarm tellgece. Itroducto Partcle Swarm Optmzato () was frst troduced by Keedy ad Eberhart 995 [,]. It s motvated from the socal behavor of orgasms, such as brd flockg ad fsh schoolg. Partcles fly through the search space by followg the prevous best postos of ther eghbors ad ther ow prevous best postos. Each partcle s represeted by a posto ad a velocty whch are updated as follows: X = X + V () V = ωv + ηrad ()( P X ) () + η rad ()( Pgd X ) where X ad X represet the curret ad the prevous postos of th partcle, V

2 ad V are the prevous ad the curret velocty of th partcle, P ad P gd are the dvuals best posto ad the best posto foud the whole swarm so far respectvely. ω < s a erta weght whch determes how much the prevous velocty s preserved, η ad η are accelerato costats, rad() geerates radom umber from terval [,]. I, each partcle shares the formato wth ts eghbors. combes the cogto compoet of each partcle wth the socal compoet of all the partcles a group. Although the speed of covergece s very fast, Oce traps to local optmum, t s dffcult to jump out of local optmum. Rataweera et.al.[3] state that lack of populato dversty algorthms s uderstood to be a factor ther covergece o local optma. Therefore, the addto of a mutato operator to should ehace ts global search capacty ad thus mprove ts performace. A frst attempt to model partcle swarms usg the quatum model(q) was carred out by Su et.al. [4]. I a quatum model, partcles are descrbed by a wave fucto stead of the stadard posto ad velocty. The quatum Delta potetal well model ad quatum harmoc oscllators are commoly used partcle physcs to descrbe the stochastc ature of partcles. I ther studes[5], the varable of gbest (the global best partcle ) ad mbest (the mea value of all partcles prevous best posto) s mutated wth Cauchy dstrbuto respectvely, ad the results show that Q wth gbest ad mbest mutato both performs better tha. The work of R. A. Krohlg et.al.[6][7] showed that how Gaussa ad Cauchy probablty dstrbuto ca mprove the performace of the stadard. Recetly, evolutoary programmg wth expoetal mutato has also bee proposed [8]. I order to prevet from fallg a local optmum, a fast (F) s proposed by troducg a Cauchy mutato operator ths paper. Beses the Cauchy mutato, F chooses the atural selecto strategy of evolutoary algorthms as the basc elmato strategy of partcles. Although F greatly overcomes the tedecy of trappg to local optma of, the covergece rate s t satsfed. Lke dstrbuted geetc algorthm, multple swarms ea s a very useful for speedg up the search. I ths paper, a multple swarms algorthm() based o F s proposed by troducg a crossover operato, the ew formato exchagg ad sharg mechasm of make t coverge fast o the global optmum. The rest of ths paper s orgazed as follows. Secto gves the aalyss of, ad the detal of F. Secto 3 gves a bref revew of the mult-populato techque ad the descrbe explctly. Secto 4 descrbes the expermet setup ad presets the expermet results. Fally, Secto 5 cocludes the paper wth a bref summary.. Fast Mult-swarm Optmzato wth Cauchy Mutato ad Crossover operato

3 . Cauchy mutato From the mathematc theoretcal aalyss of the trajectory of a partcle [9], the trajectory of a partcle X coverges to a weghted mea of P ad P gd. Wheever the partcle coverges, t wll fly to the persoal best posto ad the global best partcle s posto. Ths formato sharg mechasm makes have a very fast speed of covergece. Meawhle, because of ths mechasm, ca t guaratee to fd the global mmal value of a fucto. I fact, the partcles usually coverge to local optma. Wthout loss of geeralty, oly fucto mmzato s dscussed here. Oce the partcles trap to a local optmum, whch P ca be assumed to be the same as P gd, all the partcles coverge o P gd. At ths codto, the velocty update equato becomes: V = ω V (3) Whe the terato the equato (3) goes to fte, the velocty of the partcle V wll be close to because of ω<. After that, the posto of the partcle X wll ot chage, so that has o capablty of jumpg out of the local optmum. It s the reaso that ofte fals o fdg the global mmal value. To overcome the weakess of dscussed at the begg of ths secto, the Cauchy mutato s corporated to algorthm. The basc ea s that, the velocty ad posto of a partcle are updated ot oly accordg to () ad (), but also accordg to Cauchy mutato as follows: V = exp( δ ) (4) X V X + = V δ (5) where δ ad δ deote Cauchy radom umbers Sce the expectato of Cauchy dstrbuto does t exst, the varace of Cauchy dstrbuto s fte so that Cauchy mutato could make a partcle have a log jump. By addg the update equatos of (4) ad (5), F greatly creases the probablty of escapg from the local optmum.. Natural selecto strategy I the stadard, all partcles are drectly updated by ther offsprg o matter whether they are mproved. If a partcle moves to a better posto, t ca be replaced by the updated. However f t moves to a worse posto, t s stll replaced by ts offsprg. I fact, the most partcles fly to worse postos for most cases, therefore the whole swarm wll coverge o local optma. Lke evolutoary algorthms, F troduces a evolutoary selecto strategy whch each partcle survves accordg to a atural

4 selecto rule. Therefore, the partcle s posto at the ext step s ot oly due to the posto update but also the evolutoary selecto. Such strategy could greatly reduce the probablty of trappg to local optmum. The evolutoary selecto strategy s carred out as follows. Assume the sze of the swarm s m, par-wse comparso over the uo of parets ad offsprg (,, m) s made. For each partcle, q oppoets are radomly chose from all parets ad offsprg wth equal probablty. If the ftess of partcle s less the ts oppoet, t wll receve a w. The select m partcles that have the more wgs to be the ext geerato. The major steps of F are as follows: Step: Geerate the tal partcles by radomly geeratg the posto ad velocty for each partcle. Step: Evaluate each partcle s ftess. Step3: For each partcle, f ts ftess s smaller tha ts prevous best(p ), update P. Step4: For each partcle, f ts ftess s smaller tha the best oe (P gd ) of all partcles, update P gd. Step5: For each partcle,do ).Geerate a ew partcle t accordg to the formula () ad (). ).Geerate a ew partcle t accordg to the formula (4) ad (5). 3) Compare t wth t,chose the oe wth smaller ftess to be the offsprg. Step6: Geerate ext geerato accordg to the above evolutoary selecto strategy. Step7: f the stop crtero s satsfed, the stop, else goto Step Multple swarms optmzato techque I order to escape from the local optma ad avo premature covergece, the search for global optmum should be dverse. May researchers have mproved the performace of the by ehacg ts ablty wth a more dverse search. Specfcally, some have troduced usg multple swarms, ad the exchage formato amog them. The fast covergg behavor of the makes ths ssue so crtcal for multmodal problems. Al-Kazem ad Moha [] dved the populato to two sets to acheve a more dverse, oe set movg to the gbest whle aother movg opposte drecto. After some geeratos, f the gbest would ot mprove, the partcles would swtch ther group. Two cooperatg swarms was used by Baskar ad Sugatha [] to search cocurretly for a soluto alog wth sharg the gbest formato of two swarms. The two swarms track the gbest f t mproves. Each swarm usg dfferet update equato: Oe uses the stadard whle the other uses the Ftess-to-Dstace rato []. Ther approach mproved the performace solvg sgle objectve optmzato problems. The a mproved algorthm was proposed by El-Abd ad Kamel [3] through addg a twoway flow of formato betwee two swarms. After rug a fxed geeratos, f the best p partcles mprove, the they wlll replace the worst p partcles the other swarm. Ths guaratees exchagg ew formato from the other swarm s experece for the two

5 swarms. I ths study, A ew learg mechasm s troduced amog swarms. At each terato, the partcles ot oly update themselves accordg to the best partcle of ther ow swarm, but also lear formato from the best partcle of other swarms. The formato sharg ad learg mechasm make swarm exted ther search space ad speedup the covergece speed. The formato sharg ad learg mechasm that we call t crossover operato s descrbed as follows: Step: for each partcle of swarm k, radomly select a best partcle p from a radom swarm. Strp: for each dmeso of partcle p s posto px[] ad velocty pv[], f rad()<q c, crossover partcle p wth p as follows: px[]=(-α)*px[]+ α*p x[]. pv[]=rad()*(p x[]-px[]). Step3: f all partcles of swarm k are updated, ed the operator, else go to Step. where q c s crossover rate, α s a radom umber of (,). 4 Expermets ad Results Eght bechmark fuctos (f -f 8 ) are used ths paper. Table gves the detals of these fuctos. Algorthm parameters are as follows for all expermets: accelerato costats of η ad η are both set to be.4968, ad erta weght ω= as suggested by de Bergh [4], crossover rate q c s.8, rug tme s 5. I order to be the same umber of fucto evaluatos to, a partcle wll be evaluated two tmes each geerato F ad. The other parameters are gve the followg expermets. Two groups of expermets are carred out ths secto. Frstly, s compared wth stadard ad F to show the performace of algorthm o problems. I the expermets, the umber of populato s 6, the touramet sze (q=) s chose for F. 3 swarms are used, swarm sze s, the touramet sze (q=5) s chose ad crossover rate q c s.6 for. The other parameters are the same as the above. Fg shows the comparso results of the process wth the same evaluatos ad table 3 show the statstcal results for all test problems over 5 rus. By vewg the results of Fg ad table 3, we ca easly see that F show better performace tha o fucto f, f 3, f 6 ad f 7. All the results of are better tha. The results of Table 3 demostrate that fds the global optma for fucto f, f 3, f 5, f 6, f 7 ad f 8, especally for fucto f, f 3 ad,f 7, the global optma of them are fod for each ru over 5 rus. From the comparso results, we ca kow that Cauchy mutato s helpful for some problems, ad the multple swarms techque works for all test problems. It dcates that ot oly speeds up the search, but also mproves the optmzg precso.

6 Table.Detals of test fuctos, where s the dmeso of the fucto, fm s the mmum value of the fucto, S R f Test fucto S f m = ( x = x 3 (-5.,5.) ) f x) = = x s( x ) ( 3 (-5,5) f 3 ( x ) = 6 = x 3 (-5., 5.) (,) 4 f4( x) = = x + U 3 (-.8,.8) (s x + y ).5 f 5 ( x ) = (. +. ( x + y )) +.5 (-.,.) f 6 ( x ) = ( x ) = 4 x cos( ) + = 3 (-3.,3.) f ( x) = exp(. f 7 exp( = x = cos(π x )) + + e x) = = (( x + x ) + ( x ) 8 ) ) 3 (-3.,3.) ( 3 (-.48,.48) Secodly, the am of group s to aalyze the effect of dfferet swarms ad swarm sze( m*) for a same populato sze 8. Fucto f,f 4,f 6,f 8 are chose to test. the maxmum geerato s, m ad are respectvely the umber of swarms ad swarm sze. 4 sets expermets are coducted, Table shows the value of m ad. All the other parameters are the same as above. By vewg the comparso results of Table 4, the more umber swarms s better for fucto f 4 ad f 6,however, t s t the case for fucto f ad f 8. That s to say, although multple swarms ca speed up the covergece rate, ot the more the better. t s hard to fd a optmal swarm sze ad umber of swarms for geeral problems. Actually, the optmal swarm sze ad swarm umbers deped o the dstrbutos of optmal solutos ad umber of optmal solutos. For fuctos wth a few optmum, small umber of swarms mght be eough. However, for fuctos wth a lot of optmum, large swarms mght be eeded. Table The value of swarms(m) ad swarm sze(),q s the touramet sze Set Set Set 3 Set 4 m (q) 4(6) (5) (4) 5(3)

7 best ftess(log) FPFO best ftess F f f best ftess(log) F best ftess(log) F f 3 f 4 best ftess(log) F best ftess(log) F f 5 f 6 best ftess(log) F best ftess(log) F f 7 f 8 Fg. Comparso of the process of the mea ftess of the best partcle betwee, F ad for 5 rus, the vertcal axs s the fucto value ad the horzotal axs s the umber of evaluatos.

8 Table 3 The maxmum(max),mmum(m) ad average(avg) best ftess over 5 rus. Std s the stadard devato F evaluatos F f 6* 4 Max 3.438E e e-39 M.4857E e-4.88e-4 Avg 5.663E e- 3.43e-4 std e-.4838e e-4 f 6* 4 Max M Avg std f 3 6*5 Max M Avg std f 4 6* 4 Max M Avg std f 5 6* 4 Max M Avg std f 6 6* 4 Max M e- Avg std f 7 6* 4 Max M 3.93e e-7 Avg e-5 std f 8 6* 4 Max e-.3399e e-8 M e-5.469e-9 Avg.784e-.3645e e-9 std.34386e e e-9

9 5 Coclusos By aalyzg the advatage ad dsadvatage of the stadard, F based o Cauchy mutato ad evolutoary selecto strategy s proposed ths paper. Although F greatly overcomes the tedecy of trappg to local optma of, the covergece rate s t satsfed, so a multple swarms algorthm() based o F s proposed by troducg a crossover operato, s tested o 8 bechmark fuctos. From the expermetal results of these fuctos, t ca be see that the performed much better tha ad F o the selected problems. Oly fuctos wth the dmeso less tha 3 were tested ths paper. Further research wll focus o testg the performace of o hgher dmesoal problems order to fd whether would scale up well for the large fucto optmzato problems. Table 4 Comparso wth dfferet swarms ad swarm sze for over 5 rus, The maxmum(max),mmum(m) ad average(avg) best ftess over 5 rus. Std s the stadard devato F swarms*4 4 warms* 8 swarms* 6swarm*5 f Max.97774e e e e-34 M e e e e-36 Avg.36448e e-4.959e-4.49e-34 std 3.896e-4 8.5e e e-34 f 4 Max M Avg std f 6 Max M Avg std f 8 Max.5777e e e e- M.9573e-3 Avg e e e e- std.7555e e-9.76e e- Refereces [] J. Keedy ad R. C. Eberhart, Partcle Swarm Optmzato, IEEE Iteratoal Coferece o Neural Networks, pp , 995.

10 [] R. C. Eberhart ad J. Keedy, A New Optmzer Usg Partcle Swarm Theory, Proceedgs of the 6th Iteratoal Symposum o Mcro Mache ad Huma Scece, pp.39-43, 995. [3] A. Rataweera, S. K. Halgamuge, ad H. C. Watso, Self-orgazg herarchcal partcle swarm optmzer wth tme-varyg accelerato coeffcets, IEEE Trasactos o Evolutoary Computato, vol. 8,o. 3, pp. 4 55, 4. [4] J. Su, B. Feg, W. Xu, Partcle swarm optmzato wth partcles havg quatum behavor, Proceedgs of the IEEE Cogress o Evolutoary Computato, Portlad, Orego USA, pp , 4. [5] Jg Lu, Webo Xu, Ju Su, Quatum-behaved partcle swarm optmzato wth mutato operator. Proceedgs of the 7th IEEE Iteratoal Coferece o Tools wth Artfcal Itellgece Pages: 37-4, 5 [6] R. A. Krohlg, Gaussa partcle swarm wth jumps, Proceedgs of the IEEE Cogress o Evolutoary Computato, Edburgh, UK,pp. 6-3, 5. [7] R. A. Krohlg, L. dos Satos Coelho, -E: Partcle Swarm wth Expoetal Dstrbuto, Proceedgs of the IEEE Cogress o Evolutoary Computato, pp48-433, July 6. [8] H. Narhsa, T. Taguch, M. Ohta, ad K. Katayama, Evolutoary Programmg wth Expoetal Mutato, Proceedgs of the IASTED Artfcal Itellgece ad soft Computg, Beor, Spa, pp. 55-5, 5. [9] M. Clerc ad J. Keedy, The Partcle Swarm: Exploso, Stablty ad Covergece a Mult-Dmesoal Complex Space, IEEE Tras. o Evolutoary Computato, Vol.6,pp:58-73,. [] B. Al-Kazem ad C. K. Moha, Mult-phase dscrete partcle swarm optmzato I Proc. of 4th It. Workshop o Froters o Evolut.Alg., Research Tragle Park, NC,. [] S. Baskar, ad P. N. Sugatha, A ovel cocurret partcle swarm optmzato, I Proc. of Cog. o Evolut. Comput., Portlad, OR, pp , 4. [] T. Peram, K. Veeramachae, ad C. K. Moha, Ftess-dstacerato based partcle swarm optmzato, I Proc. of IEEE Swarm Itell. Symp., Idaapols, IN, pp , 3. [3] M. El-Abd, ad M. Kamel, Iformato exchage multple cooperatg swarms, I Proc. of Cog. o Evolut. Comput., Edburgh, UK, pp. 38-4, 5. [4] F. va de Bergh. A Aalyss of Partcle Swarm Optmzers. PhD thess, Departmet of Computer Scece, Uversty of Pretora, South Afrca,.