A novel hybrid algorithm with marriage of particle swarm optimization and extremal optimization

Transcription

1 A novel hybrd algorthm wth marrage of partcle swarm optmzaton and extremal optmzaton Mn-Rong Chen 1, Yong-Za Lu 1, Q Luo 2 1 Department of Automaton, Shangha Jaotong Unversty, Shangha , P.R.Chna 2 Department of Informaton and Communcatons Technologes, Nanjng Unversty of Informaton Scence and Technology, Nanjng , P.R. Chna emal: {aumnrongchen,yzlu}@sjtu.edu.cn,hb2-102@nust.edu.cn Abstract Partcle swarm optmzaton (PSO) has receved ncreasng nterest from the optmzaton communty due to ts smplcty n mplementaton and ts nexpensve computatonal overhead. However, PSO has premature convergence, especally n complex multmodal functons. Extremal Optmzaton (EO) s a recently developed local-search heurstc method and has been successfully appled to some NP-hard combnatoral optmzaton problems. To overcome the lmtaton of PSO, ths paper proposes a novel hybrd algorthm, called hybrd PSO-EO algorthm, through ntroducng EO to PSO for the frst tme. The hybrd approach elegantly combnes the exploraton ablty of PSO wth the explotaton ablty of EO. We also present a new hybrd mutaton operator n EO that enhances the exploratory capabltes of our algorthm. The proposed approach s valdated usng sx complex unmodal/multmodal benchmark functons. The smulaton results demonstrate that the proposed approach s capable of avodng premature convergence and s superor to three search algorthms,.e., standard PSO, Populaton-based EO (PEO) and standard Genetc Algorthm (GA). Thus, the hybrd PSO-EO algorthm can be consdered a good alternatve to solve complex numercal functon optmzaton problems. Key Words: Partcle swarm optmzaton; Extremal optmzaton; Numercal functon optmzaton 1 Introducton Partcle Swarm Optmzaton (PSO) algorthm s a recent addton to the lst of global search methods. Ths dervatve-free method s partcularly suted to contnuous varable problems and has receved ncreasng attenton n the optmzaton communty. PSO s orgnally developed by Kennedy and Eberhart [1] and nspred by the paradgm of brds flockng. PSO conssts of a swarm of partcles and each partcle fles through the mult-dmensonal search space wth a velocty, whch s constantly updated by the partcle s prevous performance and by the prevous performance of the partcle s neghbors. PSO can be easly mplemented and s computatonally nexpensve n terms of both memory requrements and CPU speed [2]. However, even though PSO s a good and fast search algorthm, t has premature convergence, especally n complex mult-peak-search problems. So far there have been many researchers devoted to ths feld to deal wth ths problem [3 16]. Recently, a general-purpose local-search heurstc algorthm named Extremal Optmzaton (EO) was proposed by Boettcher and Percus [17]. EO s based on the Bak-Sneppen (BS) model [18], 1

2 whch shows the emergence of self-organzed crtcalty (SOC) [19] n ecosystems. The evoluton n ths model s drven by a process where the weakest speces n the populaton, together wth ts nearest neghbors, s always forced to mutate. The dynamcs of ths extremal process exhbts the characterstcs of SOC, such as punctuated equlbrum [18]. EO opens the door to applyng nonequlbrum process, whle the smulated annealng (SA) apples equlbrum statstcal mechancs. In contrast to genetc algorthm (GA) whch operates on an entre gene-pool of huge number of possble solutons, EO successvely elmnates those worst components n the sub-optmal solutons. Its large fluctuatons provde sgnfcant hll-clmbng ablty, whch enables EO to perform well partcularly at the phase transtons [17]. EO has been successfully appled to some NP-hard combnatoral optmzaton problems such as graph b-parttonng [17], graph colorng [20], producton schedulng [21, 22] and dynamc combnatoral problems [23]. To enhance and mprove the search performance and effcency of EO, n our prevous work [24], we presented a novel EO strategy wth populaton based search. The newly developed EO algorthm s named populaton-based EO (PEO). PEO has been successfully appled to handlng numercal optmzaton problems [24]. To avod premature convergence of PSO, an dea of combnng PSO wth EO s addressed n ths paper; the ratonale behnd ths s that such a hybrd approach expects to enjoy the merts of PSO wth those of EO. In other words, PSO contrbutes to the hybrd approach n a way to ensure that the search converges faster, whle the EO makes the search to jump out of local optma due to ts strong local-search ablty. In ths work, we develop a novel hybrd optmzaton method, called hybrd PSO- EO algorthm, to solve those complex multmodal functons whch may be dffcult for the standard partcle swarm optmzers. To the of our knowledge, there has been no hybrd technque based on PSO and EO presented so far. Furthermore, we propose a new hybrd mutaton operator n EO that enhances the exploratory capabltes of our algorthm. Our approach has been valdated usng sx unconstraned benchmark functons and compared wth standard PSO, PEO and standard GA. The smulaton results demonstrate that the proposed approach s able to escape from local optma and s superor to the other three algorthms. Hence, the hybrd PSO-EO may be a good alternatve to deal wth complex numercal functon optmzaton problems. Ths paper s organzed as follows. In Secton 2, the PSO and EO algorthms are ntroduced n bref. In Secton 3, we propose the hybrd PSO-EO algorthm and descrbe t n detal. In Secton 4, the proposed approach s appled to handlng sx unconstraned benchmark functons. In addton, the smulaton results are compared wth those of standard PSO, PEO and standard GA. Fnally, Secton 5 concludes the paper wth an outlook on future work. 2 Partcle Swarm Optmzaton and Extremal Optmzaton 2.1 Partcle Swarm Optmzaton (PSO) PSO s a populaton based optmzaton tool, where the system s ntalzed wth a populaton of random partcles and the algorthm searches for optma by updatng generatons. Suppose that the search space s D-dmensonal. The poston of the -th partcle can be represented by a D-dmensonal vector X = (x 1, x 2,, x D ) and the velocty of ths partcle s V = (v 1, v 2,, v D ). The prevously vsted poston of the -th partcle s represented by P = (p 1, p 2,, p D ) and the global poston of the swarm found so far s denoted by P g = (p g1, p g2,, p gd ). The ftness of each partcle can be evaluated through puttng ts poston nto a desgnated objectve functon. The partcle s velocty and ts new poston are updated as follows: v t+1 d = w t v t d + c 1 r t 1(p t d x t d) + c 1 r t 2(p t gd x t d), (1) x t+1 d = x t d + v t+1 d (2) 2

3 where d {1, 2,, D}, {1, 2,, N}, N s the populaton sze, the superscrpt t denotes the teraton number, w s the nerta weght, r 1 and r 2 are two random values n the range [0,1], c 1 and c 2 are the cogntve and socal scalng parameters whch are postve constants. 2.2 Extremal Optmzaton (EO) Inspred by the Bak-Sneppen model, Boettcher and Percus [17] proposed the EO algorthm for a mnmzaton problem as Fgure Randomly generate a soluton X ( x1, x2,, x D ). Set optmal soluton X X and the mnmum cost functon C( X ) C( X ). 2. For the current soluton X, (a) evaluate the ftness for each varable (b) rank all the ftnesses and fnd the varable (c) choose one soluton state, (d) accept X X ' uncondtonally, (e) f C( X ) C( X ) then set X 3. Repeat Step 2 as long as desred. 4. Return X and C( X ). x, {1,2, D}, x j wth the lowest ftness,.e., j for all, X ' n the neghborhood of X, such that the j-th varable must change ts X and C( X ) C( X. ) Fgure 1: Pseudo-code of EO algorthm Note that n the EO algorthm, each decson varable n the current soluton X s consdered speces. In ths study, we adopt the term component to represent speces whch s usually used n bology. From the above algorthm, t can be seen that unlke genetc algorthms whch work wth a populaton of canddate solutons, EO evolves a sngle soluton X and makes local modfcaton to the worst components of X. Ths requres that a sutable representaton be selected whch permts the soluton components to be assgned a qualty measure (.e. ftness). Ths dffers from holstc approaches such as evolutonary algorthms that assgn equal-ftness to all components of a soluton based on ther collectve evaluaton aganst an objectve functon. Through always performng mutaton on the worst component and ts neghbors successvely, the soluton can mprove ts components and evolve tself toward the optmal soluton generaton by generaton. As a result, EO has a strong local-search capablty and does not easly get trapped n local optma. 3 Hybrd PSO-EO Algorthm Note that PSO has a strong global-search ablty, whle EO has a strong local-search ablty. In ths work, we propose a novel hybrd PSO-EO algorthm whch combnes the merts of both PSO and EO. Ths hybrd approach makes full use of the exploraton ablty of PSO and the explotaton ablty of EO and offsets the weaknesses of each other. Consequently, through ntroducng EO to PSO, the proposed approach s capable of escapng from a local optmum. However, f EO s ntroduced to PSO each teraton, the computatonal cost wll ncrease sharply and at the same tme the fast convergence ablty of PSO may be weakened. In order to perfectly ntegrate PSO wth EO, EO s ntroduced to PSO every IN V teratons. Therefore, the hybrd PSO-EO approach s able to keep fast convergence n most of the tme under the help of PSO, and to escape from a local optmum wth the ad of EO. The value of parameter INV s predefned by the user. Usually, accordng to our expermental experence, the value of INV can be set to when the test functon s unmodal 3

4 or multmodal wth relatvely less local optma. As a consequence, PSO wll play a key role n ths case. When the test functon s multmodal wth many local optma, the value of INV can be set to 1 50 and thus EO wll help PSO to jump out of a local optmum. 3.1 Hybrd PSO-EO Algorthm For a mnmzaton problem wth D dmensons, the hybrd PSO-EO algorthm works as Fgure 2 and s llustrated n Fgure 3. It s mportant to note that, n order to fnd out the worst component (or so-called decson varable), each component of a soluton should be assgned a ftness value n the EO procedure. In ths work, we defne the ftness of each soluton component for an unconstraned mnmzaton problem as follows. For the -th partcle, the ftness λ k of the k-th component s defned as the mutaton cost,.e. OBJ(X k ) OBJ(P g ), where X k s the new poston of the -th partcle obtaned by performng mutaton only on the k-th component and leavng all other components fxed, OBJ(X k ) s the objectve value of X k, and OBJ(P g ) s the objectve value of the poston n the swarm found so far. The EO procedure s descrbed n Fgure Intalze a swarm of N partcles wth random postons and veloctes on D dmensons. Set teraton = Evaluate the objectve functon value for each partcle, and update P ( 1,, N ) and. 3. Update the velocty and poston of each partcle usng Eq.1 and Eq.2, respectvely. 4. Evaluate the objectve functon value for each partcle, and update P ( 1,, N ) and P g. 5. If ( teraton mod INV ) 0, the EO procedure s ntroduced. Otherwse, contnue the next step. 6. If the termnal condton s satsfed, go to the next step; otherwse, set teraton = teraton 1, and go to Step 3. 7.Output the optmal soluton and the optmal objectve functon value. P g Fgure 2: Pseudo-code of PSO-EO algorthm START ntalzaton of PSO, teraton=0 PSO operators teraton=teraton+1 teraton mod INV = 0? Yes EO procedure No No termnal condton met? Yes output optmal soluton STOP Fgure 3: Flowchart of PSO-EO algorthm 4

5 1. Set the ndex of the current partcle For the poston X ( x 1, x 2,, xd ) of the -th partcle, (a) perform mutaton on each component of X one by one, whle keepng other components fxed. Then D new postons X k ( k 1,, D ) can be obtaned; (b) evaluate the ftness OBJ ( X ) OBJ ( Pg ) for each component x k, k {1,, D} ; k k (c) compare all the components accordng to ther ftness values and fnd out the worst adapted component x w, and then X w s the new poston correspondng to x w, w {1,, D} ; (d) f OBJ ( X w) OBJ ( X ), then set X X w and OBJ ( X ) OBJ ( X w), and contnue the next step. Otherwse, X keeps unchanged and go to Step 3; (e) update P and. P g 3. If equals to the populaton sze N, return the results; otherwse, set 1 and go to Step Mutaton Operator Fgure 4: Pseudo-code of EO procedure Snce there s merely mutaton operator n EO, the mutaton plays a key role n EO search. In ths work, we present a new mutaton method based on mxng Gaussan mutaton and Cauchy mutaton and choose t as the mutaton operator n EO. The mechansms of Gaussan and Cauchy mutaton operatons have been studed by Yao et al. [25]. They ponted out that Cauchy mutaton s better at coarse-graned search whle Gaussan mutaton s better at fne-graned search. Inspred by the dea proposed by Yao et al. [25], we present a new mutaton method based on mxng Gaussan mutaton and Cauchy mutaton. For the sake of convenence, we call t hybrd GC mutaton. It must be ndcated that, unlke the method n [25], ths mutaton doesn t compare the antcpated outcomes between Gaussan mutaton and Cauchy mutaton due to the characterstcs of EO. In the hybrd GC mutaton, the Cauchy mutaton s frst adopted. It means that the large step sze wll be taken frst at each mutaton. If the new generated varable after mutaton goes beyond the range of varable, the Cauchy mutaton wll be used repeatedly for some tmes (T C), untl the new generated offsprng falls nto the range. Otherwse, Gaussan mutaton wll be carred out repeatedly for another some tmes (T G), untl the offsprng satsfes the requrement. That s, the step sze wll become smaller than before. If the new generated varable after mutaton stll goes beyond the range of varable, then we choose the upper or lower bound of the decson varable as the new generated varable. Thus, our approach combnes the advantages of coarse-graned search and fne-graned search. The above analyses show that the hybrd GC mutaton s very smple yet effectve. Unlke some swtchng algorthms whch have to decde when to swtch between dfferent mutatons durng search, the hybrd GC mutaton does not need to make such decsons. In ths work, the Gaussan mutaton performs wth the followng representaton: x k = x k + N k (0, 1) (3) where x k and x k denote the k-th decson varables before mutaton and after mutaton, respectvely, N k (0, 1) denotes the Gaussan random number wth mean zero and standard devaton one and s generated anew for k-th decson varable. The Cauchy mutaton performs as follows: x k = x k + δ k (4) where δ k denotes the Cauchy random varable wth the scale parameter equal to one and s generated anew for the k-th decson varable. In the hybrd GC mutaton, the values of parameters T C and T G are set by the user beforehand. The value of T C decdes the coarse-graned searchng tme, whle the value of T G has an effect on 5

6 Table 1: Test functons Functon Functon expresson Search space Optmal objectve value f 1 : Mchalewcz n =1 sn (x ) sn 2m ( x2 ), m = 10 (0, π π)n f 2 : Schwefel n =1 (x sn( x )) ( 500, 500) n f 3 : Grewank n 4000 =1 x2 n =1 cos( x ) + 1 ( 600, 600) n 0 f 4 : Rastrgn n =1 [x2 10 cos (2πx ) + 10] ( 5.12, 5.12) n 0 f 5 : Ackley 20 + e 20e [ P n n =1 x2 ] e 1 P n n =1 cos(2πx ) ( , ) n 0 f 6 : Rosenbrock n 1 =1 [100(x +1 x 2 ) 2 + (x 1) 2 ] ( 30, 30) n 0 the fne-graned searchng tme. Therefore, both values of the two parameters can not be large because t wll prolong the search process and hence ncrease the computatonal overhead. Accordng to our expermental experence, the moderate values of T C and T G can be set to Experments and Results 4.1 Test Functons In order to compare the performance of the proposed hybrd PSO-EO wth standard PSO, PEO and standard GA, we use sx classcal benchmark functons shown n Table 1. All the functons are to be mnmzed. For these functons, there are many local optma and/or saddles n ther soluton spaces. The amount of local optma and saddles ncreases wth ncreasng complexty of the functons,.e. wth ncreasng dmenson. The global mnmum of Mchalewcz functon s -9.66, and t occurs when x 1 = 2.2, x 2 = 1.57, x 3 = 1.29, x 4 = 1.92, x 5 = 1.72, x 6 = 1.57, x 7 = 1.45, x 8 = 1.76, x 9 = 1.66, x 10 = Ths functon s a multmodal test functon (n! local optma). The parameter m defnes the steepness of the valleys or edges. Larger m leads to more dffcult search. For very large m the functon behaves lke a needle n the haystack (the functon values for ponts n the space outsde the narrow peaks gve very lttle nformaton on the locaton of the global optmum). The global mnmum of Schwefel functon s , and t occurs when x = for all = 1,, n. The surface of ths functon s composed of a great number of peaks and valleys. The functon has a second mnmum far from the global mnmum where many search algorthms are trapped. Moreover, the global mnmum s near the bounds of the doman. The search algorthms are potentally prone to convergence n the wrong drecton n the optmzaton of ths functon. The functons Grewank, Rastrgn and Ackley are all hghly multmodal. They have wdespread local mnma whch are regularly dstrbuted. The dffcult part about fndng optmal solutons to these functons s that an optmzaton algorthm easly can be trapped n a local optmum on ts way towards the global optmum. The global mnmum of the three functons s zero, and the optmum soluton s x = 0 for all = 1,, n. The global mnmum of Rosenbrock functon s zero, and t occurs when x = 1 for all = 1,, n. It s a unmodal optmzaton problem. The global optmum s nsde a long, narrow, parabolc shaped flat valley, popularly known as the Rosenbrock s valley. To fnd the valley s trval, but to acheve convergence to the global optmum s a dffcult task. 4.2 Expermental Settngs To testfy the effcency and effectveness of the the proposed approach, the expermental results of our approach are compared wth those of standard PSO, PEO and standard GA. Note that all the algorthms were run on the same hardware (.e., Intel Pentum M wth 900 MHz CPU and 256M memory) 6

7 and software (.e., JAVA) platform. Each algorthm was run ndependently for 20 trals. Table 2 shows the settngs of problem dmenson n, maxmum generaton, populaton sze and ntalzaton range for each test functon. Table 2: Parameter settngs for sx test functons Functon Dmenson Maxmum generaton Populaton sze Intalzaton range INV f (0, π) n 20 f ( 500, 500) n 1 f ( 600, 600) n 100 f ( 5.12, 5.12) n 100 f ( , ) n 100 f ( 30, 30) n 1 For the hybrd PSO-EO, the cogntve and socal scalng parameters,.e. c 1 and c 2, were both set to 2, the nerta weght w vared from 0.9 to 0.4 lnearly wth the teratons, the upper and lower bounds for velocty on each dmenson,.e., (v mn, v max ) were set to be the upper and lower bounds of each dmenson,.e. (v mn, v max ) = (x mn, x max ). The parameters T C and T G n the hybrd GC mutaton were both set to 3. Table 2 also shows the value of parameter INV for each test functon. From Table 2, we can see that EO was ntroduced to PSO more frequently on functons f 2, f 4 and f 6 than on other three functons due to the complexty of problems. For the standard PSO, all the parameters,.e., c 1, c 2, w and (v mn, v max ), were set to the same as those used n the hybrd PSO-EO. For the PEO, we also adopted the hybrd GC mutaton wth both parameters T C and T G equal to 3. For the standard GA, we used eltsm mechansm, roulette wheel selecton mechansm, sngle pont unform crossover wth the rate of 0.3, non-unform mutaton wth the rate of 0.05 and the system parameter of 0.1. In order to compare the dfferent algorthms, a far tme measure must be selected. The number of teratons cannot be used as a tme measure, as the algorthms do dfferng amounts of work n ther nner loops. It s also notce that each component of a soluton n the EO procedure has to be evaluated each teraton, and thus the calculaton of the number of functon evaluatons wll brng some trouble. Therefore, the number of functon evaluatons s not very sutable as a tme measure. In ths work, we adopt the average runtme of twenty runs when the near-optmal soluton s found or otherwse the maxmum generaton s reached as a tme measure. 4.3 Smulaton Results and Dscussons After 20 trals of runnng each algorthm for each test functon, the smulaton results were obtaned and shown n Tables 2 7. Denote F as the result found by the algorthms and F as the optmum value of the functons. The smulaton s consdered successful, or n other words, the near-optmal soluton s found, f F satsfes that (F F )/F < 1E 3 (for the case F 0) or F F < 1E 3 (for the case F = 0). In these tables, Success represents the successful rate, and Runtme s the average runtme of twenty runs when the near-optmal soluton s found or otherwse the maxmum generaton s reached. The worst, mean, solutons and the standard devatons found by the four algorthms are also lsted n these tables. Note that the standard devatons ndcate the stablty of the algorthms, and the successful rate represents the robustness of the algorthms. It can be observed from Table 3 and Table 4 that PSO-EO sgnfcantly outperformed PSO, PEO and GA n the optmzaton of functons Mchalewcz and Schwefel, whch are hghly multmodal. PSO-EO shows ts great search ablty and the promnent capablty of escapng from local optma n Schwefel functon, whch s well known for ts deceptve property n ts locaton of the global mnmum. From Table 5, we can see that PSO-EO, PSO and PEO could fnd the global optmum wth 7

8 Table 3: Comparson results for Mchalewcz functon f 1 (F1 = 9.66) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO E-3 PSO PEO GA Table 4: Comparson results for Schwefel functon f 2 (F2 = ) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO PSO PEO GA Table 5: Comparson results for Grewank functon f 3 (F3 = 0) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO PSO PEO E E E E-05 GA Table 6: Comparson results for Rastrgn functon f 4 (F4 = 0) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO PSO PEO GA E-3 Table 7: Comparson results for Ackley functon f 5 (F5 = 0) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO PSO PEO E-3 GA Table 8: Comparson results for Rosenbrock functon f 6 (F6 = 0) Algorthm Runtme(s) Success(%) Worst Mean Best St.Dev. PSO-EO E E E E-5 PSO PEO GA

9 100% successful rate on Grewank functon. However, PSO-EO converged to the optmum soluton more quckly than PSO and PEO, and found more accurate soluton than PEO. As can be observed from Table 6, only PSO-EO and PSO were capable of fndng the global optmum on Rastrgn functon. Wth respect to convergence speed, PSO-EO performed a lttle better than PSO. On Ackley functon, as can be seen from Table 7, only PSO-EO and PSO were able to fnd the global optmum, but PSO-EO converged faster than PSO. It can be found from Table 8 that PSO-EO performed sgnfcantly better than other three algorthms on Rosenbrock functon, whch s unmodal. On the convergence speed, PSO-EO was the fastest algorthm n comparson wth other three algorthms for all the functons. It s mportant to notce that PSO-EO was about 40 tmes faster than other three algorthms on Mchalewcz functon. Thus, the hybrd PSO-EO can be consdered a very fast algorthm when solvng numercal optmzaton problems, especally those complex multmodal functons. On the stablty of algorthms, PSO-EO showed the stablty as compared to PSO, PEO and GA. Overall, PSO-EO s a hghly stable algorthm that s capable of obtanng reasonable consstent results. On the robustness of algorthms, PSO-EO sgnfcantly outperformed the other three algorthms. PSO-EO was capable of fndng the global optmum wth 100% successful rate on all the test functons, especally on functons Schwefel and Rosenbrock, whch are very dffcult to solve for many optmzaton algorthms. Hence, PSO-EO can be regarded as a hghly robust optmzaton algorthm. From the aforementoned comparsons, t s obvous that our approach possesses superor performance n accuracy, convergence speed, stablty and robustness, as compared to standard PSO, PEO and standard GA. As a result, our approach s a perfectly good performer n optmzaton of those complex hgh dmensonal functons. 5 Concluson and Future Work In ths paper, we develop a novel hybrd optmzaton method, called hybrd PSO-EO algorthm, through ntroducng EO to PSO for the frst tme. The hybrd approach elegantly combnes the exploraton ablty of PSO wth the explotaton ablty of EO, and s capable of preventng premature convergence. Compared wth standard PSO, PEO and standard GA wth sx well-known benchmark functons, t has been shown that our approach posses superor performance n accuracy, convergence rate, stablty and robustness. The smulaton results demonstrate that PSO-EO s well suted for those multmodal functons wth hgh dmenson. As a consequence, PSO-EO may be a promsng and vable tool to deal wth complex numercal optmzaton problems. The future work ncludes the studes on how to extend PSO-EO to handle multobjectve optmzaton problems. It s desrable to further apply PSO-EO to solvng those more complex real-world optmzaton problems. References [1] J. Kennedy, R.C. Eberhart, Partcle swarm optmzaton. Proceedngs of the 1995 IEEE Internatonal Conference on Neural Networks, vol. 4, pp IEEE Servce Center, Pscat away (1995) [2] R.C. Eberhart, J. Kennedy, A New Optmzer Usng Partcle Swarm Theory. Proceedngs of the Sxth Internatonal Symposum on Mcromachne and Human Scence, Nagoya, Japan. pp , [3] P. J. Angelne, Usng selecton to mprove partcle swarm optmzaton. Proceedngs of the IEEE Congress on Evolutonary Computaton, Anchorage, Alaska, USA. pp , [4] M. Løvbjerg, T. K. Rasmussen, T. Krnk, Hybrd partcle swarm optmser wth breedng and subpopulatons. Proceedngs of the Genetc and Evolutonary Computaton Conference, San Francsco, Morgan Kaufmann Publshers, pp ,

10 [5] K. E. Parsopoulos, V. P. Plaganakos, G. D. Magoulas, M. N. Vrahats, Stretchng technque for obtanng global mnmzers through partcle swarm optmzaton. Proceedngs of the Workshop on Partcle Swarm Optmzaton, Indanapols, IN, pp , [6] K. E. Parsopoulos, M. N. Vrahats, Intalzng the partcle swarm optmzer usng the nonlnear smplex Method. Advances n Intellgent Systems, Fuzzy Systems, Evolutonary Computaton, WSEAS Press (2002). [7] N. Hgash, H. Iba, Partcle swarm optmzaton wth gaussan mutaton. Proceedngs of the IEEE Swarm Intellgence Symposum 2003, Indanapols, Indana, USA. pp , [8] X. H. Sh, Y.H. Lu, C.G. Zhou, H.P. Lee, W.Z. Ln, Y.C. Lang, Hybrd evolutonary algorthms based on PSO and GA. Proceedngs of IEEE Congress on Evolutonary Computaton 2003, Canbella, Australa. pp , [9] S. K. S. Fan, E. Zahara. A hybrd smplex search and partcle swarm optmzaton for unconstraned optmzaton. European Journal of Operatonal Research 181(2007) [10] Y. Jang, T. Hu, C. Huang, X. Wu, An Improved Partcle Swarm Optmzaton Algorthm.Appled Mathematcs and Computaton (2007), do: /j.amc [11] R. Brts, A.P. Engelbrecht, F. van den Bergh. Locatng multple optma usng partcle swarm optmzaton. Appled Mathematcs and Computaton 189 (2007) [12] P.S. Shelokar, Patrck Sarry, V.K. Jayaraman, B.D. Kulkarn. Partcle swarm and ant colony algorthms hybrdzed for mproved contnuous optmzaton. Appled Mathematcs and Computaton 188 (2007) [13] T. Xang, K. Wong, X. Lao. A novel partcle swarm optmzer wth tme-delay. Appled Mathematcs and Computaton 186 (2007) [14] T. Xang, X. Lao, K. Wong. An mproved partcle swarm optmzaton algorthm combned wth pecewse lnear chaotc map. Appl. Math. Comput. (2007), do: /j.amc [15] B. Nu, Y. Zhu, X. He, H. Wu. MCPSO: A mult-swarm cooperatve partcle swarm optmzer. Appled Mathematcs and Computaton 185 (2007) [16] X. Yang, J. Yuan, J. Yuan, H. Mao. A modfed partcle swarm optmzer wth dynamc adaptaton.appled Mathematcs and Computaton 189 (2007) [17] S. Boettcher, A. G. Percus, Nature s way of optmzng. Artfcal Intellgence 2000; 119; [18] P. Bak, K. Sneppen, Punctuated equlbrum and crtcalty n a smple model of evoluton. Physcal Revew Letters 1993; 71 (24); [19] P. Bak, C Tang, K. Wesenfeld, Self-organzed crtcalty. Physcal Revew Letters 1987; 59; [20] S. Boettcher, A.G. Percus, Extremal optmzaton at the phase transton of the 3-colorng problem. Physcal Revew E 69, (2004) [21] Yong-Za Lu, Mn-Rong Chen, Yu-Wang Chen, Studes on extremal optmzaton and ts applcatons n solvng real world optmzaton problems. Proceedngs of 2007 IEEE Seres Symposum on Computaton Intellgence, Hawa, USA, Aprl 1-5, [22] Yu-Wang Chen, Yong-Za Lu, Genke Yang, Hybrd evolutonary algorthm wth marrage of genetc algorthm and extremal optmzaton for producton schedulng. Internatonal Journal of Advanced Manufacturng Technology. Accepted [23] I. Moser, T. Hendtlass, Solvng problems wth hdden dynamcs-comparson of extremal optmzaton and ant colony system. Proceedngs of 2006 IEEE Congress on Evolutonary Computaton (CEC 2006), pp , [24] Mn-Rong Chen, Yong-Za Lu, Genke Yang, Populaton-Based Extremal Optmzaton wth Adaptve Lévy Mutaton for Constraned Optmzaton. Proceedngs of 2006 Internatonal Conference on Computatonal Intellgence and Securty (CIS 06), pp , [25] X. Yao, Y. Lu, G. Ln, Evolutonary programmng made faster. IEEE Transactons on Evolutonary Computaton 1999; 3(2);