A mixed-discrete Particle Swarm Optimization algorithm with explicit diversity-preservation

Transcription

1 Struct Multdsc Optm (213) 47: DOI 1.17/s z RESEARCH PAPER A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton Souma Chowdhury Weyang Tong Achlle Messac Je Zhang Receved: 28 October 211 / Revsed: 4 October 212 / Accepted: 15 October 212 / Publshed onlne: 13 December 212 c Sprnger-Verlag Berln Hedelberg 212 Abstract Engneerng desgn problems often nvolve nonlnear crteron functons, ncludng nequalty and equalty constrants, and a mxture of dscrete and contnuous desgn varables. Optmzaton approaches ental substantal challenges when solvng such an all-nclusve desgn problem. In ths paper, a modfcaton of the Partcle Swarm Optmzaton (PSO) algorthm s presented, whch can adequately address system constrants whle dealng wth mxed-dscrete varables. Contnuous search (partcle moton), as n conventonal PSO, s mplemented as the prmary search strategy; subsequently, the dscrete varables are updated usng a determnstc nearest-feasble-vertex crteron. Ths approach s expected to allevate the undesrable dfference n the rates of evoluton of dscrete and contnuous varables. The premature stagnaton of canddate solutons (partcles) due to loss of dversty s known to be one of the prmary drawbacks of the basc PSO dynamcs. To address ths ssue n hgh dmensonal desgn problems, a new adaptve dversty-preservaton technque s developed. Ths technque characterzes the populaton dversty at each teraton. The estmated dversty measure s then used to apply () a dynamc repulson away from the best global soluton n the case of contnuous varables, and () a stochastc update of the dscrete varables. For performance valdaton, the Mxed-Dscrete PSO algorthm s appled to a wde varety of standard test problems: () a set of 9 unconstraned problems, and () a comprehensve set of 98 Mxed-Integer Nonlnear Programmng (MINLP) problems. We also explore the applcablty of ths algorthm to a large scale engneerng desgn problem -wnd farm layout optmzaton. Keywords Constrant Dscrete varable Mxed-nteger nonlnear programmng (MINLP) Partcle Swarm Optmzaton Populaton dversty Wnd farm layout Parts of ths manuscrpt have been presented at the 53rd AIAA Structures, Structural Dynamcs and Materals Conference, n Aprl, 212, at Honolulu, Hawa - Paper Number: AIAA S. Chowdhury J. Zhang Multdscplnary Desgn and Optmzaton Laboratory, Department of Mechancal, Aerospace and Nuclear Engneerng, Rensselaer Polytechnc Insttute, Troy, NY 1218, USA W. Tong Multdscplnary Desgn and Optmzaton Laboratory, Department of Mechancal and Aerospace Engneerng, Syracuse Unversty, Syracuse, NY 13244, USA A. Messac (B) Department of Mechancal and Aerospace Engneerng, Syracuse Unversty, Syracuse, NY 13244, USA e-mal: messac@syr.edu 1 Introducton Partcle Swarm Optmzaton (PSO) s a stochastc optmzaton algorthm that mtates the dynamcs of socal behavor observed n nature. Ths algorthm was ntroduced by an Electrcal Engneer, Russel C. Eberhart, and a Socal Psychologst, James Kennedy (1995). The underlyng phlosophy of PSO and swarm ntellgence can be found n the book by Kennedy et al. (21). PSO has emerged over the years to be one of the most popular populatonbased heurstc optmzaton approaches. Several varatons of PSO have been reported n the lterature, and appled to dverse optmzaton problems n engneerng, basc scences and fnance (Banks et al. 28). The modfcatons of the PSO algorthm presented n ths paper are nspred by

2 368 S. Chowdhury et al. the authors research n product famly desgn (Chowdhury et al. 21a, 211) and wnd farm optmzaton (Chowdhury et al. 21b, 212). Both of these optmzaton problems (defned as sngle objectve) nvolve complex multmodal crteron functons and a hgh dmensonal system of mxeddscrete desgn varables. These problems are challengng, and generally requre a large number of system-model evaluatons. In the case of constraned sngle-objectve optmzaton problems, populaton-based algorthms (e.g., evolutonary and swarm-based optmzaton methods) often suffer from premature stagnaton (Banks et al. 28). Ths undesrable property can be attrbuted to an excessve and mostly unopposed pressure of exploraton or evoluton. The smultaneous presence of contnuous and dscrete desgn varables that may experence dfferng rates of evoluton further complcates the optmzaton scenaro. In ths paper, a new method s developed to both characterze and adaptvely nfuse dversty nto the populaton of canddate solutons. Ths method s an evoluton from earler dversty-preservaton methods reported n the PSO lterature, whch are later dscussed n Secton 1.2. The PSO algorthm presented n ths paper can address a mxture of dscrete and contnuous desgn varables. Two dstnct yet mutually coherent approaches are developed to address the dversty-preservaton ssues for dscrete and contnuous varables. In the followng two sectons (Sectons 1.1 and 1.2), we provde bref surveys of Mxed-Dscrete Nonlnear Optmzaton (MDNLO) methodologes and the major varatons of the Partcle Swarm Optmzaton algorthm reported n the lterature. Secton 1.3ntroducesthebasc prncplesand objectves of the Mxed-Dscrete Partcle Swarm Optmzaton (MDPSO) algorthm developed n ths paper. Sectons 2 and 3 descrbe the development of the MDPSO algorthm and the generalzed dversty characterzaton/preservaton technque, respectvely. Results and subsequent dscussons regardng the applcaton of MDPSO to varous standard test problems, and a real-lfe engneerng problem, a wnd farm layout optmzaton, are gven n Secton Exstng mxed-dscrete optmzaton approaches A sgnfcant amount of research has been done n developng algorthms for solvng Mxed-Integer Non-Lnear Programmng (MINLP) problems. Most of these algorthms are gradent-based search technques. Three major categores of gradent-based algorthms are () the branch and bound, () the cuttng plane, and () the outer approxmaton algorthms. A lst of these algorthms, related dscusson, and bblography can be found n the webstes of the MINLP World (21) and the CMU-IBM Cyber-Infrastructure for MINLP (21). These algorthms possess attractve numercal propertes, namely () fast convergence, () proof of optma, and () an ntrnsc ablty to deal wth constrants. However, gradent-based algorthms do not readly apply to the broad scope of engneerng desgn problems that may nvolve hghly nonlnear, non-smooth and multmodal crteron functons. Among populaton-based optmzaton methods, bnary Genetc Algorthms (GAs) (Goldberg 1989; Deb 29) have been reported to be effectve for dscrete optmzaton. Bnary GAs convert the desgn varables nto bnary strngs. Ths process leads to an approxmate dscrete representaton of the contnuous varables. A populaton of canddate solutons, each represented by a bnary strng, evolve over generatons generally through four stages: () ftness assgnment, () selecton, () crossover, and (v) mutaton. One of the most popular bnary GAs s the bn-nsga-ii developed by Deb et al. (22). Genetc algorthms have been successfully mplemented on MINLP problems, such as batch plant desgn (Ponsch et al. 27, 28). Another class of dscrete optmzaton algorthms, whch belong to Ant Colony Optmzaton (ACO), have also been reported n the lterature (Corne et al. 1999; Bonabeau et al. 1999). Applcatons of ACO-based algorthms to dscrete optmzaton problems nclude vehcle routng, sequental orderng, and graph colorng. There exsts n the lterature a handful of varatons of the PSO algorthm that can address dscrete and/or nteger varables. A summary of these varatons of PSO s dscussed n the followng secton. 1.2 Exstng Partcle Swarm Optmzaton algorthms A comprehensve revew of the background and the development of Partcle Swarm Optmzaton-based algorthms (untl 27) can be found n a chapter by Banks et al. (27). An extensve follow up revew of the varous attrbutes of PSO, and the applcablty of PSO to dfferent classes of optmzaton problems, such as unconstraned/constraned, combnatoral, and multcrtera optmzaton, can be found n a book chapter by Banks et al. (28). In ths secton, we provde a bref survey of reported varatons of PSO that address the followng crtcal optmzaton attrbutes: () mxed-dscrete varables, () populaton dversty preservaton, and () constrant handlng. A balance between exploraton, explotaton, and populaton-dversty n PSO requres approprate quantfcaton of the PSO coeffcents, or what s more popularly termed parameter selecton. One of the earlest strateges to balance exploraton and explotaton was the ntroducton of the nerta weght (Banks et al. 27). Eberhart (1998) nvestgated the nfluences of the nerta weght and the maxmum velocty on the algorthm performance.

3 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 369 Usng numercal experments, they proposed partcular values (and/or range of values) for the nerta weght and the maxmum velocty, and also suggested the applcaton of tme varyng nerta weght to further mprove the algorthm performance. Trelea (23) used standard results from dynamc systems theory to provde graphcal parameter selecton gudelnes. The applcatons of control theory by Zhang et al. (29), and chaotc number generaton by Alatas et al. (29) are among the recently proposed methods used to establsh parameter selecton gudelnes (for PSO). Several varatons of the PSO algorthm that can solve combnatoral optmzaton problems have been reported n the lterature. Kennedy and Eberhart (1997) presented one of the earlest modfcaton of PSO to address bnary varables. They defned the trajectores of the bnary varables n terms of the change n the probablty that a value of one or zero wll be taken. Tasgetren et al. (27) used constructon/destructon operators to perturb the dscrete component of the varable vector of a partcle n solvng a Travelng Salesman problem. A smlar combnatoral-pso concept was also developed and used by Jarbou et al. (28) for resource-constraned project schedulng. These varatons of the PSO algorthm provde effcent and robust performances, typcally for combnatoral optmzaton problems that are smlar to the correspondng reported applcatons. A majorty of these methods do not readly apply to the broad scope of mxed-dscrete optmzaton that nvolves problems wth: () ntegers and/or real-valued dscrete varables, () non-unformly spaced dscrete varable values (e.g., x [1, 3, 1, 1,...]) and () wdely df ferent szes of the set of feasble values for the dscrete varables (e.g., x 1 [, 1] and x 2 [1, 2,...,1]). Ktayama et al. (26) developed a more generalzed approach to address dscrete varables usng a penalty functon -dscrete varables are treated as contnuous varables by penalzng at the ntervals. However, the addtonal multmodal constrant n the penalty functon-based approach may undesrably ncrease the complexty of the desgn problem. Sngh et al. (21) presented an nterestng approach to address dscrete varables, whch manpulates the random operators n the partcle-velocty update step. Ths approach can be very helpful n mantanng consstency n the rates of evoluton of the contnuous and the dscrete varables. The needed stochastc and mutually ndependent attrbutes of the random operators that regulate the PSO dynamcs are restrcted n ths approach. Preservaton of the populaton dversty to avod premature convergence has been a long-standng challenge for PSO. Rapd swarm convergence, whch s one of the key advantages of PSO over other populaton-based algorthms, can however lead to stagnaton of partcles n a small suboptmal regon. Effcent and tme-varant parameter selecton has been tradtonally used as an mplct method to avod partcle stagnaton, thereby preservng populaton dversty. Over the years, the use of explct dversty preservaton technques have proved to be more effectve (Kennedy and Eberhart 1995). Krnk et al. (22) ntroduced a collson-avodance technque to mtgate premature convergence. Partcles comng wthn a defned vcnty of each other were allowed to bounce off; bouncng back along the old velocty vector (U-turn approach) was found to be most effectve. Blackwell and Bentley (22) also developed a dversty preservng swarm based on a smlar collson-avodance concept. The collson avodance schemes however requre an ntutve specfcaton of the threshold radus. A more globally applcable approach was developed by Rget and Vesterstrom (22), where the usual attracton phase was replaced by a repulson phase, when the entre populaton dversty fell below a predefned threshold. In ths case, the usual PSO locaton update formula s appled wth the drecton reversed. A metrc smlar to the standard devaton of the partcle locatons was used as the measure of dversty. Ths measure, however, does not readly account for the combned effects of the dstrbuton of the partcles and the overall spread of the partcles n the varable space. In other words, wth ther method (Rget and Vesterstrom 22), nfrequent extreme devatons (.e., a hgher kurtoss such as [,,,, 1, 1]) may yeld the same measure of dversty as frequent moderate devatons (e.g., [5, 6, 7, 5, 6, 7]), whch s msleadng. Other nterestng methodologes to address populaton dversty nclude: () ntroducton of a predatory partcle (Slva et al. 22), and () ntroducton of the concept of negatve entropy from thermodynamcs (e and Yang 22). Nevertheless, the consderaton of populaton dversty n a mxeddscrete/combnatoral optmzaton scenaro (n PSO) has rarely been reported n the lterature. The basc dynamcs of PSO does not account for system constrants. Several varatons of the PSO algorthm that ncorporate a constrant handlng capablty have been proposed: () a straght-forward method of consderng only feasble partcles for the best global and the best local solutons (Hu and Eberhart 22), () the use of conventonal dynamc penalty functons (Parsopoulos and Vrahats 22), () an effectve b-objectve approach where the net constrant serves as the the second objectve (Venter and Haftka 29), and (v) the use of the effcent constraned non-domnance prncples (Zavala et al. 25). In ths paper, we mplement the rules of constraned nondomnance ntroduced by Deb (29). Interestngly, the constraned non-domnance prncple can be perceved as an aspect of natural swarm ntellgence: communcaton of nformaton from partcle to partcle regardng whether they are beyond the feasble doman boundares, and/or how far beyond they are.

4 37 S. Chowdhury et al. 1.3 Mxed-Dscrete Partcle Swarm Optmzaton: prncples and objectves Ths paper presents fundamental modfcatons to the orgnal dynamcs of PSO, wth the am to solve hghly constraned sngle-objectve mxed-dscrete optmzaton problems. The development of ths Mxed-Dscrete PSO (MDPSO) s drven by the followng specfc objectves:. Develop an approxmaton technque that can address mxed-dscrete desgn varables through contnuous optmzaton;. Include a constrant handlng technque to deal wth both equalty and nequalty constrants; and. Formulate an explct dversty preservaton technque to avod the stagnaton of partcles. Effcent dversty preservaton (the thrd objectve) provdes an envronment conducve to accomplshng the frst and the second objectves. Hence, the thrd objectve s consdered to be the prmary contrbuton of ths paper. A method s formulated to characterze the exstng dversty n the populaton and adjust the dversty parameter(s)/coeffcent(s) at every teraton. Ths approach provdes a generalzed adaptve regulaton of the populaton dversty, whch can be mplemented n a majorty of populaton-based optmzaton algorthms and s not restrcted to PSO. For example, the concerned dversty parameter can be () the mutaton probablty n genetc algorthms (Deb 29), or () thetme-varyng acceleraton coef f cents (TVAC) n PSO (Ratnaweera et al. 24) or () the wndow-sze of the hypercube operator n Predator Prey algorthms (Chowdhury and Dulkravch 21), or (v) the random selecton rate n Ant Colony Optmzaton (Nakamch and Arta 24). A majorty of the exstng Mxed-Dscrete PSO algorthms are hndered by the effects of dfferng rates of evoluton of the contnuous and dscrete desgn varables. To avod ths undesrable scenaro, contnuous optmzaton s appled as the prmary search strategy for all varables, whether they are contnuous or dscrete. After the partcles have moved to ther new locatons, the dscrete component of the desgn vector for each partcle s approxmated to the nearest feasble dscrete doman locaton. In ths case, nearness s determned usng the Eucldan dstance n the dscrete varable space. As a result, although the varables evolve through contnuous search dynamcs, system-functon evaluatons are performed only at the allowed dscrete locatons. Ths approach s partly smlar to the strategy presented by Laskar et al. (22). A schematc of the proposed mxed-dscrete optmzaton approach for a sngle partcle at a partcular teraton s shown n Fg. 1.The term feasble dscrete space locaton as dscussed n ths Fg. 1 Flowchart for the mxed-dscrete optmzaton approach n MDPSO Secton (and as appears n Fg. 1) pertans to the feasblty wth respect to the constrants mposed by the dscreteness of the varable space, and not to the system constrants. Constrant handlng n MDPSO s performed usng the prncple of constraned non-domnance that was ntroduced by Deb et al. (22). Ths method has been successfully mplemented n the Non-domnated Sortng Genetc Algorthm-II (Deb et al. 22), Modfed Predator Prey algorthm (Chowdhury and Dulkravch 21), and other standard evolutonary algorthms. The MDPSO algorthm nvolves a set of coeffcents that regulate the nerta, the personal behavor, the socal behavor, and the dversty preservng behavor of the partcles. Parameter selecton n PSO s far from trval, as dscussed n the prevous secton. However, detaled analyss of the selecton of PSO parameters, and the ensung numercal behavor of the partcle dynamcs are not wthn the scope of ths paper. In ths paper, we specfcally ntend to provde:. the detaled formulaton of the Mxed-Dscrete PSO algorthm,. the underlyng hypothess supportng the proposed modfcatons, and. the performance of ths modfed algorthm on a wde varety of test cases. Over the past decade, a substantal amount of nterestng research n PSO has been reported n the lterature. Effectve characterstc modfcatons can therefore be adopted from

5 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 371 the exstng varatons of the algorthm to further advance the performance of the MDPSO algorthm. For valdaton purposes, the MDPSO algorthm s appled to () a set of standard unconstraned nonlnear optmzaton problems (Pohlhem 21; Mele and Cantrell 1969), and () a comprehensve set of MINLP problems (Schttkowsk 29). 2 Development of Mxed-Dscrete Partcle Swarm Optmzaton (MDPSO) 2.1 Basc swarm dynamcs A general mxed-dscrete sngle objectve constraned mnmzaton problem nvolvng m dscrete varables and a total of n desgn varables can be expressed as Mn f () subject to g j (), j = 1, 2,..., p h k () =, k = 1, 2,..., q where = [ ] x 1 x 2... x m x m+1... x n where p and q are the number of nequalty and equalty constrants, respectvely. In (1), s the desgn varable vector, where the frst m varables are dscrete and the next n m varables are contnuous. To solve ths optmzaton problem, the PSO algorthm s ntalzed wth N random partcles. To ths end, the Sobol s quasrandom sequence generator (Sobol 1976) s appled. Sobol sequences use a base of two to form successvely fner unform parttons of the unt nterval, then reorder the coordnates n each dmenson. The locaton of each partcle n the swarm s updated usng a velocty vector at each teraton; the velocty vector of a partcle s varable, and s tself updated at every teraton. In the MDPSO algorthm, the velocty vector update formula s redefned to allow for an explct dversty preservaton term. The modfed dynamcs of the partcle moton can be represented as (1) r 1, r 2 and r 3 are real random numbers between and 1; P s the best canddate soluton found for the th partcle; P g s the best canddate soluton for the entre populaton (also known as the current best global soluton); α, β l and β g are the user defned coeffcents that respectvely control the nertal, the explotve, and the exploratve attrbutes of the partcle moton; γ c s the dversty preservaton coeffcent for contnuous desgn varables; and the last term γ c r 3 ˆV t n the velocty update expresson s the dversty preservaton term, n whch the parameter ˆV t s a dvergng velocty vector. The conventonal partcle dynamcs n PSO encourages the partcles to congregate, often leadng to premature convergence. The purpose of the dvergng velocty vector, ˆV t, s to ntroduce a drecton of moton (n each partcle) that opposes such premature congregaton of partcles. Two dfferent choces for the dvergng velocty vector, ˆV t,are explored n ths paper: () the vector connectng the mean of the populaton of solutons to the concerned partcle, and () the vector connectng the current best global soluton to the concerned partcle, whch s ˆV t = t P g. Numercal experments showed that the vector t P g that repels the partcle from the best global solutons s more sutable for dversty preservaton. A representatve llustraton of the velocty vectors gudng the moton of a partcle (at every teraton) accordng to the modfed velocty update expresson s shown n Fg. 2. In ths fgure, the dotted vector (ponted up and rght) represents the orgnal new velocty vector of partcle- j,.e., when the dversty term s not ncluded. It s seen from Fg. 2 that the presence of the dversty vector (ponted down and left) reduces the tendency of the partcles to congregate towards the global best partcle. The determnaton of the dversty preservaton coeffcent (γ c ) s dscussed n Secton 3. The best global (P g )and the best local (P ) solutons are updated at every teraton usng the soluton comparson prncple. Ths soluton t+1 = t + V t+1, V t+1 where, = αv t ( + β l r 1 P t ) ( + βg r 2 Pg t + γ c r 3 ˆV t (2) ) t and t+1 are the locatons of the th partcle at the t th and the (t + 1) th teratons, respectvely; V t and V t+1 are the velocty vectors of the th partcle at the t th and the (t + 1) th teratons, respectvely; Fg. 2 Modfed partcle dynamcs as gven by (2)

6 372 S. Chowdhury et al. comparson prncple s based on the values of the objectve functons and the constrant functons of the canddate solutons beng compared. Ths prncple s dscussed n Secton 2.3. The contnuous update process (2) s appled to all the desgn varables of a partcle, rrespectve of whether they are contnuous or dscrete. Ths approach promotes coherent rates of evoluton of the contnuous and dscrete varables. Followng the contnuous update process, the dscrete component of the desgn vector s updated to nearby feasble dscrete locatons. As n the prevous Secton, feasblty n ths case pertans to the constrants mposed by the dscreteness of the varable space, and not to the system constrants. 2.2 Updatng dscrete desgn varables In a mxed-dscrete optmzaton scenaro, the desgn space can be dvded nto a contnuous doman and a dscrete doman, whch correspond to the contnuous and the dscrete components of the desgn varable vector, respectvely. Followng a contnuous search PSO step (2), the locaton of a partcle n the dscrete doman s defned by a local hypercube that s expressed as {( ) ( ) ( )} H d = x1 L, xu 1, x2 L, xu 2,..., xm L, xu m, x L x x U, = 1, 2,...,m (3) In (3), m s the number of dscrete desgn varables, and x denotes the current locaton of the canddate soluton n the dscrete doman. The parameters x L and x U represent two consecutve feasble values of the th dscrete varable that defne the boundares of the local hypercube. The total number of vertces n the hypercube s equal to 2 m. The values, x L and x U, can be obtaned from the dscrete vectors that need to be specfed a pror for each dscrete desgn varable. A relatvely straght-forward crteron, called the Nearest Vertex Approach (NVA), s developed to approxmate the current dscrete-doman locaton of the canddate soluton to one of the vertces of ts local hypercube, H d (3). The NVA approxmates the dscrete-doman locaton to the nearest vertex of the local hypercube (H d ), on the bass of the Eucldean dstance. Ths approxmaton s represented by = [ x 1 x 2 x m ], where { x L x =, f x x L x x U x U, otherwse = 1, 2,...,m (4) In (4), represents the approxmated dscrete-doman locaton based on the NVA. Another approach to approxmate dscrete doman locatons was also explored ths approach s called the Shortest Normal Approach (SNA) (Chowdhury et al. 21a). The SNA approxmates the dscrete doman locaton of a partcle to the local hypercube vertex that has the shortest normal dstance from the latest velocty vector of the partcle. Numercal experments showed that the NVA s sgnfcantly less expensve and more relable than the SNA; hence, NVA s used for the applcaton of MDPSO to the test problems n ths paper. An llustraton of the NVA and the SNA for a 2-D dscrete doman s shown n Fg. 3. Ths determnstc approxmaton seeks to retan the search characterstcs of the contnuous PSO dynamcs, whle ensurng that the system-model s evaluated only at the allowed dscrete doman locatons. Such an approxmaton strategy can be readly mplemented n other nongradent based contnuous optmzaton algorthms as a post process to the usual contnuous search step at every teraton. 2.3 Soluton comparson and constrant handlng Soluton comparson s essental n PSO at every teraton, to determne and update the best global soluton n the populaton and the best local soluton for each partcle. The prncple of constraned non-domnaton (Deb et al. 22)s used to compare solutons. Accordng to ths prncple, canddate soluton- s sad to domnate canddate soluton- j f and only f one of the followng scenaros occur: I. Soluton- s feasble and soluton- j s nfeasble or, II. Both solutons are nfeasble and soluton- has a smaller net constrant volaton than soluton- j or, Local hypercube Parent soluton x 2 U SNA vertex x 1 L Chld soluton Shortest Eucldean Dstance Shortest Normal Dstance Connectng Vector Fg. 3 Illustraton of the NVA and the SNA approxmaton NVA vertex x 1 U x 2 L

7 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 373 III. Both solutons are feasble; n addton, soluton- s not worse than soluton- j n any objectve, and soluton- s better than soluton- j n at least one objectve. the populaton s lkely to become computatonally prohbtve n the case of hgh dmensonal optmzaton problems. A novel dversty characterzaton/metrc s developed n ths paper. Salent features of ths metrc are: In the case of a mult-objectve problem, t s possble that none of the above scenaros apply, whch mples that the solutons are non-domnated wth respect to each other. The net constrant volaton f c () s determned by f c () = p j=1 max ( g j, ) + q k=1 max ( h k ɛ, ) (5) where g j and h k represent the normalzed values of the j th nequalty constrant and k th equalty constrant, respectvely. In (5), ɛ represents the tolerance specfed to relax each equalty constrant; a tolerance value of 1.e 6 s used for the case studes n ths paper. The soluton comparson approach n MDPSO favors feasblty over the objectve functon value. Ths approach has a tendency to drve solutons towards and nto the feasble regon durng the ntal teratons of the algorthm (Chowdhury and Dulkravch 21; Chowdhury et al. 29). Throughout ths ntal phase, domnance scenaros I and II are promnently actve. When a majorty of the partcles have moved nto the feasble space, scenaro III takes over; soluton comparsons are then progressvely determned by the magntude of the objectve functon. In the case of hghly constraned sngle-objectve problems, ths soluton comparson approach, together wth the ntrnsc swarm dynamcs, can lead to an apprecable loss n dversty. Ths undesrable phenomenon occurs prmarly durng the feasblty-seekng process of optmzaton. To counter ths undesrable characterstc of the partcle moton n the MDPSO, the explct dversty preservaton term, γ c r 3 ˆV t (refer (2)), s added to the velocty vector, as ntroduced n Secton Dversty preservaton The frst step n dversty preservaton s to characterze and quantfy the exstng populaton dversty wth respect to the desgn varable space. A consstent measure of dversty should smultaneously capture the overall spread and the dstrbuton of the partcles n the populaton. Deb et al. (22) used a performance metrc to measure the spread of solutons along the computed Pareto front n the objectve space. A smlar metrc, mplemented n the varable space, would be an almost deal choce for dversty characterzaton. However, the requred determnaton of the nearest-neghbor Eucldan dstances for every member of It seeks to effectvely capture the two dversty attrbutes: the overall spread and the dstrbuton of partcles. It s computatonally nexpensve to mplement, f requred, at every teraton. Separate dversty metrcs and dversty preservaton mechansms are formulated for contnuous and dscrete desgn varables. The dversty metrcs and the correspondng dversty preservaton coeffcents are estmated for the entre populaton at the start of an teraton. The dversty metrcs are then updated usng a common factor that seeks to account for the partcle dstrbuton. In the case of contnuous desgn varables, the ntal dversty metrc s gven by the normalzed sde length of the smallest hypercube that encloses all the partcles. Ths metrc s expressed as D c = n =m+1 x t,max x max x t,mn x mn 1 n m where x t,max and x t,mn are respectvely the maxmum and the mnmum values of the th desgn varable n the populaton at the t th teraton; and x max and x mn, respectvely, represent the specfed upper and lower bounds of the th desgn varable. The parameters n and m represent the total number of varables and the number of dscrete varables, respectvely. It s mportant to note that the concept of enclosng hypercube (of the entre populaton or an elte subset of the populaton) has been used n dfferent ways to mprove the effcences of heurstc optmzaton algorthms. For example, Wang et al. (29)usedthehypercube that encloses a set of elte solutons (surroundng the feasble regon) to shrnk the search space, and consequently accelerate the feasble regon seekng process durng constraned evolutonary optmzaton. An undesrable and common scenaro n heurstc algorthms s the presence of one or more outler partcles, when the majorty of the partcles are concentrated n a sgnfcantly smaller regon. Occurrence of ths scenaro leads to an apprecable overestmaton of the populaton dversty (D c ). To overcome ths deleterous scenaro, as well as to account for the dstrbuton of canddate solutons, the dversty metrc s further modfed. A hypercubod regon s frst constructed around the best global canddate soluton n the overall varable-space (ncludng contnuous and dscrete varables). Ths hypercubod regon s defned such that ts (6)

8 374 S. Chowdhury et al. volume s a fracton of the volume of the smallest hypercube enclosng all the partcles. The user-def ned fracton s represented by the parameter λ, where < λ < 1. The number of partcles wthn the fractonal hypercubod regon s then determned and used to adjust the contnuous dversty metrc (enclosng-hypercube sde length), n order to better account for the partcle dstrbuton. The boundares of the fractonal hypercubod regon s gven by x t,max x t,mn [ x t,mn = max + λ x t mn ( P g, +.5λ x t, xt,max [ x t,max λ x t = mn ( max P g,.5λ x t, xt,mn = 1, 2,...,n ) ) ], where x t = x t,max x t,mn ; the parameters x t,max and x t,mn respectvely represent the upper and the lower boundares of the fractonal doman for the desgn varable x ; and P g, s the th varable of the best global soluton. The adjusted contnuous dversty metrc D c s then expressed as D c = ] (7) ( λ N + 1 ) 1 n Dc (8) N λ + 1 where N λ s the number of partcles n the λ-fractonal doman. The dversty coeffcent, γ c, for contnuous varables s then defned as a functon of the contnuous dversty metrc, whch s gven by ( D c 2 γ c = γ c exp σ c = 2σ 2 c 1 2ln(1/γmn ) ), where and γ c and γ mn are specfed constants that respectvely control the scale of the dversty coeffcent and the varance of the dversty coeffcent wth the dversty metrc. The order of magntude of the dversty-scalng constant γ c should be one; or, n other words, t should be comparable to that of the exploratve coeffcent, β g.intherangeto1 for D c, the dversty coeffcent s a monotoncally decreasng functon. The nature of ths functon for dfferent orders of magntude of γ mn s shown n Fg. 4. In the case of dscrete desgn varables, the dversty s characterzed ndependently for each varable n order to address the followng two factors:. The effectve dversty n the th dscrete varable depends on (1) the number of feasble values avalable for that varable and (2) the dstrbuton of these feasble values. (9) Dversty coeffcents, γ c and γ d, γ mn = 1 1 ; M = 1 γ mn = 1 2 ; M = 1 2 γ mn = 1 3 ; M = 1 3 γ mn = 1 4 ; M = 1 4 γ mn = 1 5 ; M = 1 5 γ mn = 1 6 ; M = 1 6 γ mn = 1 7 ; M = 1 7 γ mn = 1 8 ; M = 1 8 γ mn = 1 9 ; M = 1 9 γ mn = 1 1 ; M = Dversty metrcs, D and D c d, Fg. 4 Varaton of the dversty coeffcents γ c and γ d, wth the dversty metrcs D c and D d,, respectvely, llustrated at () dfferent values of γ mn for contnuous varables, and () dfferent szes (M) ofthe feasble set for dscrete varables, wth γ d = 1. Dversty preservaton n dscrete varables should seek to avod the stagnaton of partcles nsde a local dscrete-space hypercube H d. The ntal dversty metrc (D d ) for dscrete desgn varables s a vector of the normalzed dscrete varable ranges that span the current populaton. Ths metrc s expressed as D d, = xt,max x max x t,mn x mn, = 1, 2,...,m (1) where D d, s the component of the dscrete dversty metrc correspondng to the th dscrete varable. Subsequently, n order to better account for the dstrbuton of solutons, the dscrete dversty metrc s adjusted as D d, = ( λ N + 1 ) 1 n Dd, (11) N λ + 1 where D d, s the adjusted dscrete dversty metrc. It s mportant to note how the parameter λ couples the dversty n contnuous and dscrete desgn varables. As a result, the dversty preservaton mechansms for contnuous and dscrete varables are expected to work n coherence wth each other. Dversty preservaton for dscrete varables s accomplshed through modfcaton of the dscrete update process descrbed n Secton 2.2. The otherwse determnstc approxmaton of the partcle to a nearby feasble dscrete locaton s replaced by a stochastc update process. Ths stochastc update gves a partcle the opportunty to jump out of a local hypercube, thereby reducng the possblty of stagnaton of the swarm s dscrete component. A vector

9 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 375 of dscrete-varable dversty coeffcents, γ d, s defned to further regulate the updatng of dscrete varables, wth the objectve to mnmze the possblty of ther premature stagnaton. A random number (r 4 ) s generated between and 1, and the stochastc update for the generc th dscrete varable (x ) of a partcle s then appled usng the followng rules:. If r 4 s greater than the dversty coeffcent γ d,,then update the dscrete varable usng (4).. If r 4 s less than or equal to γ d,, then randomly approxmate x to ether x L or x U (defned n (4)). The dscrete-varable dversty coeffcent, γ d,, that regulates the stochastc update rules s desgned to adapt to the sze of the set of feasble values for the th dscrete varable. Ths approach avods a false mpresson of consderable dversty, n the case of dscrete varables that take a relatvely small szed set of feasble values. The dscrete dversty coeffcent s defned as ( D 2 ) d, γ d, = γ d exp, where σ d, = 1 2lnM = 1, 2,...,m 2σ 2 d, (12) and where M represents the sze of the set of feasble values for the th dscrete varable, and γ d s a prescrbed constant between and 1. For any estmated value of the populaton dversty, a hgher value of the prescrbed parameter, γ d, makes the random update of the dscrete doman locaton more lkely. It s mportant to note that, whle the contnuousvarable dversty coeffcent (γ c ) drectly regulates the partcle moton (n the locaton update step), the dscretevarable dversty coeffcents (γ d, ) control the updatng of the dscrete varables as a post-process (durng the NVA applcaton) n every pertnent teraton. In addton, the same value of γ c s used for all desgn varables at a partcular teraton, whereas a dfferent value of γ d, s used for each generc th dscrete varable. An llustraton of the dscrete dversty coeffcent for dfferent szes of the set of feasble values sshownnfg.4. 4 Numercal experments To valdate the Mxed-Dscrete Partcle Swarm Optmzaton (MDPSO) algorthm, we apply t to two dfferent classes of sngle-objectve optmzaton problems: () standard unconstraned problems, most of whch are multmodal, and () Mxed-Integer Non-Lnear Programmng (MINLP) problems. The MDPSO algorthm s also appled to a large scale real lfe engneerng problem: wnd farm optmzaton. These three sets of numercal experments are dscussed n the followng three sub-sectons. The values of the prescrbed MDPSO parameters for the three sets of numercal experments are gven n Table Unconstraned standard optmzaton problems The new MDPSO algorthm s appled to a set of nne standard unconstraned nonlnear optmzaton test problems wth only contnuous varables to compare ts performance wth that of the basc PSO. For a majorty of these test problems, the basc PSO s expected to offer an effectve soluton. The MDPSO s specfcally desgned to address complex constraned and/or mxed-dscrete optmzaton problems. Wth ths set of numercal experments, we partcularly nvestgate whether the new MDPSO features related to dversty preservaton, ntroduce any unexpected characterstcs. The frst eght test problems have been borrowed from the lst of sample sngle objectve optmzaton problems provded n the MATLAB Genetc and Evolutonary Algorthm Toolbox (GEATbx) Documentaton (Pohlhem Table 1 User-defned constants n PSO Parameter Unconstraned MINLP Wnd farm problems optmzaton α β g β l γ c.1,.5, γ d.7 1. γ mn 1.e 1 1.e 1 1.e 5 Populaton sze (N) 1 n 1 n 2 n Fractonal doman sze (λ N).25 N.1 N.1 N Allowed number of functon calls 1, 5, 6,

10 376 S. Chowdhury et al. Table 2 Standard unconstraned optmzaton problems Test problem Functon name Number of varables Complexty attrbute 1 Rosenbrock s valley 2 Long relatvely flat valley 2 Rastrgn s functon 2 Hghly multmodal 3 Schwefel s functon 2 Hghly multmodal 4 Grewangk s functon 2 Hghly multmodal 5 Ackley s path functon 2 Hghly multmodal 6 Mchalewcz s functon 1 Flat regons and multmodal 7 Easom s functon 2 Mostly flat search space 8 Goldsten Prce s functon 2 Extensve flat regon 9 Mele Cantrell 4 Multmodal 21). The GEATbx problems were orgnally developed and reported by dfferent researchers from the desgn and optmzaton communty. The last test problem from Table 2 (Mele Cantrell functon) has been borrowed from the paper by Mele and Cantrell (1969). Detals of the standard unconstraned test problems are summarzed n Table 2. The MDPSO algorthm s appled to each test problem, usng three dfferent values of the dversty coeffcent scalng constant: γ c =.1,.5, 1.. Each test problem s run 1 tmes, wth a partcular γ c value, to compensate for the effects of the random operators on the overall algorthm performance. Results of the conventonal PSO was obtaned by specfyng the dversty coeffcent scalng constant, γ c,to be zero, whle other basc PSO parameters were fxed at the same values as gven n Table 1. The algorthms are termnated when the best global soluton does not mprove by at least 1.e 1 tmes ts objectve value n 1 consecutve teratons. The convergence hstores for the Mele Cantrell test functon from representatve runs of the MDPSO and a representatve run of the conventonal PSO are shown n Fg. 5. The actual mnmum objectve value for ths test functon s.. It can be observed from Fg. 5 that the algorthms perform very well for the multmodal Mele Cantrell test functon. Wth the dversty scalng constant equal to.1 (black dashed lne), the rate of convergence of MDPSO s approxmately twce that of the conventonal PSO the objectve functon reduces to 1.e 7 n half the number of functon calls. Wth the dversty scalng constant equal to 1. (grey long-dashed lne), the MDPSO algorthm converges slghtly slower than the conventonal PSO algorthm. Ths phenomenon can be attrbuted to the ncreased reducton n the partcle veloctes towards the global optmum, caused by the preservaton of a larger amount of populaton dversty among the partcles. For each test problem n Table 2, the actual mnmum of the objectve functon s known. Usng the actual mnmum objectve functon value, a normalzed relatve error s evaluated to represent the optmzaton performance. Ths normalzed relatve error (ε f ) s expressed as ε f = f comp mn comp f mn f act fmn act mn f act, mn, f f act mn = f f act mn = (13) where f comp mn and fmn act are the computed mnmum and the actual mnmum of the objectve functon, respectvely. The normalzed relatve errors gven by the best and the worst optmzed solutons among the 1 runs of each test problem are shown n Fg. 6a and b. Fgure 6a specfcally shows the resultng errors when conventonal PSO and MDPSO are appled to the same test problems. Fgure 6b specfcally shows the resultng errors when MDPSO s appled to the test problems usng dfferent values of the specfed dversty scalng constant (γ c ). Further detals, regardng the Objectve Functon PSO MDPSO (γ c =.1) MDPSO (γ c =.5) MDPSO (γ c =1.) Number of Functon Evaluatons Fg. 5 Convergence hstores of the MDPSO and the conventonal PSO for the Mele Cantrell functon

11 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 377 Best Soluton: PSO Worst Soluton: PSO 1 4 Best Soluton: MDPSO 1 2 Worst Soluton: MDPSO Best Soluton: γ c =.1 Worst Soluton: γ c =.1 Best Soluton: γ c =.5 Worst Soluton: γ c =.5 Best Soluton: γ c =1. Worst Soluton: γ c =1. NormalzedError Normalzed Error (a) Usng PSO, and MDPSO wth γ c = 1. (b) Usng MDPSO wth γ c =.1,.5, 1. Fg. 6 Maxmum and mnmum values (among 1 runs) of the normalzed relatve error obtaned for the standard unconstraned test problems performance of the MDPSO algorthm wth the dversty scalng constant equal to 1., are provded n Table 3. Fgure 6a shows that the MDPSO algorthm performs as well as or better than the conventonal PSO algorthm for most of the standard unconstraned test problems, except for test problem 2. It s observed from Fg. 6a and Table 3 that nether of the algorthms could provde satsfactory solutons for test problem 6, Mchalewcz s functon; these observatons can be attrbuted to the exstence of extensve flat regons and multmodalty n the Mchalewcz s functon (Pohlhem 21). Overall, Fg. 6a llustrates that the addtonal dversty preservaton features n MDPSO does not ntroduce any undesrable characterstcs nto the dynamcs of the PSO algorthm. Fgure 6b llustrates that the performance of MDPSO s margnally senstve to the specfed value of the dversty scalng constant (γ c )n the case of these unconstraned contnuous problems the relatve errors gven by MDPSO for the three dfferent values of the dversty scalng constant are close to each other. The standard devaton n the computed mnma obtaned from the 1 runs for each test problem (Table 3) s observed to be relatvely small when compared to the correspondng actual mnma. Ths observaton further llustrates the consstency n the performance of the MDPSO algorthm. 4.2 Mxed-Integer Nonlnear Programmng (MINLP) problems The MDPSO algorthm s appled to an extensve set of nnety-eght Mxed-Integer Non-Lnear Programmng (MINLP) test problems; these test problems were obtaned from the comprehensve lst of one hundred MINLP problems reported by Schttkowsk (29). The problems Table 3 Performance of MDPSO (wth γ c = 1.) on the standard unconstraned test problems Test problem Actual mnmum Best computed Worst computed Standard devaton mnmum mnmum of computed mnma 1.E E E E 11 2.E E E E E E E E+1 4.E E E E 2 5.E E E E E E E E E+ 1.E+ 1.E E E+ 3.E+ 3.E E 11 9.E E E E 8

12 378 S. Chowdhury et al. numbered 1 and 1 n the orgnal lst (Schttkowsk 29) have not been tested n ths paper. A majorty of these MINLP test problems belong to the GAMS Model Lbrary MINLPlb (Busseck et al. 27), and have been wdely used to valdate and compare optmzaton algorthms (Schttkowsk 29). These MINLP test problems present a wde range of complextes: the total number of desgn varables vares from 2 to 5; the numbers of bnary desgn varables and nteger desgn varables vary from to 16 and to 5, respectvely; the total number of constrants (ncludng equalty and nequalty) vares from to 54; the number of equalty constrants vares from to 17. Smlar to the prevous set of numercal experments, each MINLP test problem s run 1 tmes to compensate for the performance effects of the random operators n the algorthm. For each run, the algorthm s termnated when the best global soluton does not mprove by at least 1.e 6 tmes ts objectve value n 1 consecutve teratons. For ease of llustraton and ready nterpretaton of the results obtaned by MDPSO, we dvde the set of 98 MINLP test problems nto sx classes based on the number of desgn varables and the presence of contnuous varables. These MINLP problem classes are shown n Table 4. The normalzed relatve errors correspondng to the best and the worst solutons among the 1 runs obtaned (by MDPSO) for each test problem from the sx classes are llustrated separately n Fg. 7a f. These fgures also show the number of desgn varables (sold gray lne) and the number of constrants (dashed gray lne) n each test problem, to help understand ther mpact on the optmzaton performance. A hstogram of the relatve errors s shown n Fg. 8. Its helpful to note that, n the case of several purely nteger test problems, a zero relatve error s obtaned through optmzaton. In order to allow a logarthmc scale llustraton of the errors, these zero errors are replaced by an artfcal error value of 1.e 12 n the fgures. Fgure 7a and b show that MDPSO performs sgnfcantly better for the purely nteger problems (Class-1A) than for the mxed-nteger problems (Class-1B). Ths observaton Table 4 MINLP problem classes Number Wthout contnuous Wth contnuous of varables (n) varables (n = m) varables (n > m) 2 n 5 Class-1A Class-1B 6 n 1 Class-2A Class-2B 11 n 5 Class-3A Class-3B can be partly attrbuted to the practcal possblty of fndng the exact mnmum (no error) f the varables are all ntegers nstead of contnuous real numbers. Overall, a majorty of the Class-1 test problems (wth n 5) are observed to have converged to relatve errors less than 1.e 4. Smlarly, for Class-2 problems, Fg. 7c and d show that MDPSO performs better n the case of purely nteger problems (Class-2A). Interestngly, for the Class-2A problems, the best soluton among 1 runs s observed to mostly converge to the exact mnma (ε f = 1.e 12 n the fgures). It s observed that, for a majorty of the Class-3 problems (Fg. 7e and f), MDPSO has not converged beyond a relatve errorof1.e 4, whch can be prmarly attrbuted to the hgh number of constrants n these MINLP problems. Some of the Class-3 problems have more than 3 constrants. A hgher number of desgn varables (11 n 5) mght also have partly contrbuted to the lack of convergence n the Class-3 problems. The hstogram n Fg. 8 llustrates the dstrbuton of the relatve error, on a logarthmc scale, for all the 1 98 test problem runs. It s observed that the MDPSO algorthm has converged to acceptable relatvely errors of less than 1.e 3 n approxmately 5 % of the test problem runs. It s mportant to note that the same prescrbed parameter values were specfed for all MINLP test problems (Table 1) for a far llustraton of algorthm performance. The set of MINLP test problems present a wde varety of nonlnear crtera functons and problem complextes, whch deally demands dfferent parameter values to be specfed for characterstcally dfferent MINLP problems. The development of general gudelnes regardng how to specfy the prescrbed MDPSO parameter values for dfferent types of MINLP problems s therefore an mportant topc for further research. Fgure 9a and b llustrate the net constrant volaton (5) correspondng to the best and the worst solutons obtaned by MDPSO, respectvely for MINLP problems wthout and wth equalty constrants. It s observed from Fg. 9a that the MDPSO algorthm has successfully found the feasble space n a majorty of the MINLP problems wthout nequalty constrants; the handful of exceptons s observed to generally nvolve more than 35 desgn varables and 35 constrants. The feasblty success of MDPSO s lower for the MINLP problems wth equalty constrants than those wthout equalty constrants (Fg. 9b). Overall, MDPSO found the feasble regon wth the best soluton n a majorty of the test problems (9 out 98). Cabrera and Coello (27) stated that sgnfcant advancement s necessary to extend the applcaton of standard PSO to solve constraned optmzaton problems. From that perspectve, the performance of MDPSO n fndng the feasble regon seems promsng, partcularly consderng that the concerned test problems are of mxed-dscrete nature. However, addressng equalty

13 A mxed-dscrete Partcle Swarm Optmzaton algorthm wth explct dversty-preservaton 379 Normalzed Error Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants (a) Class-1A problems: n = m and n Number of Varbales/Constrants Normalzed Error Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants (b) Class-1B problems: n > m and n Number of Varbales/Constrants Normalzed Error Normalzed Error Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants (c) Class-2A problems: n = m and 6 n 1 Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants (e) Class-3A problems: n = m and 11 n Number of Varbales/Constrants Number of Varbales/Constrants Normalzed Error Normalzed Error Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants (d) Class-2B problems: n > m and 6 n (f) Class-3B problems: n > m and 11 n 5 Best Soluton Error Worst Soluton Error No. of Varables No. of Constrants Number of Varbales/Constrants Number of Varbales/Constrants Fg. 7 Normalzed relatve errors for the best and the worst solutons (among 1 runs) obtaned by MDPSO for the MINLP problems