Particle Swarm Optimization Combining Diversification and Intensification for Nonlinear Integer Programming Problems

Transcription

1 Paricle Swarm Opimizaion Combining Diversificaion and Inensificaion for Nonlinear Ineger Programming Problems Takeshi Masui, Masaoshi Sakawa, Kosuke Kao and Koichi Masumoo Hiroshima Universiy 1-4-1, Kagamiyama, Higashi-Hiroshima, JAPAN { ak-masui, sakawa, Absrac In his research, focusing on nonlinear ineger programming problems, we propose an approximae soluion mehod based on paricle swarm opimizaion proposed by Kennedy e al. And we developed a new paricle swarm opimizaion mehod which is applicable o discree opimizaion problems by incoporaing a new mehod for generaing iniial search poins, he rounding of values obained by he move scheme and he revision of move mehods. Furhermore, we showed he efficiency of he proposed paricle swarm opimizaion mehod by comparing i wih an exising mehod hrough he applicaion of hem ino he numerical examples. Moreover we expanded revised paricle swarm opimizaion mehod for applicaion o nonlinear ineger programming problems and showed more effeciency han geneic algorihm. However, variance of he soluions obained by he PSO mehod is large and accuracy is no so high. Thus, we consider improvemen of accuracy inroducing diversificaion and inensificaion. I. INTRODUCTION In general, acual various decision making siuaions are formulaed as large scale mahemaical programming problems wih many decision variables and consrains. If a value of he decision variables is ineger, he problem is called an ineger programming problem. For ineger programming problems, we can have opimal soluion by applicaion of he dynamic programming fundamenally. However, since opimizaion problems become larger and more complicaed, a high speed and accurae opimizaion mehod is expeced. In paricular, for nonlinear ineger programming problems (NLIP), here are no he general sric mehod or approximaion mehod, such as branch and bound mehod for linear programming problems. In such a case, a soluion mehod depended on propery in problems is proposed. In recen years, a paricle swarm opimizaion (PSO) mehod was proposed by Kennedy e al. [2] and has araced considerable aenion as one of promising opimizaion mehods wih higher speed and higher accuracy han hose of exising soluion mehods. And Kao e al. showed he efficiency of improved PSO mehod han geneic algorihm for nonlinear programming problems [1]. Moreover we expanded revised paricle swarm opimizaion mehod for applicaion o NLIP and showed more effeciency han geneic algorihm [5]. However, variance of he soluions obained by he PSO mehod is large and accuracy is no so high. In his research, we focus on NLIP and consider improvemen of accuracy combining diversificaion and inensificaion. II. NONLINEAR INTEGER PROGRAMMING PROBLEMS In his research, we consider general nonlinear ineger programming problem wih consrains as follows: minimize f(x) subjec o g i (x) 0, i = 1, 2,..., m l j x j u j, j = 1, 2,..., n x = (x 1, x 2,..., x n ) T Z n where f( ), g i ( ) are convex or nonconvex real-valued funcions, l j and u j are he lower bound and he upper bound of each decision variable x j. III. PARTICLE SWARM OPTIMIZATION Paricle swarm opimizaion [2] mehod is based on he social behavior ha a populaion of individuals adaps o is environmen by reurning o promising regions ha were previously discovered [3]. This adapaion o he environmen is a sochasic process ha depends on boh he memory of each individual, called paricle, and he knowledge gained by he populaion, called swarm. In he numerical implemenaion of his simplified social model, each paricle has hree aribues: he posiion vecor in he search space, he curren direcion vecor, he bes posiion in is rack and he bes posiion of he swarm. Sep 1: Generae he iniial swarm involving N paricles a random. Sep 2: Calculae he new direcion vecor for each paricle based on is aribues. Sep 3: Calculae he new search posiion of each paricle from he curren search posion and is new direcion vecor. Sep 4: If he erminaion condiion is saisfied, sop. Oherwise, go o Sep 2. To be more specific, he new direcion vecor of he i-h paricle a ime, is calculaed by he following scheme inroduced by Shi and Eberhar [7]. (1) i := ω v i + c 1 R 1(p i x i) + c 2 R 2(p g x i) (2) 43

2 In (2), R1 and R2 are random numbers beween 0 and 1, p i is he bes posiion of he i-h paricle in is rack a ime and p g is he bes posiion of he swarm a ime. There are hree parameers such as he ineria of he paricle ω, and wo parameers c 1, c 2. Then, he new posiion of he i-h paricle a ime, x +1 is calculaed from (3). x +1 i := x i + i (3) where x i is he curren posiion of he i-h paricle a ime. Afer he i-h paricle calculaes he nex search direcion vecor i by (2) in consideraion of he curren search direcion vecor v he direcion vecor going from he curren search posiion x i o he bes search posiion in is rack p i and he direcion vecor going from he curren search posiion x i o he bes search posiion of he swarm p g, i moves from he curren posiion x i o he nex search posiion x+1 i calculaed by (3). In general, he parameer ω is se o large values in he early sage for global search, while i is se o small values in he lae sage for local search. For example, i is deermined as: ω := ω 0 (ω0 ω Tmax ) (4) 0.75 T max where is he curren ime, T max is he maximal value of ime, ω 0 is he iniial value of ω and ω T max is he final value of ω. The search procedure of PSO is shown in Fig. 1. If he nex search posiion of he i-h paricle a ime, x +1 is beer han he bes search posiion in is rack a ime, p i.e., f(x +1 i ) f(p i ), he bes search posiion in is rack is updaed as p +1 i := x +1 i. Oherwise, i is updaed as p +1 i := p i. Similarly, if p+1 i is beer han he bes posiion of he swarm, p g, i.e., f(p +1 i ) f(p g), hen he bes search posiion of he swarm is updaed as p +1 g := p +1 i. Oherwise, i is updaed as p +1 g := p g. Moreover we expanded revised paricle swarm opimizaion mehod for applicaion o NLIP () [5]. In, we enabled applicaion of he revised PSO mehod o NLIP by incorporaing using ineger random number in generaing iniial search posiion and rounding search direcion vecor i in updaing equaion (3). However, we simply make he search direcion vecor ineger value and all elemens of he search direcion vecor become 0. Thus, on he elemen ha absolue value is maximum in elemens of he search direcion vecor before rounding, we se he search direcion vecor 1 or -1 depending on he plus or minus. Therefore, we revised ha all of he paricles always move. The process of can be shown in Fig. 2. r 0 f x T S x p g Fig. 1. p i x i v i v i +1 x i +1 Movemen of an individual. In he original PSO mehod, however, here are drawbacks ha i is no direcly applicable o consrained problems and is liable o sopping around local soluions. To deal wih hese drawbacks of he original PSO mehods, Kao e al. incorporaed he bisecion mehod and a homomorphous mapping o carry ou he search considering consrains. In addiion, Kao e al. incorporaed he muliple sreching echnique and modified move schemes of paricles o resraining he sopping around local soluions [1]. Fig. 2. The algorihm of. IV. RPSO COMBINING DIVERSIFICATION AND INTENSIFICATION We expanded revised paricle swarm opimizaion mehod for applicaion o NLIP and showed more effeciency han geneic algorihm [5]. However, variance of he soluions obained by is large and accuracy is no so high. In his research, we consider improvemen of accuracy of. A firs we inroduce new move scheme wih loop in order o preven a paricle from sopping around boundary. Nex, for diversificaion (reinforcemen of global search), we inroduce new move scheme o resrain o sopping around local opimal soluions. Furhermore, for inensificaion (reinforcemen of convergence search in promising region), we inroduce local search in he bes search posiion of he swarm. 44

3 A. new move scheme wih loop To deal wih drawbacks, we analyzed search process in deail and found ha he decision variable of some paricles was fixed on he upper or lower bound (boundary) on he way of search. Such a paricle is easy o sop a he boundary and causes depression of search efficiency. Moreover, for example in such a siuaion, in case ha he decision variable aking upper (lower) bound value a he curren search posiion akes lower (upper) bound value a he opimal soluion, i is no easy for he paricle moving in feasible region o move around he opimal soluion. Thus, moving wih loop from upper o lower or from lower o upper for he decision variable ha absolue value is maximum in he elemen of he search direcion vecor o boundary ouside on he decision variable fixed in he boundary (upper or lower bound), we resrain o he sopping around he boundary for a paricle (Fig. 3). boundary akes ineger, his echnique does no work well, and here is he case ha a paricle sops around local opimal soluions. We propose new move scheme o resrain o sopping around local opimal soluions if he decision variable akes ineger. To be more specific, if he bes search posiion of he swarm is no updaed for a long generaion, we muliply he hird erm of righ-hand side in (2) by -1 and incorporae he direcion away from p g, hen decide nex search direcion vecor p +1 i following equaion (Fig. 4). Fig. 4. New move scheme if he bes search posiion of he swarm is no updaed for a long generaion. Fig. 3. Move wih loop. We show he resuls of he applicaion o quadraic ineger programming maximizing problem wih n = 50 and m = 5 in Table I. In hese experimens, we se he maximal search generaion number T max = 5000 and he swarm size N = 100. And he number of rial is 40. TABLE I mehod wih loop bes mean wors ime (sec) From Table I, incorporaing new move scheme wih loop proposed in his research, we can improve efficiency of rp- SONLIP. On he oher hand, he bes value of wih loop is inferior o and i urns worse on accuracy. B. diversificaion There is ofen ha he bes search posiion of he swarm is no updaed for a long generaion in PSO. If he bes search posiion of he swarm is no global opimal soluion, a paricle has o move from his search posiion. Kao e al. [1] incorporaed muliple sreching echnique o resrain o sopping around local opimal soluions. and we apply i in. However, for he problem ha he decision variable i = ω v i + c 1 R 1(p i x i) c 2 R 2(p g x i) (5) In his move scheme we can resrain o sopping around curren bes search posiion of he swarm. However, convergence o plural local opimal soluions may occur. Thus, we preven convergence o plural local opimal soluions by adding he penaly depending on disance from he local opimal soluions. To be more specific, for q local opimal soluions x k, k = 1,..., q, we consider he funcion S(x) as follows: d k (x) = x x k (6) P k (x) = { G dk (x) 1 0 d k (x) > 1 S(x) = f(x) + (7) q P k (x) (8) k=1 Here, d k (x) is disance beween curren search posiion and he local opimal soluion x k. P k (x) is he funcion adding he penaly if d k (x) 1, if no, no adding he penaly. S(x) is he new evaluaion value added summaion of he penaly o he objecive funcion value depending on he disance beween curren search posiion and q local opimal soluions. Using S(x), he evaluaion value includes he penaly in a cerain region from local opimal soluions and we can preven convergence o local opimal soluions. We show he resuls of he applicaion of wih loop and diversificaion o quadraic ineger programming maximizing problem wih n = 50 and m = 5 in Table II. In hese experimens, we se he maximal search generaion number T max = 5000 and he swarm size N = 100. And he number of rial is 40. From Table II, incorporaing modified new move scheme, efficiency and accuracy of are improved. In addiion, we can reduce compuaional ime more compared ha we apply muliple sreching echnique. 45

4 TABLE II mehod wih loop and divesificaion wih loop bes mean wors ime (sec) C. inensificaion For furher improvemen of efficiency and accuracy, inensificaion (reinforcemen of convergence search in promising region) is needed. In his research, we inroduce local search in he bes search posiion of he swarm since we regard he region around he bes search posiion of he swarm as he promising region. To be more concree, a paricle searches neighborhood in he bes search posiion of he swarm if he bes search posiion of he swarm is no updaed for a long generaion and moves he bes search posiion besides ha of he swarm (Fig. 5). However, visiaion occurs if he bes search posiion of he swarm jus before is he local opimal soluion. Thus, we inroduce he mehod o forbid his siuaion (abu). Fig. 5. :he bes search posiion of he swarm :he bes search posiion of he swarm sopping Local search in he bes search posiion of he swarm. We show he resuls of he applicaion of wih loop, diversificaion and inensificaion (rpsodinlip) o quadraic ineger programming maximizing problem wih n = 50 and m = 5 (same problem as Table I, II) in Table III. In hese experimens, we se he maximal search generaion number T max = 5000 and he swarm size N = 100. And he number of rial is 40. TABLE III rpsodinlip mehod (proposed) wih loop and diversificaion bes mean wors ime (sec) From Table III, he proposed rpsodinlip is beer han wih loop and diversificiaion wih respec o he bes objecive funcion value, he mean one. From hese resuls, efficiency and accuracy are improved more. D. The procedure of rpso combining diversificaion and inensificaion The procedure of he proposed rpso combininig diversificaion and inensificaion for NLIP (rpsodinlip) is summarized as follows. Sep 1: Find an ineger feasible soluion by PSO in consideraion of he degree of violaion of consrains, and use i as he basepoin of he homomorphous mapping, r. Le := 0 and go o Sep 2. Sep 2: Generae feasible iniial ineger search posiions based on he homomorphous mapping proposed by Koziel and Michalewicz [4]. To be more specific, map N poins generaed randomly in he n dimensional hypercube [ 1, 1] n o he feasible region X using he homomorphous mapping, and le hese poins in X be iniial search posiions x 0 i = 1,..., N. In addiion, le he iniial search posiion of each paricle, x 0 be he iniial bes posiion of he paricle in is rack, p 0 and le he bes posiion among x 0 i = 1,..., N be he iniial bes posiion of he swarm, p 0 g. Go o Sep 3. Sep 3: Calculae he value of ω by (4). For each paricle, using he informaion of p i and p g, deermine he direcion vecor i o he nex search posiion x +1 i by he modified move schemes proposed by Kao e al. [1]. Nex, move i o he nex search posiion by (3) and go o Sep 4. Sep 4: If he paricle does no move since he curren search posiion and he nex search posiion are he same eiher, revise o 1 or 1 depending on he plus and minus on he elemen ha he absolue value is maximum in he elemen of i before revising an ineger value. Go o Sep 5. Sep 5: Move wih loop from upper o lower or from lower o upper for he decision variable ha absolue value is maximum in he elemen of he search direcion vecor o boundary ouside if he decision variable fixed in he boundary (upper or lower bound). Go o Sep 6. Sep 6: Check if he curren search posiion of each paicle in i he subswarm wih repair based on he bisecion mehod, x +1 is feasible. If no, repair i o be feasible using he bisecion mehod, and go o Sep 7. Sep 7: Deermine wheher he new move scheme o resrain o sopping around local opimal soluions is applied or no. If i is applied, go o Sep 8. Oherwise, go o Sep 9. Sep 8: Deermine he direcion ecor i o he nex search posiion x +1 i by he modified new move scheme explained in secion IV-B. Nex, move i o he nex search posion by (3) and evaluae each paricle by he value of S( ) for x +1 i = 1,..., N. Go o Sep 10. Sep 9: Evaluae each paricle by he value of f( ) (objecive funcion) for x +1 i = 1,..., N. Go o Sep 10. Sep 10: If he evaluaion funcion value S(x +1 i ) or f(x +1 i ) is beer han he evaluaion funcion value for he bes search posiion of he paricle in is rack, p updae he bes search posiion of he paricle in is rack as p +1 i := x +1 i. If no, le p +1 i := p i and go o Sep 11. Sep 11: If he minimum of S(x +1 i ), i = 1,..., N or he minimum of f(x +1 i ), i = 1,..., N is beer han he 46

5 evaluaion funcion value for he curren bes search posiion of he swarm, p g, updae he bes search posiion of he swarm as p +1 g := x +1 i min. Oherwise, le p +1 g := p +1 g and go o Sep 12. Sep 12: Deermine wheher he local search is applied or no. If i is applied, go o Sep 13. Oherwise, go o Sep 14. Sep 13: Search neighborhood in he bes search posiion of he swarm and updae he bes search posiion of he swarm. Go o Sep 14. Sep 14: If he condiion of he secession is saisfied, apply he secession o every paricle according o a given probabiliy, and go o Sep 15. Sep 15: Finish if = T max (he maximal value of ime). Oherwise, le := + 1 and reurn o Sep 3. V. NUMERICAL EXAMPLES We apply he proposed PSO (rpsodinlip) and geneic algorihm for nonlinear ineger programming problems (GAN- LIP) [6] which is one of he exisiing efficien mehods, o wo nonlinear ineger programming maximizing problems wih differen scale. The number of rial is 40 for rpsodinlip and GANLIP. Tables IV and V show he resuls obained by boh mehods: he bes objecive funcion value of 40 rials, he mean one, he wors one, and he mean compuaional ime. In hese experimens, he parameers of GANLIP are se as he populaion size N = 100. On he oher hand, he pareameers of rpsodinlip are se as he size of he swarm N = 100. And we se he maximal search generaion number T max = 5000 for all problems. For he problem, as shown in Table IV, he proposed rpsodinlip can always obain he opimal value (0.942), while GANLIP cannno; and he mean compuaional ime of rpsodinlip is shorer han ha of GANLIP. For he problem, as shown in Table V, he proposed rpsodinlip is beer han GANLIP wih respec o he bes objecive funcion value, he mean one, he wors one, and he mean compuaional ime. From hese resuls, i is indicaed ha he proposed rp- SODINLIP is superior o GANLIP and rpsodinlip is promising as an opimizaion mehod for nonlinear ineger programming problems. TABLE V RESULTS OF THE APPLICATION TO THE PROBLEM WITH n = 3 AND m = 3 mehod rpsodinlip GANLIP bes mean wors ime (sec) and reinforcemen of convergence search in promising region (inensificaion). We showed he efficiency of he proposed rp- SODINLIP mehod by comparing i wih an exising mehod, GANLIP, hrough heir applicaion in some numerical examples. ACKNOWLEDGMENT This research was parially suppored by The 21s Cenury COE Program on Hyper Human Technology oward he 21s Cenury Indusrial Revoluion. REFERENCES [1] K. Kao, T. Masui, M. Sakawa and K. Morihara, An approximae soluion mehod based on paricle swarm opimizaion for nonlinear programming problems, Journal of Japan Sociey for Fuzzy Theory and Inelligen Informaics, Vol.20, No.3, pp , [2] J. Kennedy and R.C. Eberhar, Paricle swarm opimizaion, Proceedings of IEEE Inernaional Conference on Neural Neworks, pp , [3] J. Kennedy and W. M. Spears, Maching algorihms o problems: an experimenal es of he paricle swarm and some geneic algorihms on he mulimodal problem generaor, Proceedings of IEEE Inernaional Conference on Evoluionary Compuaion, pp.74 77, [4] S. Koziel and Z. Michalewicz, Evoluionary algorihms, homomorphous mappings, and consrained parameer opimizaion, Evoluionary Compuaion, Vol.7, No.1, pp.19 44, [5] T. Masui, K. Kao, M. Sakawa, T. Uno and K. Masumoo, Paricle swarm opimizaion for nonlinear ineger programming problems, Proceedings of Inernaional MuliConference of Engineers and Compuer Scieniss 2008, pp , [6] M. Sakawa, K. Kao, M.A.K. Azad and R. Waanabe, A geneic algorihm wih double sring for nonlinear ineger programming problems, IEEE SMC 2005 Conference Proceedings, pp , [7] Y.H. Shi and R.C. Eberhar, A modified paricle swarm opimizer, Proceedings of IEEE Inernaional Conference on Evoluionary Compuaion, pp , TABLE IV RESULTS OF THE APPLICATION TO THE PROBLEM WITH n = 20 AND m = 3 mehod rpsodinlip GANLIP bes mean wors ime (sec) VI. CONCLUSION In his research, focusing on a paricle swarm opimizaion, we considered is applicaion o NLIP. In order o improve accuracy of, we incorporaed new move scheme wih loop, reinforcemen of global search (diversificaion) 47