A Novel Genetic Algorithm using Helper Objectives for the 0-1 Knapsack Problem

Transcription

1 A Novel Geetic Algorithm usig Helper Objectives for the 0-1 Kapsack Problem Ju He, Feidu He ad Hogbi Dog 1 arxiv: v1 [cs.ne] 3 Apr 2014 Abstract The 0-1 kapsack problem is a well-kow combiatorial optimisatio problem. Approximatio algorithms have bee desiged for solvig it ad they retur provably good solutios withi polyomial time. O the other had, geetic algorithms are well suited for solvig the kapsack problem ad they fid reasoably good solutios quickly. A aturally arisig questio is whether geetic algorithms are able to fid solutios as good as approximatio algorithms do. This paper presets a ovel multi-objective optimisatio geetic algorithm for solvig the 0-1 kapsack problem. Experimet results show that the ew algorithm outperforms its rivals, the greedy algorithm, mixed strategy geetic algorithm, ad greedy algorithm + mixed strategy geetic algorithm. Idex Terms geetic algorithm, kapsack problem, multi-objective optimisatio, solutio quality I. INTRODUCTION The 0-1 kapsack problem is oe of the most importat ad also most itesively studied combiatorial optimisatio problems [1]. Several approximatio algorithms have proposed for solvig the 0-1 kapsack problem [1]. These algorithms always ca retur provably good solutios, whose values are withi a factor of the value of the optimal solutio. I last two decades, evolutioary algorithm, especially geetic algorithms (GAs), have bee well adopted for tacklig the kapsack problem [2], [3]. The problem has received a particular iterest from the evolutioary computatio commuity for the followig reaso. The biary vector represetatio is a atural ecodig of of the cadidate solutios to the 0-1 kapsack problem. Thereby, it provides a ideal settig for the applicatios of GAs [4, Chapter 4]. Empirical results ofte assert that GAs produce reasoably good solutios to the kapsack problems [5], [6], [7]. A aturally arisig questio is to compare the solutio quality (reasoably good versus provably good) betwee GAs ad approximatio algorithms. There are two approaches to aswer the questio. Oe approach is to make a theoretical aalysis. A GA is prove that it ca produce a solutio withi a polyomial rutime, the value of which is withi a factor of the value of a optimal solutio. This is a stadard approach used i the study of approximatio algorithms. Aother approach is to coduct a empirical study. A GA is compared with a approximatio algorithm via computer experimets. If the GA ca produce solutios better or ot worse tha a approximative algorithm does i all istaces withi polyomial time, the GA is able to reach the same solutio quality as the approximatio algorithm does. The curret paper is a empirical study of a GA which uses the multi-objectivizatio techique [8]. I multi-objectivizatio, sigle-objective optimisatio problems are trasferred ito multi-objective optimisatio problems by decomposig the origial objective ito several compoets [8] or by addig helper objectives [9]. Multi-objectivizatio may brig both positive ad egative effects [10], [11], [12]. This approach has bee used for solvig several combiatorial optimisatio problems, for example, the kapsack problem [13], vertex cover problem [14] ad miimum label spaig tree problem [15]. This paper focusses o the 0-1 kapsack problem. A ovel GA usig three helper objectives is desiged for solvig the 0-1 kapsack problem. The the solutio quality of the GA is compared with a well-kow approximatio algorithm via computer experimets. The remaider of the paper is orgaized as follows. The 0-1 kapsack problem, a greedy algorithm ad a GA for it are itroduced i Sectio II. I Sectio III we preset a ovel GA usig helper objectives. Sectio IV is devoted to a empirical compariso amog several algorithms. Sectio V cocludes the article. II. KNAPSACK PROBLEM, GREEDY ALGORITHM AND GENETIC ALGORITHM I a istace of the 0-1 kapsack problem, give a set of items with weights w i ad profits p i, ad a kapsack with capacity C, the task is to maximise the sum of profits of items packed i the kapsack without exceedig the capacity. More formally the target is to fid a biary vector x = (x 1 x ) so as to max f( x) = p i x i, subject to w i x i C, (1) x This work was supported by the EPSRC uder Grat No. EP/I009809/1. Ju He is with Departmet of Computer Sciece, Aberystwyth Uiversity, Aberystwyth, SY23 3DB, UK ( ju.he@aber.ac.uk). Feidu He is with School of Iformatio Sciece ad Techology, Southwest Jiaotog Uiversity, Chegdu, Chia Hogbi Dog is with College of Computer Sciece ad Techology, Harbi Egieerig Uiversity, Harbi, Chia

2 2 where x i = 1 if the item i is selected i the kapsack ad x i = 0 if the item i is ot selected i the kapsack. A feasible solutio is a x which satisfies the costrait. A ifeasible oe is a x that violates the costrait. Several approximatio algorithms have bee proposed for solvig the 0-1 kapsack problem (see [1, Chapter 2] for more details). Amog these, the simplest oe is the greedy algorithm described below. The algorithm aims at puttig the most profitable items as may as possible ito the kapsack or the items with the highest profit-to-weight ratio as may as possible, withi the kapsack capacity. 1: iput a istace of the 0-1 kapsack problem; 2: resort all the items via the ratio of their profits to their correspodig weights so that p1 w 1 p w ; 3: greedily add the items i the above order to the kapsack as log as addig a item to the kapsack does ot exceedig the capacity of the kapsack. Deote the solutio by y; 4: resort all the items accordig to their profits so that p 1 p 2 p ; 5: greedily add the items i the above order as log as addig a item to the kapsack does ot exceedig the capacity of the kapsack. Deote the solutio by z; 6: output the best of y ad z. This algorithm is a 1/2-approximatio algorithm for the 0-1 kapsack problem [1, Sectio 2.4], which meas it always ca retur a solutio o worse tha 1/2 of the value of the optimal solutio. The greedy algorithm stops after fidig a approximatio solutio, ad it has o ability to seek the global optimal solutio. Therefore GAs are ofte applied for solvig the 0-1 kapsack problem. I order to hadle the costrait i the kapsack problem, we use repair methods sice they are claimed to be the most efficiet for the kapsack problem [4], [16]. A repair method is explaied as follows. 1: iput a ifeasible solutio x; 2: while x is ifeasible do 3: i =: choose a item from the kapsack; 4: set x i = 0; 5: if x iw i C the 6: x is feasible; 7: ed if 8: ed while 9: output a feasible solutio x. There are differet methods available for choosig a item i the repair procedure, described as follows. 1) Profit-greedy repair: sort all items accordig to the decreasig order of their correspodig profits. The choose the item with the smallest profit ad remove it from the kapsack. 2) Ratio-greedy repair: sort all items accordig to the decreasig order of the correspodig ratios. The choose the item with the smallest ratio ad remove it from the kapsack. 3) Radom repair: choose a item from the kapsack at radom ad remove it from the kapsack. Thaks to the repair method, all of the ifeasible solutios are repaired ito the feasible oes. The followig pseudo-code is a mixed strategy GA (MSGA) which chooses oe of three repair methods i a probabilistic way ad the applies the repair method to geerate a feasible solutio. 1: iput a istace of the 0-1 kapsack problem; 2: iitialize populatio Φ 0 cosistig of N feasible solutios; 3: for t = 0,1,,t max do 4: geerate a radom umber r i [0,1]; 5: if r < 0.9 the 6: childre populatio Φ t.c bitwise-mutate Φ t ; 7: else 8: childre populatio Φ t.c oe-poit crossover Φ t ; 9: ed if 10: if a child is a ifeasible solutio the 11: choose oe method from the ratio-greedy repair, radom repair ad value-greedy repair with probability 1/3 ad repair the child ito feasible; 12: ed if 13: the best idividual i the paret ad childre populatios is selected ito populatio Φ t+1 ; 14: N 1 idividuals from the paret ad childre populatios ito populatio Φ t+1 by roulette wheel selectio; 15: ed for 16: output the maximum of the fitess fuctio. The geetic operators used i the above GA are explaied below. Bitwise Mutatio: Give a biary vector (x 1 x ), flip each bit x i with probability 1/.

3 3 Oe-Poit Crossover: Give two biary vectors (x 1 x ) ad (y 1 y ), radomly choose a crossover poit k {1,,}, swap their bits at poit k. The geerate two ew biary vectors as follows, (x 1 x k y k+1 y ), (y 1 y k x k+1 x ). Like most of GAs, the MSGA may fid reasoably good solutios but has o guaratee about the solutio quality. Thus it is ecessary to desig evolutioary approximatio algorithms with provably good solutio quality. The most straightforward approach is that we first apply the greedy algorithm for geeratig approximatio solutios ad the take these solutios as the startig poit of the MSGA. We call this approach greedy algorithm + MSGA. Sice the MSGA starts at local optima, it becomes hard for the MSGA to leave the absorbig basi of this local optimum for seekig the global optimum. This is the mai drawback of the approach. III. GENETIC ALGORITHM USING HELPER OBJECTIVES FOR THE 0-1 KNAPSACK PROBLEMS I this sectio, we propose a ovel multi-objective optimisatio GA (MOGA) which ca beat the combiatio of greedy algorithm + MOGA metioed i the previous sectio. The algorithm is based o the multi-objectivizatio techique. The origial sigle objective optimizatio problem (1) is recast ito a multi-objective optimizatio problem usig helper objectives. The desig of helper objectives depeds o problem-specific kowledge. The first helper objective comes from a observatio o the followig istace. Item Profit Weight Capacity 20 The global optimum i this istace is I the optimal solutio, the average profit of packed items is the largest. Thus the first helper objective is to maximize the average profit of items i a kapsack. The objective fuctio is h 1 ( x) = 1 x i p i. (2) x 1 where x 1 = x i. The secod objective is ispired from a observatio o aother istace. Item Profit Weight Capacity 20 The global optimum i this istace is I the optimal solutio, the average profit-to-weight ratio of packed items is the largest. However, the average profit of these items is ot the largest. The the secod helper objective is to maximize the average profit-to-weight ratio of items i a kapsack. The objective fuctio is Fially let s look at the followig istace. h 2 ( x) = 1 x 1 Item Profit Weight Capacity 120 x i p i w i. (3) It is ot difficult to verify that the global optimum i this istace is I the optimal solutio, either the average profit of packed items or average profit-to-weight ratio is the largest, but the umber of packed items is the largest. Thus the third helper objective is to maximize the umber of items i a kapsack. The objective fuctio is We the come to the followig multi-objective optimizatio problem: max x {f( x),h 1( x),h 2 ( x),h 3 ( x)}, h 3 ( x) = x 1. (4) subject to w i x i C. (5)

4 4 Besides the above three helper objectives, it is possible to add more helper objectives, for example, to miimise the average weight of packed items. The multi-objective optimisatio problem (5) is solved by a MOGA usig bitwise mutatio, oe-poit crossover ad multicriteria selectio, plus a mixed strategy of three repair methods. 1: iput a istace of the 0-1 kapsack problem; 2: iitialize Φ 0 cosistig of N feasible solutios; 3: for t = 0,1,,t max do 4: geerate a radom umber r i [0,1]; 5: if r < 0.9 the 6: childre populatio Φ t.c bitwise mutate Φ t ; 7: else 8: childre populatio Φ t.c oe-poit crossover Φ t ; 9: ed if 10: if ay child is a ifeasible solutio the 11: choose oe repair method from the ratio-greedy repair, radom repair ad value-greedy repair with probability 1/3; 12: repair the child ito a feasible solutio; 13: ed if 14: populatio Φ t+1 multi-criterio select N idividuals from Φ t ad Φ t.c ; 15: ed for 16: output the maximum of f( x) i the fial populatio. The multi-criteria selectio operator, adopted i the above MOGA, is ovel ad ispired from multi-objective optimisatio. Sice the target is to maximise several objectives simultaeously, we select idividuals which have higher fuctio values with respect to each objective fuctio. The pseudo-code of multi-criteria selectio is described as follows. 1: iput the paret populatio Φ t ad child populatio Φ t.c ; 2: merge the paret ad childre populatios ito a temporary populatio which cosists of 2N idividuals; 3: sort all idividuals i the temporary populatios i the descedig order of f( x), deote them by x (1) 1,, x(1) 2N ; 4: select all idividuals from left to right (deote them by x (1) k 1,, x (1) k m ) which satisfy h 1 ( x (1) k i ) < h 1 ( x (1) k i+1 ) or h 2 ( x (1) k i ) < h 2 ( x (1) k i+1 ) for ay k i. 5: if the umber of selected idividuals is greater tha m N 3 the 6: trucate them to N 3 idividuals; 7: ed if 8: add these selected idividuals ito the ext populatio Φ t+1 ; 9: resort all idividuals i the temporary populatio i the descedig order of h 1 ( x), still deote them by x 1,, x 2N ; 10: select all idividuals from left to right (still deote them by x k1,, x km ) which satisfy h 3 ( x ki ) < h 3 ( x ki+1 ) for ay k i. 11: if the umber of selected idividuals is greater tha N 3 the 12: trucate them to N 3 idividuals; 13: ed if 14: add these selected idividuals ito the ext populatio Φ t+1 ; 15: resort all idividuals i the temporary populatios i the descedig order of h 2 ( x), still deote them by x 1,, x 2N ; 16: select all idividuals from left to right (still deote them by x k1,, x km ) which satisfy h 3 ( x ki ) < h 3 ( x ki+1 ) for ay k i. 17: if the umber of selected idividuals is greater tha N 3 the 18: trucate them to N 3 idividuals; 19: ed if 20: add these selected idividuals ito the ext populatio Φ t+1 ; 21: while the populatio size of Φ t+1 is less tha N do 22: radomly choose a idividual from the paret populatio ad add it ito Φ t+1 ; 23: ed while 24: output a ew populatio Φ t+1. I the above algorithm, Steps 3-4 are for selectig the idividuals with higher values of f( x). I order to preserve diversity, we choose these idividuals which have differet values of h 1 ( x) or h 2 ( x). Similarly Steps 9-10 are for selectig the idividuals with a higher value of h 1 ( x). We choose the idividuals which have differet values of h 3 ( x) for maitaiig diversity. Steps are for selectig idividuals with a higher value of h 2 ( x). Agai we choose these idividuals which have differet values of h 3 ( x) for preservig diversity. We do t explicitly select idividuals based o h 3 ( x). Istead we implicitly do it durig Steps 9-10, ad Steps Steps 5-7, Steps 11-13, Steps 17-19, plus Steps are used to maitai a ivariat populatio size N.

5 5 The beefit of usig multi-criterio selectio is its ability of makig search alog differet directios f( x),h 1 ( x), h 2 ( x) ad implicitly h 3 ( x). Hece the MOGA may ot get trapped ito the absorbig area of a local optimum. IV. EXPERIMENTS I this sectio, we implemet computer experimets. Accordig to [1], [4], the istaces of the 0-1 kapsack problem are ofte classified ito two categorises. 1) Restrictive capacity kapsack: the kapsack capacity is so small that oly a few items ca be packed i the kapsack. A istace with restrictive capacity kapsack is geerated i the followig way. Choose a parameter B which is a upper boud o the weight of each item. I the experimets, set B =. For item i, its profit p i ad weight w i are geerated at uiformly radom i [1,B]. Set the capacity of the kapsack C = B. 2) Average capacity kapsack: the kapsack capacity is so large that it is possible to pack half of items ito the kapsack. A istace with average capacity kapsack is geerated as follows. Choose a parameter B which is the upper boud o the weight of each item. I the experimets, set B =. For item i, its profit p i ad weight w i are geerated at uiformly radom i [1,B]. Sice the average weight of each item is 0.5B, thus the average of the total weight of items is 0.5B. So we set the capacity to be the half of the total weight, that is C = 0.25B. For each type of the 0-1 kapsack problem, 10 istaces are geerated at radom. For each istace, the umber of items is 100. The populatio size is 3. The umber of maximum geeratios is 30 for the MSGA ad 10 for the MOGA. All idividuals i the iitial populatio are geerated at radom. If a idividual is a ifeasible solutio, it is repaired to feasible usig radom repair. Besides the above radomly geerated istaces, we also cosider two special istaces. Special istace I is give Table I. Item i TABLE I SPECIAL INSTANCE I: = 500 AND α = 0.2 Profit p i 1 Weight w i 1 Capacity 1,, 1+α 1+α +1 1+α +2,, α 1+α 1+α α 4+4α 1+α Iitialisatio 0 1 half bits are 1 Special istace II is give i Table II. Item i 1,, 4 TABLE II SPECIAL INSTANCE II: = ,, 2 Profit p i Weight w i Capacity ,, Iitialisatio oe bit is 1, others 0 0 half bits are 1 The populatio size is for Istaces I ad II. The umber of maximum geeratios is 15 for the MSGA ad 5 for the MOGA. The iitialisatio of idividuals i both MSGA ad MOGA refer to the above tables. Tables III gives experimet results of comparig the greedy algorithm, MSGA, greedy algorithm + MSGA ad MOGA. From the table, we observe that the solutio quality of MSGA is better or ot worse tha the greedy algorithm i 20 radom istaces. However for Istace I, the MSGA oly fids a solutio whose value is about 20% of the optimal value. the solutio quality of greedy algorithm + MSGA is better or ot worse tha the greedy algorithm i all istaces. However for Istace II, the algorithm gets trapped ito a local optimum, ad is worse tha the MOGA. the MOGA is the wier amog 4 algorithms ad its the solutio quality is better or ot worse tha the greedy algorithm ad MSGA i all istaces. V. CONCLUSIONS A ovel MOGA usig helper objectives is proposed i this paper for solvig the 0-1 kapsack problem. First the origial 0-1 kapsack problem is recast ito a multi-objective optimizatio problem (i.e. to maximize the sum of profits packed i the

6 6 TABLE III A COMPARISON AMONG 4 ALGORITHMS IN 20 RANDOMLY GENERATED INSTANCES AND 2 SPECIAL INSTANCES. THE FIRST 10 INSTANCES BELONG TO THE RESTRICTIVE CAPACITY KNAPSACK PROBLEM. THE SECOND 10 INSTANCES BELONG TO THE AVERAGE CAPACITY KNAPSACK PROBLEM. MAX : THE MAXIMUM VALUE OF f( x) PRODUCED DURING 10 RUNS. AVERAGE : THE AVERAGE VALUE OF f( x) OVER 10 RUNS. STDEV : THE STANDARD DERIVATION OF f( x) IN 10 RUNS. Greedy MSGA Greedy + MSGA MOGA Istace max average stdev max average stdev max average stdev I II kapsack, to maximize the average profit-to-weight ratio of items, to maximize the average profit of items, ad to maximize the umber of packed items). The a MOGA (usig bitwise mutatio, oe-poit crossover ad multi-criterio selectio plus a mixed strategy of three repair methods) is desiged for the multi-objective optimizatio problem. Experimet results demostrate that the MOGA usig helper objectives outperforms its rivals, which are the greedy algorithm, MSGA ad greedy algorithm + MSGA. The results also show that the MSGA ca fid reasoably good solutios but without a guaratee; ad the greedy algorithm + MSGA sometimes gets trapped ito a local optimum. REFERENCES [1] S. Martello ad P. Toth, Kapsack Problems. Chichester: Joh Wiley & Sos, [2] Z. Michalewicz ad J. Arabas, Geetic algorithms for the 0/1 kapsack problem, i Methodologies for Itelliget Systems. Spriger, 1994, pp [3] S. Khuri, T. Bäck, ad J. Heitkötter, The zero/oe multiple kapsack problem ad geetic algorithms, i Proceedigs of the 1994 ACM Symposium o Applied Computig. ACM, 1994, pp [4] Z. Michalewicz, Geetic Algorithms + Data Structures = Evolutio Programs, 3rd ed. New York: Spriger Verlag, [5] E. Zitzler ad L. Thiele, Multiobjective evolutioary algorithms: A comparative case study ad the stregth pareto approach, IEEE Trasactios o Evolutioary Computatio, vol. 3, o. 4, pp , [6] A. Jaszkiewicz, O the performace of multiple-objective geetic local search o the 0/1 kapsack problem-a comparative experimet, IEEE Trasactios o Evolutioary Computatio, vol. 6, o. 4, pp , [7] M. Eugéia Captivo, J. Clìmaco, J. Figueira, E. Martis, ad J. Luis Satos, Solvig bicriteria 0 1 kapsack problems usig a labelig algorithm, Computers & Operatios Research, vol. 30, o. 12, pp , [8] J. D. Kowles, R. A. Watso, ad D. W. Core, Reducig local optima i sigle-objective problems by multi-objectivizatio, i Evolutioary Multi- Criterio Optimizatio. Spriger, 2001, pp [9] M. T. Jese, Helper-objectives: Usig multi-objective evolutioary algorithms for sigle-objective optimisatio, Joural of Mathematical Modellig ad Algorithms, vol. 3, o. 4, pp , [10] J. Hadl, S. C. Lovell, ad J. Kowles, Multiobjectivizatio by decompositio of scalar cost fuctios, i Parallel Problem Solvig from Nature PPSN X. Spriger, 2008, pp [11] D. Brockhoff, T. Friedrich, N. Hebbighaus, C. Klei, F. Neuma, ad E. Zitzler, O the effects of addig objectives to plateau fuctios, IEEE Trasactios o Evolutioary Computatio, vol. 13, o. 3, pp , [12] D. F. Lochtefeld ad F. W. Ciarallo, Helper-objective optimizatio strategies for the job-shop schedulig problem, Applied Soft Computig, vol. 11, o. 6, pp , [13] R. Kumar ad N. Baerjee, Aalysis of a multiobjective evolutioary algorithm o the 0 1 kapsack problem, Theoretical Computer Sciece, vol. 358, o. 1, pp , [14] T. Friedrich, J. He, N. Hebbighaus, F. Neuma, ad C. Witt, Approximatig coverig problems by radomized search heuristics usig multi-objective models, Evolutioary Computatio, vol. 18, o. 4, pp , [15] X. Lai, Y. Zhou, J. He, ad J. Zhag, Performace aalysis of evolutioary algorithms for the miimum label spaig tree problem, IEEE Trasactios o Evolutioary Computatio, 2014, (accpeted, olie). [16] J. He ad Y. Zhou, A compariso of GAs usig pealizig ifeasible solutios ad repairig ifeasible solutios II, i Proceedigs of the 2d Iteratioal Symposium o Itelligece Computatio ad Applicatios. Wuha, Chia: Spriger, 2007, C1, pp