IN RECENT YEARS, the number of publications on evolutionary

Transcription

1 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE On the Effets of Adding Ojetives to Plteu Funtions Dimo Brokhoff, Tois Friedrih, Nils Heinghus, Christin Klein, Frnk Neumnn, nd Ekrt Zitzler Astrt In this pper, we exmine how dding ojetives to given optimiztion prolem ffets the omputtionl effort required to generte the set of Preto-optiml solutions. Experimentl studies show tht dditionl ojetives my hnge the running time ehvior of n lgorithm drstilly. Often it is ssumed tht more ojetives mke prolem hrder s the numer of different trdeoffs my inrese with the prolem dimension. We show tht dditionl ojetives, however, my e oth enefiil nd ostrutive depending on the hosen ojetive. Our results re otined y rigorous running time nlyses tht show the different effets of dding ojetives to wellknown plteu funtion. Additionl experiments show tht the theoretilly shown ehvior n e oserved for prolems with more thn one ojetive. Index Terms Multiojetive optimiztion, running time nlysis, theory. I. MOTIVATION IN RECENT YEARS, the numer of pulitions on evolutionry multiojetive optimiztion hs een rpidly growing; however, most of the studies investigte prolems where the numer of onsidered ojetives is low, i.e., etween two nd four, while studies with mny ojetives re rre [5]. The reson is tht lrge numer of ojetives leds to further diffiulties with respet to deision mking, visuliztion, nd omputtion. Nevertheless, from prtil point of view it is desirle with most pplitions to inlude s mny ojetives s possile without the need to speify preferenes mong the different riteri. An open question in this ontext is how the inlusion of dditionl ojetives ffets the serh effiieny of n evolutionry lgorithm to generte the set of Pretooptiml solutions. There is some evidene in the literture tht dditionl ojetives n mke prolem hrder. Winkler [31] proved Mnusript reeived Ferury 23, 2007; revised June 30, 2008 nd Septemer 17, 2008; epted Septemer 29, Current version pulished June 10, D. Brokhoff hs een supported y the Swiss Ntionl Siene Foundtion under Grnt T. Friedrih hs een supported y postdotorl fellowship from the Germn Ademi Exhnge Servie (DAAD). D. Brokhoff nd E. Zitzler re with the Computer Engineering nd Networks Lortory, Elgenössihe Tehnishe Hohshule Zurih, 8092 Zurih, Switzerlnd (e-mil: rokho@tik.ee.ethz.h; zitzler@tik.ee.ethz.h). T. Friedrih is with the Algorithm Group t the Interntionl Computer Siene Institute, Berkeley, CA 94704, USA (e-mil: Tois@ICSI.Berkeley.edu). N. Heinghus, C. Klein, nd F. Neumnn re with the Deprtment 1: Algorithms nd Complexity t Mx-Plnk-Institut für Informtik, Srrüken, Germny (e-mil: nils.heinghus@mpi-inf.mpg.de; hristin.klein@mpi-inf.mpg.de; frnk.neumnn@mpi-inf.mpg.de). Digitl Ojet Identifier /TEVC /$ IEEE tht the numer of inomprle solutions inreses if further rndomly generted ojetives re dded. Therefore, on the one hnd the Preto-optiml front my eome lrger nd, on the other hnd, the power of the dominne reltion to guide the serh my diminish these re the min rguments tht vrious reserhers, e.g., [5], [7], [10], [11], [14], [27], list in fvor of the ssumption tht the serh eomes hrder more ojetives re involved. Tht, in ft, stte-ofthe-rt evolutionry lgorithms like NSGA-II nd SPEA2 hve prolems to find good pproximtion of the Preto-optiml front for seleted test prolems ws empirilly shown in [30] nd [27]. Furthermore, the investigtions of Purshouse nd Fleming [27] show tht the ehvior of multiojetive evolutionry lgorithm on prolem with few ojetives nnot e generlized to lrger numer of ojetives. In ontrst, few pulitions point out tht reformulting prolem in terms of more ojetive funtions n redue the omputtionl ost of the optimiztion proess. For exmple, Jensen [17] suessfully used dditionl helperojetives to guide the serh of evolutionry lgorithms in high-dimensionl spes. A similr pproh ws proposed y Knowles et l. [18], where single-ojetive prolems re multiojetivized, i.e., deomposed into multiojetive prolems whih re esier to solve thn the originl prolems. Also the ide of turning onstrints of single-ojetive prolems into dditionl ojetives hs een shown to redue optimiztion ost until good solutions re found [21], lthough Runrsson nd Yo [28] pointed out tht this is not effetive on ll kinds of prolems due to wrong serh is. Besides these empirilly oriented studies, there re theoretil results supporting the hypothesis tht multiojetiviztion n help. Shrnow et l. [29] showed tht the single soure shortest pth prolem is esier to solve for simple evolutionry lgorithms (EAs) when formulted s i-riterion prolem; Neumnn nd Wegener [25] proved for the minimum spnning tree prolem tht formultion with two ojetives leds to lower running time omplexity of simple EAs thn the originl singleojetive version. This disussion indites tht generl sttement on the effet of inresing the numer of ojetives is not possile. For some prolems, with higher numer of ojetives it is more diffiult to generte the Preto-optiml front; for other prolems, it is esier. However, given the previous work, the question rises whether one nd the sme prolem n e mde oth esier nd hrder depending on the dded ojetive. This pper nswers this question oth experimentlly nd

2 592 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 theoretilly. Bsed on simple multiojetive optimizer, nmely SEMO, whih is known from vrious theoretil nlyses, we show: 1) y mens of running time nlyses tht one nd the sme prolem n eome oth hrder nd esier to solve if different ojetives re dded in ontrst to [29] nd [25] where the originl ojetive is repled y two other ojetives, we here onsider the se tht the originl ojetive remins in the ojetive set; 2) for two eqully diffiult single-ojetive funtions tht the omintion of the two yields i-riterion prolem tht is esier to solve thn either of the two singleojetive prolems; 3) nd experimentlly tht prolems with more ojetives exist tht n lso e mde oth hrder nd esier, only depending on the type of the dded ojetive. The min oservtion ehind oth the running time nlyses nd the experimentl studies is tht prolems my ontin solled plteus. A plteu is prt of the serh spe where the prolem does not indite ny serh diretion. As we will show in the reminder of this pper, n dditionl ojetive n remove or introdue those plteus. Sine n EA, e it single- or multiojetive one, performs rndom wlk on these plteus s ws shown for some of the well-known omintoril optimiztion prolems [13], [26], [32], the removl or introdution of plteus y dding ojetives n hnge the running time ehvior of EAs drstilly. Depending on whether the dditionl ojetive introdues the right or deeptive serh diretion on former plteu, or good or deeptive diretion is eliminted y introduing plteu, the prolem eomes hrder or esier to solve for n EA. This pper extends its onferene version [2] in severl wys. On one hnd, the nlyses hve een improved nd dditionl lower ounds on the running time ehvior of the onsidered multiojetive evolutionry lgorithms re presented. The new lower ounds mke the nlyses tight, s ll of them mth with the proven symptoti upper ounds. On the other hnd, the effet of dding ojetives is investigted for prolems with more thn two ojetives y rrying out experimentl studies. The pper is orgnized s follows. First, we review si onepts suh s reltion grphs nd ojetive onflits nd disuss how dditionl ojetives n ffet the dominne struture (Setion II). In Setion III, we detil the lgorithms onsidered in this pper nd define the setting for the running time nlyses to follow. Setion IV provides the proofs showing tht simple plteu funtion n eome either hrder or esier with n dditionl ojetive; Setion V extends these results nd demonstrtes tht even the omintion of two eqully diffiult single-ojetive funtions n yield n esier i-riterion prolem. Tht n dditionl ojetive n even mke prolems with more thn one ojetive either hrder or esier is shown experimentlly in Setion VI. Conlusions re presented in Setion VII. II. ADDING OBJECTIVES: FOUNDATIONS AND EFFECTS Without loss of generlity, we onsider mximiztion prolems with k ojetive funtions f i : X R, 1 i k, where f 1 f 2 f () vlues f 1 f 2 f 3 () ojetives Fig. 1. () Ojetive vlues nd () orresponding prllel oordintes plot for three solutions,, X. the vetor funtion f := ( f 1,..., f k ) mps eh solution x X to n ojetive vetor f (x) R k. Furthermore, we ssume tht the underlying dominne struture is given y the wek Preto-dominne reltion whih is defined s follows: F := {(x, y) X 2 f i F : f i (x) f i (y)}, where F is set of ojetives with F F :={f 1,..., f k }. We sy x wekly domintes y w. r. t. the ojetive set F (x F y) if (x, y) F nd distinguish etween the following three ses: 1) the solution pir x, y is lled omprle if x wekly domintes y nd/or y wekly domintes x; 2) two solutions x, y re inomprle if neither wekly domintes the other one; 3) two solutions hving the sme ojetive vetor re lled indifferent. A solution x X is lled Preto-optiml if every x X is either indifferent to x or does not wekly dominte x w. r. t. the set of ll ojetives. The set of ll Preto-optiml solutions is lled Preto (optiml) set, its imge in the ojetive spe is lled Preto front. Given these si terms, we will now illustrte on the sis of simple exmple wht hppens if ojetives re dded. To this end, we repitulte some onepts introdued in [3], [4]. Assume the serh spe X onsists of three solutions,, nd nd F onsists of three ojetive funtions f 1, f 2, nd f 3. In Fig. 1, the ojetive funtions re shown nd the solutions re depited in prllel oordintes plot. To see wht hppens when merging, e.g., the ojetives f 1 nd f 2 into i-riterion prolem, the visuliztion of the dominne reltions f1, f2, nd f1 f 2 s reltion grphs is useful. In suh reltion grph, eh solution orresponds to vertex nd direted edge from vertex v to vertex w is drwn iff v F w. Fig. 2 shows the reltion grphs for f1, f2, f3, nd their orresponding omintions. With single ojetive only, the three solutions re pirwise omprle, see Fig. 2() (). When merging f 1 nd f 2 to i-riterion prolem, omprilities dispper, Fig. 2(d). For exmple, the edge etween nd is not present in the resulting reltion grph of {f1, f 2 }. Beuse f 1 () > f 1 (), solution wekly domintes with respet to f 1 ut does not wekly dominte. With respet to f 2, solution wekly domintes euse f 2 () > f 2 (). When tking oth f 1 nd f 2 into ount, neither does wekly dominte nor does wekly dominte y definition of ; the solution pir (, ) is inomprle, i.e., no edge etween nd is drwn in {f1, f 2 }.Thesme holds for the solution pir (, ).

3 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS 593 () reltion grph of () reltion grph of () reltion grph of {f 1 } {f 2 } {f 3 } (d) reltion grph of (e) reltion grph of (f) reltion grph of {f 2, f 3 } {f 1, f 3 } = {f 1, f 2, f 3 } {f 1, f 2 } Fig. 2. Reltion grphs for the three solutions,, nd nd different ojetive susets. Wht hppens now if f 3 is dded to the i-riterion prolem, desried y f 1 nd f 2? The omprle solutions nd eome lso inomprle euse f 3 () > f 3 (), ut wekly domintes with respet to {f 1, f 2 }, i.e., f 1 () > f 1 () nd f 2 () > f 2 (). The edge etween nd is lso removed in the reltion grph of {f1, f 2, f 3 }, Fig. 2(f). We oserve tht dditionl ojetives result in the disppering of edges in the reltion grphs new edges nnot pper if ojetives re dded. On the one hnd, if solution pir is omprle with respet to ll ojetives, i.e., n edge is drwn, the two solutions re omprle with respet to ny suset of ojetives nd the edge is lredy inluded in the reltion grphs for ll ojetive susets. On the other hnd, if solution x is etter thn solution y with respet to the ojetives in F 1, i.e., n edge in F1 is only drwn from x to y ut not the other wy round, nd y is etter thn solution x with respet to the ojetive set F 2, i.e., (y, x) F2,ut (y, x) F1, the solution pir eomes inomprle with respet to F 1 F 2 ; the edges etween x nd y dispper in F1 F 2. Considering our exmple, the edges of the new reltion grphs n lwys e derived from the reltion grphs of the smller ojetive sets: n edge is drwn iff the edge is present in the reltion grphs of oth ojetive susets; the new edge set is the intersetion of the previous edge sets. This oservtion n e summrized s follows, see [3] for detils. Let F F ={f 1,..., f k } e set of ojetive funtions. Then i F i= F. Bsed on this result, one n define two sets of ojetives s onfliting ording to [4] if the reltion grphs re different. Let F 1, F 2 F e two sets of ojetives. F 1 is onfliting with F 2 iff F1 F2. Note tht the ddition of n ojetive to prolem n, therefore, ffet the running time of dominne reltionsed EA, e.g., simple evolutionry multiojetive optimized (SEMO), only if the dditionl ojetive is onfliting with the set of ojetives, defining the originl prolem. As the exmple in Figs. 1 nd 2 shows, the ddition of f 2 to the prolem defined y f 1 nd f 3 does not hnge the underlying dominne reltion nd, therefore, does not hnge the running time of evolutionry lgorithms whih onsider the dominne reltion solely. Now, the question rises, how the ddition of onfliting ojetive ffets the omplexity of prolem, or more preisely, how n dditionl ojetive hnges the running time of n evolutionry lgorithm. Addressing the ove-mentioned question, we sketh the fundmentl ide of this pper. When dding n ojetive f i to n ojetive set F, there n e two situtions: 1) omprle solutions n eome inomprle; 2) n indifferent reltion etween solutions n eome omprle one. Of ourse, oth ses n our simultneously, if n ojetive is dded. 1 Surprisingly, in oth ses, prolem n eome esier or hrder to solve s is shown nlytilly in the following setions. Generlly speking, se 1) turns region with given serh spe diretion into plteu of inomprle solutions, wheres se 2) turns plteu of indifferent solutions into region where the wek Preto dominne indites diretion. The different ehvior of dditionl ojetives in oth ses depends on the diretion in whih the wek Preto dominne points. In se 1), where omprle solutions eome inomprle, the omprility etween solutions n either led to the Preto front or e deeptive. The ddition of n ojetive will use new plteu of inomprle solutions, ut in the ltter se the inomprility will help to solve the prolem, wheres in the former se the inomprility will mke the prolem hrder. In se 2), the prolem n either eome hrder or esier when hnging the dominne struture from plteu of indifferent solutions into region of omprle solutions. Depending on whether the newly introdued omprility will led to the Preto front or ehve deeptively, the omputtionl effort to identify the Preto optim my derese or inrese. III. ALGORITHMS This setion defines the setting for the running time nlyses to follow. As to the serh spe, we onsider pseudo oolen funtions f : {0, 1} n R k, i.e., X = {0, 1} n. Conerning the lgorithms, we exmine oth single-ojetive EA nd multiojetive EA. For single-ojetive optimiztion prolems (where k = 1), our nlyses re sed on the (1 + 1) EA (Algorithm 1) whih hs een onsidered in theoretil investigtions on pseudo oolen funtions [9] s well s some of the est known omintoril optimiztion prolems [13], [26], [32]. 1 The other wy round, n omission of n ojetive n mke inomprle solutions omprle nd omprle solutions indifferent.

4 594 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 Algorithm 1 (1+1) EA Choose x {0, 1} n uniformly t rndom repet Crete x y flipping eh it of x with proility 1/n if f (x ) f (x) then set x := x until stop Algorithm 2 GLOBAL SEMO Choose x {0, 1} n uniformly t rndom Determine f (x) P {x} repet Choose x P uniformly t rndom Crete x y flipping eh it of x with proility 1/n Determine f (x ) if x is not dominted y ny other serh point in P then inlude x into P nd delete ll solutions dominted y x or with ojetive vetor f (x ) from P until stop The lgorithm works with popultion of size 1 together with elitism seletion nd retes in eh itertion one offspring y flipping eh it with proility 1/n. Anlyzing single-ojetive rndomized serh heuristis with respet to their running time ehvior, we re interested in the numer of onstruted solutions until n optiml one hs een reted for the first time. This is lled the running time or optimiztion time of the onsidered lgorithm. Often, the expettion of this vlue is onsidered nd lled the expeted optimiztion time or expeted running time. We ompre the (1 + 1) EA with its multiojetive ounterprt lled Glol SEMO (Algorithm 2) [12], [20], whih hs een investigted in the ontext of different multiojetive prolems, e.g., spnning tree prolems [23], [25]. Glol SEMO strts with n initil popultion P tht onsists of one single rndomly hosen individul. In eh genertion, n individul x of P is hosen rndomly to produe one hild x y muttion. In the muttion step, eh it of x is flipped with proility 1/n to produe the offspring x. After tht, x is dded to the popultion if it is not dominted y ny individul in P. Ifx is dded to P, ll individuls of P tht re dominted y x or hve the sme ojetive vetor s x re removed from P. In detil, Glol SEMO is defined in the Algorithm 2. Anlyzing multiojetive evolutionry lgorithms with respet to their running time ehvior, we onsider the numer of onstruted solutions until for eh Preto-optiml ojetive vetor solution hs een inluded into the popultion nd ll this the optimiztion time of the lgorithm the expeted optimiztion time refers to the expettion vlue of the optimiztion time. Let x 1 denote the numer of 1s nd x 0 denote the numer of 0s in given itstring x. We re lso interested in vrints of the introdued lgorithms using the following symmetri muttion opertor proposed in [15]. Algorithm 3 ASYMMETRIC MUTATION OPERATOR Crete x y flipping eh it x i of x with proility 1/(2 x 1 ) if x i = 1 nd with proility 1/(2 x 0 ) otherwise We denote y (1 + 1) EA sy nd Glol SEMO sy the lgorithms tht differ from the (1 + 1) EA nd Glol SEMO y using the muttion opertor given in Algorithm 3. IV. ADDING OBJECTIVES TO A PLATEAU Our im is to exmine the effet of dding different ojetives to well-known plteu funtion. Plteus re regions in the serh spe where ll serh points hve the sme ojetive vetors. Consider funtion f : {0, 1} n R nd ssume tht the numer of different ojetive vlues for tht funtion is V. Then there re t lest 2 n /V serh points with the sme ojetive vlue. Often, the numer of different ojetive vlues for given funtion is polynomilly ounded. This implies n exponentil numer of solutions with the sme ojetive vlue. Nevertheless, suh funtions where V is polynomilly ounded re esy to optimize for EAs if for eh nonoptiml solution there is etter Hmming neighor, whih mens tht n improvement n e rehed y flipping single it of nonoptiml solution. If this is not the se, the serh for rndomized serh heuristi my eome muh hrder. In the extreme se, we end up with the funtion NEEDLE where only one single solution hs ojetive vlue 1 nd the remining ones get n ojetive vlue of 0 [16]. The ehvior of the (1 + 1) EA on plteus of different strutures hs een studied in [16] y rigorous running time nlysis. The funtion PLATEAU1, whih we exmine in the following, ontins set of n 1 serh points tht form plteu hving ojetive vlue n+1. We denote y SP 1 :={1 i 0 n i, 1 i < n} this set of serh points nd define PLATEAU1 s x 0 : x SP 1 PLATEAU1(x) := n + 1: x SP 1 n + 2: x = 1 n. Note, tht this funtion is similr to the funtion SPCn lredy investigted in [16]. The reltion grph of PLATEAU1 for n = 4isshownin Fig. 3. The serh is direted to the ll-zero string s long s no serh point with ojetive vlue t lest n + 1 hs een produed. This hs the effet for simple rndomized serh heuristis suh s the (1 + 1) EA tht fter hving rehed the plteu the Hmming distne to the optiml serh point is lrge. Nevertheless, the struture of the plteu dmits fir rndom wlk. The following theorem shows n expeted optimiztion time of (n 3 ). Theorem 1: The expeted running time of the (1 + 1) EA on PLATEAU1 is (n 3 ). Proof: As the reltive struture of PLATEAU1 nd SPCn (s defined in [16]) re identil esides the inlusion of 0 n in the plteu or not, we n reuse ll ides used in the proof of [16] for the expeted running time O(n 3 ) of the (1 + 1) EA on SPCn. Therefore, lso on PLATEAU1 the expeted running time of the (1 + 1) EA n e ounded y O(n 3 ).Tothe

5 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS Fig. 3. Reltion grph for the ojetive funtion PLATEAU1 : {0, 1} 4 R. Reflexive nd trnsitive edges re omitted for lrity. Fig. 4. Reltion grph for the i-riterion prolem PLOM: {0, 1} 4 R 2. Reflexive nd trnsitive edges re omitted for lrity. est of our knowledge, there is, up to now, no mthing lower ound in the literture. We will now prove lower ound of (n 3 ). In the initiliztion step of the (1 + 1) EA, solution x {0, 1} n is produed tht fulfills x 1 (2/3)n with proility 1 o(1) y Chernoff ounds. As long s the urrent solution is not in SP 1 nd not equl to 0 n,thevlue x 1 is noninresing. Thus, the first individul x hosen y the (1 + 1) EA tht is in the set SP 1 hs the property x 1 (2/3)n with proility 1 o(1). One the urrent serh point is in the set SP 1, only hildren lso from the set SP 1 re epted. Hene, only the following muttions re llowed for n epted muttion step. The first omponents of x tht re 0 s or the lst omponents of x tht re 1 s n e flipped. The proility to flip four or more omponents in n epted step is t most ni=4 2(1/n) i (n 1/n) n i = O(n 4 ). Thus, with proility 1 o(1) no suh muttion will e epted in time (n 3 ). The proility for muttion step onsisting of three flips to e epted is t most 2(1/n) 3 (n 1/n) n 3 = O(n 3 ). With proility 1 o(1) there will e only onstnt numer of suh muttion steps in time (n 3 ). By the sme rguments, there re only O(n) epted muttion steps with extly two flips nd only O(n 2 ) epted muttion steps with extly one flipped it in time (n 3 ). Therefore, in time (n 3 ) the two nd three-it flip muttions n only derese the Hmming distne of the urrent serh point x to the point 1 n y t most O(n 1/2 ) with proility 1 o(1), sine the two it flip muttions nd the three-it flip muttions oth perform rndom wlk on the line SP 1. Thus, the serh point hs to over distne of order (n) y one-it flip muttions. This tkes (n 2 ) epted one-it flips with proility 1 o(1) using similr rguments s in [8]. Sine the expeted time for n epted one-it flip is (n), the time until the (1 + 1) EA hs rehed the serh point 1 n is (n 3 ). The nlyses of vrints of the (1 + 1) EA in [8], [13], [24] point out tht some of the well-known omintoril optimiztion prolems suh s mximum mthing or Eulerin yle hve nturl ojetive funtions where plteus hve similr struture s in PLATEAU1. Therefore, this funtion plys key role when onsidering the ehvior of rndomized serh heuristis on plteus nd understnding the effet of dding ojetives to tht funtion my led to more effiient serh heuristis y using dditionl ojetives. We investigte the effet of dding two of the simplest nontrivil ojetive funtions to the prolem nd onsider the ehvior of Glol SEMO on these funtions. Nmely, we onsider the i-ojetive prolems PLOM(x) := (PLATEAU1(x), x 1 ) PLZM(x) := (PLATEAU1(x), x 0 ) nd show tht Glol SEMO is fster (Theorem 2) on PLOM nd exponentilly slower on PLZM (Theorem 4) ompred to the (1 + 1) EA on PLATEAU1. Note tht the optimum of PLATEAU1 is inluded in the Preto-optiml sets of PLOM nd PLZM. In ddition, the Preto fronts of the i-ojetive prolems PLOM nd PLZM re of onstnt size 1, nd 2, respetively. Aording to [18], multiojetiviztion only mkes sense if the single-ojetive optimum is inluded in the not too lrge Preto front of the new prolem whih is given for oth PLOM nd PLZM. We now onsider Glol SEMO on the prolem PLOM. The first oservtion is tht ll x SP 1 re omprle in PLOM while they re indifferent in PLATEAU1. The seond ojetive x 1 of PLOM lso gives the Glol SEMO the right diretion to move on the former plteu (n + 1, ) up to the only Preto optimum 1 n. This n e seen niely in the reltion grph of PLOM in Fig. 4. The following theorem shows tht Glol SEMO is indeed signifintly fster on PLOM thn (1 + 1) EA on PLATEAU1. Theorem 2: The expeted optimiztion time of Glol SEMO on PLOM is (n 2 log n). Proof: The single Preto optimum of PLOM is 1 n with the orresponding ojetive vetor (n + 2, n). The popultion size is ounded y O(n) s eh ojetive funtion ttins t most n + 3 different vlues. If the initil rndom x {0, 1} n is in SP 1, Glol SEMO will wlk long the ojetive vetors

6 596 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 (n + 1, ) up to 1 n in expeted O(n 2 log n) steps. This follows from the Coupon Colletor s Prolem [22] nd the ft tht in eh step the lgorithm hooses with proility 1/n the uppermost serh point of SP 1. If the initil solution is not in SP 1, Glol SEMO produes solutions tht trdeoff etween the numer of 1s nd 0s. In this se, we onsider the numer of steps until solution with ojetive vetor (n + 1, ) is inluded or solution 1 n is found. Sine the popultion size is ounded y O(n), the expeted numer of steps to go from n x with x 1 = k to n x with x 1 = k + 1 is O(n n/(n k)). Therefore, fter O(n 2 n k=1 1/k) = O(n 2 log n) steps, the single Preto-optiml serh point 1 n is found. For the proof of the lower ound we mke slight modifition to the Glol SEMO model nd rgue fterwrds why this is dmissile. We ssume tht every newly generted hild is epted y Glol SEMO. This is indeed the se in the phse until Glol SEMO hs determined the first solution x with PLATEAU1(x) >n. We will show tht with proility 1 o(1) the modified model is not different from Glol SEMO in the phse we re nlyzing. Sine the initil individul is uniformly distriuted in {0, 1} n nd the muttion step produes from uniformly distriuted prent uniformly distriuted hild, every element x hving x 1 = i tht is newly generted in our modified model is uniformly distriuted in {x {0, 1} n, x 1 = i}. For every suh element x the proility tht it is mutted to n element x with x 1 = i nd PLATEAU1(x ) n is extly 1/ ( n ) x 1. Thus, the proility tht every produed x {0, 1} n with 3 x 1 n 3in (n 2 log n) steps fulfills PLATEAU1(x )<n is t lest ( ) O(n 2 1 ( 1 log n) n = 1 o(1). 3) In other words, with proility 1 o(1) our new model ehves in (n 2 log n) steps extly like Glol SEMO nd produes no solution x with PLATEAU1(x )>n s long s every element x of the urrent popultion fulfills 3 x 1 n 3. For the lower ound proof it is enough to restrit ourselves to this frtion of ses. Let min := min x P min{ x 0, x 1 }, where P denotes the urrent popultion. So min is the miniml numer of 1s respetively 0s of n individul in the urrent popultion. Until the first individul x {0, 1} n with PLATEAU1(x) > n is produed, the vlue min is deresing nd the popultion size is inresing. After the initiliztion, min n/3 holds with high proility (w. h. p.) using Chernoff ounds [22]. We regrd the phse where min is in the rnge etween n/3 nd n/4 nd show tht the popultion size fter this phse is of order (n) with proility t lest 1/2. Let us onsider only steps tht derese min. We show tht the expeted derese of min in ll suh steps in the phse min [n/4, n/3] is ounded y 2. To otin from step tht dereses min y i step tht dereses min y i + 1, one of the remining (t most n/3) 1s respetively 0s hs to e flipped. The proility for this extr flip is t most (n/3)/n = 1/3. Thus, the expeted derese of min in suh steps is t most 2 (geometri series). Therefore, the verge derese of min in the phse min [n/4, n/3] is lrger thn 4 with proility less thn 1/2. It follows tht with proility t lest 1/2 the popultion size is (n) when hving otined for the first time solution with t most n/4 1s respetively 0s. With high proility, min is greter or equl 2 n 1/4 t this time. In other words, we n ssume tht there re t lest 2 n 1/4 1s respetively 0s left in every element of the urrent popultion of size (n). For every x in the urrent popultion, we define (x) := min{ x 0, x 1 }. Now we onsider the time to redue min from n 1/4 to 3. The proility to produe from solution y with (y) > min + n 1/4 n improving z is of order O(n n1/4 ) nd therefore suh n event does not hppen within polynomil numer of steps with proility lose to 1. We ll step k-step iff it retes solution z with z 1 > x 1 y flipping k of the remining 0-its respetively the remining 1-its. The proility to flip k of these its in single muttion step of solution y with (y) min + n 1/4 is upper ounded y (( min + n 1/4 )/n) k = O(n 3k/4 ). Sine the proility tht y with min y 1 min + n 1/4 will e hosen for the muttion from the urrent popultion is of order O(n 3/4 ), the proility for k-step mutting y from tht region is O(n 3(k+1)/4 ). Hene, for k 2 this does not hppen within (n 2 log n) steps with proility 1 o(1) using Mrkov s inequlity. This implies tht with proility 1 o(1) solution z with (z) < min n only e produed y mutting the t most 2 elements of the popultion with vlue min. The expeted time to redue the urrent min to min 1 y one step under the ondition tht n x with (x) = min hs een hosen for muttion is n/ min. Thus, the expeted time to redue the vlue min from n 1/4 to 3 is of the order n 1/4 (n) r=4 n r = (n2 log n). This shows tht the expeted time until the first x {0, 1} n with PLATEAU1(x) >n is determined y Glol SEMO is (n 2 log n), whih ompletes the proof. Using the symmetri muttion opertor, the funtion PLATEAU1 eomes muh hrder. Jnsen nd Sudholt [15] hve shown tht the proility tht (1 + 1) EA sy optimizes PLATEAU1 in 2 O(n1/4) steps is ounded ove y 2 (n1/4).in ontrst to this, the serh gets esier for Glol SEMO sy on PLOM. Theorem 3: The expeted optimiztion time of Glol SEMO sy on PLOM is (n 2 ). Proof: First ssume tht the popultion ontins n element x {1 i 0 n i, 1 i n}. For suh n element x, Glol SEMO sy ehves on PLOM like the (1 + 1) EA sy on x 1. Aording to [15], (1 + 1) EA sy needs n expeted time of O(n) to optimize x 1. As the popultion size is t most O(n), the optimum is rehed fter n expeted numer of O(n 2 ) steps. Now ssume tht we strt with n element x {1 i 0 n i, 1 i n}. We will nlyze the expeted numer of steps to reh the optimum ssuming tht no element from {1 i 0 n i, 1 i n} enters the popultion. Otherwise, we lredy know tht we need t most n dditionl numer of O(n 2 ) steps in expettion to reh the optimum. To mutte n element x

7 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS 597 towrds the optimum, muttion whih flips no one-it nd t lest one zero-it n e used. The proility tht suh muttion hppens for given x is Sine p(x) := ( x 1 ) ( x 1 1 ( ) k e 1/2 2k ( 1 1 ) ) x 0. 2 x 0 we n ound this proility y p(x) ((1 e 1/2 )/2). As two elements x, y ({0, 1} n \{1 i 0 n i, 1 i n}) with x 0 y 0 do not dominte eh other, s soon s muttion retes n element with k 1s, the popultion will ontin one suh element until the end of the lgorithm. Hene, we need n expeted numer of ( ) n 1 2 O n 1 e 1/2 = O(n 2 ) i=0 steps to reh the optimum, s speifi element of the popultion is piked with proility (1/n). The proof of the lower ound lrgely follows the proof of Theorem 2. Agin we show tht the popultion size is liner fter min first leves [n/4, n/3] y ensuring tht the expeted derese of min is onstnt in this intervl. For this, oserve tht to otin from step tht dereses min y i step tht dereses it y i + 1, one of the remining 1s (respetively 0s) hs to e flipped. The proility for this flip is t most (n/3)(1/2(n/4)) = 2/3, whih then leds to onstnt expeted derese in eh step. This in turn shows liner popultion size fter this phse. Hene, when min leves this intervl, the popultion P is w. h. p. of size P = (n). Alsow.h.p.,wehve min n/5. Now onsider the proility p ij to produe in the next muttion step from x with x 0 = i solution x with x 0 = j where i < j. Jnsen nd Sudholt [15] hve shown tht p ij 2 j i.letx P e the solution with the lrgest numer of zeros. Denote y D = x 0 the distne of x to 0 n. Consider solution y P with y 0 = x 0 + k, where k [0, x 1 ]. A muttion step of y redues D in expettion y t most n 2 i (i k) <2 k 2 i i = 2 k+1. i=k+1 Then the expeted derese of D in the next muttion step is t most 1 n 2 k+1 < 2 2 k = 4 P P P k=0 i=1 k=0 s P ontins t most one individul with k 0s for eh k. Sine P = (n) holds, the expeted derese of D in eh itertion is t most O(1/n). Hene, (n 2 ) itertions re neessry to redue the vlue of D y (n) whih ompletes the proof. It remins to exmine the prolem PLZM. An exponentil deelertion omes from the x SP 1. These serh points re now omprle in PLZM, ut this time the seond ojetive x 0 of PLZM is leding Glol SEMO nd Glol SEMO sy in the opposite diretion of the Preto optimum 1 n. The following theorem shows the more thn ler effet of dding the wrong ojetive. Theorem 4: The optimiztion times of Glol SEMO nd Glol SEMO sy on PLZM re e (n) with proility 1 e (n). Proof: The ojetive vetors (n + 2, 0), (n, n), nd (n + 1, n 1) with the orresponding serh points 1 n,0 n, nd 10 n 1 re the three Preto optim of PLZM. We show tht the limed lower ound holds for otining the serh point 1 n. The initil serh point onsists with proility 1 e (n) of t most 2n/3 1s using Chernoff ounds. Aepted steps inresing the numer of 1s hve to produe solution of SP 1 {1 n }. The proility to reh 1 n diretly from serh point x SP 1 is upper ound y 2 n/3 for oth lgorithms s ll 0-its hve to e flipped. The other opportunity to otin the serh point 1 n is to produe it from serh point of SP 1. The first solution of SP 1 found during the run of the lgorithm hs with proility 1 e (n) t most 3n/4 1-its s the proility of flipping (n) its in single muttion step is e (n) for oth lgorithms. Afterwrds, the numer of 1s n only e inresed y produing the serh point 1 n diretly. As eh individul in the popultion hs t most 3n/4 1s with proility 1 e (n), the proility of otining 1 n is upper ounded y 2 n/4 for oth lgorithms. Hene, overll the time to hieve the serh point 1 n is e (n) with proility 1 e (n). V. COPING WITH TWO PLATEAUS In Setion IV, the dded ojetives were esy to solve individully for the (1 + 1) EA. The min reson for the smller running time of PLOMs ompred to PLATEAU1 is tht oth funtions hve the sme glol optimum. The question rises whether omining two ojetives my result in fster optimiztion proess thn optimizing the different ojetive funtions seprtely. We show tht the omintion of two eqully omplex prolems yields n esier prolem if oth funtions re optimized s i-riterion prolem. We know from Theorem 1 tht Glol SEMO hs n expeted running time of (n 3 ) on PLATEAU1. Let SP 2 :={0 i 1 n i, 1 i < n}; then this result lso holds for the funtion x 1 : x SP 2 PLATEAU2(x) = n + 1: x SP 2 n + 2: x = 0 n due to the symmetry with PLATEAU1. We now onsider the multiojetive funtion PLATEAUS = (PLATEAU1(x), PLATEAU2(x)) where Glol SEMO hs to ope with plteu in eh ojetive nd show tht this is esier thn solving the singleojetive prolems seprtely. Theorem 5: The expeted optimiztion time of Glol SEMO on PLATEAUS is (n 2 log n). Proof: The ojetive vetors (n + 2, n) nd (n, n + 2) with the orresponding serh points 1 n nd 0 n re Pretooptiml, s they re the optim of the two ojetive funtions

8 598 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 PLATEAU1nd PLATEAU2. There does not exist n ojetive vetor (n + 1, n + 1) for the onsidered prolem whih shows tht the serh points 1 n nd 0 n re the only Preto-optiml ones. The popultion size is lwys ounded y O(n) s eh ojetive funtion ttins t most n + 3 different vlues. We onsider the numer of steps until solutions with ojetive vetors (n + 1, ) nd (, n + 1) hve een inluded into the popultion nd ssume tht the Preto-optiml solutions with ojetive vetors (n + 2, n) nd (n, n + 2), respetively, hve not een otined efore. We investigte the se to otin (n + 1, ). As long s suh solution hs not een otined, we onsider the solution x with the lrgest PLATEAU1 vlue in the popultion. This is determined y the numer of zeros in x. Assume tht x 0 = k holds. Then, the proility to produe from x solution x with higher numer of zeros is t lest (n k)/(en). The proility of hoosing x in the next step is (1/n). Hene, the numer of zeros inreses fter n expeted numer of O(n 2 /(n k)) steps. Summing up over the different vlues of k, the serh point 0 n with ojetive vetor (n, n + 2) hs een otined fter O(n 2 log n) steps if no solution with ojetive vetor (n + 1, ) hs een produed efore. Flipping the first it in 0 n leds to solution with ojetive vetor (n +1, ) nd n e otined in n dditionl phse of O(n 2 ) steps. The expeted time to otin solution with ojetive vetor (, n +1) n e ounded y O(n 2 log n) using the sme rguments. After P inludes solutions with ojetive vetors (n + 1, ) nd (, n + 1) or suset of Preto-optiml solutions dominting these vetors, the popultion size is lwys ounded y 2. We onsider how to otin the serh point 1 n.letx e the serh point with ojetive vetor (n + 1, k) in the popultion. Flipping the it x k+1 in x leds to solution x with ojetive vetor (n + 1, k + 1). The popultion size is t most 2 nd the proility of flipping one single speifi it is t lest 1/(en), whih implies tht the expeted witing time forsuhstepiso(n). Thevlueofk will e inresed t most n 1 times until the serh point 1 n hs een inluded into P. Hene, the expeted time until this solution hs een otined is O(n 2 ). The sme holds for inluding the serh point 0 n using the sme rguments. Altogether, the expeted optimiztion of Glol SEMO on PLATEAUS is O(n 2 log n). The lower ound proof is nlogous to the lower ound proof of Theorem 2, sine the funtions PLATEAUS nd PLOMre the sme on the set {0, 1} n \ ({0 i 1 n i, 0 i n} {1 i 0 n i, 0 i n}). Jnsen nd Sudholt [15] hve shown tht the (1 + 1) EA sy is totlly ineffiient on PLATEAU1. The sme rguments hold for PLATEAU2 s it differs from PLATEAU1only y exhnging the roles of zeros nd ones. Surprisingly, this does not hold for Glol SEMO sy nd PLATEAUS. In the following, we show tht Glol SEMO sy is quite effiient on PLATEAUS. Theorem 6: The expeted optimiztion time of Glol SEMO sy on PLATEAUS is (n 2 ). Proof: As in the proof of Theorem 5, we first ound the expeted numer of steps until the popultion inludes serh points with ojetive vetors (n + 1, ) nd (, n + 1) nd ssume tht the Preto-optiml ojetive vetors (n +2, n) respetively (n, n + 2) hve not een otined efore. For otining (n + 1, ), onsider the serh point x with the lrgest PLATEAU1vlue. Assume tht it hs x 0 = k zeros. The proility to otin from x solution with more zeros n e ounded y (1 e 1/2 )/2, s shown in the proof of Theorem 3. Summing this up for ll vlues of k nd using the ft tht the popultion size is lwys ounded y O(n), solution with ojetive vetor (n + 1, ) is otined fter n expeted numer of O(n 2 ) steps. By symmetry, the sme holds for otining serh point with ojetive vetor (, n + 1). Now ssume tht two serh points with ojetive vetors (n + 1, ) nd (, n + 1) re inluded in the popultion. Sine they dominte ll other points, the popultion size is upper ounded y 2 in this se. If the ojetive vetor of the first serh point is (n + 1, k), it onsists of k ones followed y n k zeros. Its ojetive vetor n e improved y flipping the (k + 1)th zero to one. The proility for this to hppen is p(x) = ( 1 1 ) k ( 2k whih n e ounded y p(x) (n k) 1 2(n k) ( 1 )( 1 ) 1 1 2(n k) 2 = ) 1 n k 1 2(n k) ( ) 1. n Hene, fter n expeted numer of O(n 2 ) steps the ojetive vetor will reh (n +2, n). By symmetry, the sme holds for otining the serh point with ojetive vetor (n, n + 2). The lower ound proof n e done nlogously to the lower ound proof of Theorem 3, sine the funtions PLATEAUS nd PLOMre the sme on the set {0, 1} n \ ({0 i 1 n i, 0 i n} {1 i 0 n i, 0 i n}). VI. EXPERIMENTAL STUDIES In the previous setions, we investigted plteus of indifferent solutions in single-ojetive prolems nd exmined how n dditionl ojetive hnges the dominne reltion on this plteu nd therefore influenes the running time for simple lgorithms like the Glol SEMO nd the (1 + 1) EA. With the following experimentl study, we wnt to tkle three questions tht remin open fter our theoretil investigtions: 1) n the symptotil results lso e oserved for smll instnes; 2) n the effet of mking prolem hrder or esier y dding n ojetive e reported for multiojetive prolem insted of the single-ojetive PLATEAU1 nd; 3) n we oserve the sme ehvior lso on other types of plteus, e.g., sets of inomprle solutions? In the following, we investigte experimentlly for oth multiojetive prolems nd plteus of inomprle solutions whether the running time of Glol SEMO n e inresed nd deresed with n dditionl ojetive. First, we investigte i-ojetive prolem with the sme plteu SP 1 tht ws onsidered ove, wheres Setion VI-B shows tht n ddition of ojetives n inrese or derese the running time of Glol SEMO lso for other kinds of plteus. The generl explntion of wht hppens remins the sme s in the previous setions: if plteus re introdued y n dditionl ojetive, the running time inreses if good

9 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS 599 diretion on the serh points vnishes nd dereses if deeptive diretion vnishes; if plteus re eliminted y dding diretion to the orresponding serh spe region, the new diretion inreses or dereses the running time depending on whether the introdued diretion is deeptive or not. A. Similr Plteus With More Ojetives First, we investigte the influene of the ddition of third ojetive to i-ojetive prolem, sed on the two funtions nd LEADINGONES(x) = TRAILINGZEROS(x) = n i i=1 j=1 n i=1 j=i x j n (1 x j ) whih were first investigted in [19] s the prolem LOTZ. Here, we onsider the slightly hnged funtions LEADINGONES(x) if x SP 1 f 1 (x) = n + 1 if x SP 1 n + 2 if x = 1 n nd TRAILINGZEROS(x) if x SP 1 f 2 (x) = n + 1 n + 2 if x SP 1 if x = 1 n tht re to e mximized t the sme time leding to the modified LOTZ prolem ( f 1, f 2 ) where SP 1 ={1 i 0 n i, 1 i < n} s defined ove. Note, tht f 1 (x) = f 2 (x) holds if x SP 1. In this se, we hve to ope with the sme plteu s given y the funtion PLATEAU1. The only differene etween the i-ojetive prolem ( f 1, f 2 ) nd the funtion PLATEAU1 is given y the serh points not on the plteu. Here, the popultion of Glol SEMO my grow due to numer of inomprle solutions. Strting with the modified LOTZ prolem ( f 1, f 2 ), we investigte the effet of dding either the funtion x 1 or the funtion x 0 to the prolem. Adding x 1 dereses the running time of Glol SEMO, wheres dding x 0 inreses it. The effet is used y the sme priniple oserved in Setion IV. Right efore finding the Preto-optiml front, Glol SEMO hs to overome the plteu 1 i 0 n i (1 i < n) of indifferent solutions. If x 1 is dded, this third ojetive indues diretion to the optimum on this plteu; if x 0 is dded, the generted diretion on the plteu is deeptive. Fig. 5 shows the ox plots of the running times of 31 independent Glol SEMO runs on ll three prolems for different itstring lengths (n {5, 10, 15, 20, 25, 30}). 2 The 2 The oxplots hve een produed y the uilt-in oxplot ommnd of MATLAB showing the lower qurtile, medin, nd upper qurtile vlues. The defult mximum whisker length of 1.5 times the interqurtile rnge hs een used. Dt points lying eyond the ends of the whiskers re mrked y +. runtime [genertions] Glol SEMO on modified LOTZ with dditionl third ojetive numer of deision vriles Fig. 5. Comprison of the running times for Glol SEMO if third ojetive is dded to the modified LOTZ prolem: originl prolem (solid line) ( f 1, f 2 ), (dshed line) ( f 1, f 2, x 1 ), nd (dotted line) ( f 1, f 2, x 0 ). Note, tht the runs re orted if no Preto-optiml point hs een found in the first genertions. For lrity, the three oxplots orresponding to speifi numer of deision vriles hve een slightly shifted horizontlly. nonprmetri Kruskl Wllis test with the extension to multiple omprisons 3 hs een performed to support the ovestted hypotheses tht Glol SEMO needs more time for optimizing ( f 1, f 2, x 0 ) thn for ( f 1, f 2 ) nd tht Glol SEMO needs less time for optimizing ( f 1, f 2, x 1 ) thn for the originl prolem ( f 1, f 2 ). The null hypothesis of equl distriutions ws rejeted t the signifine level of 0.01 for ll onsidered deision spe sizes supporting the visul illustrtion of Fig. 5. Note tht the runs were orted if no Preto-optiml serh point hs een found in the first genertions. This nd the lrge vrine of the single runs explin the unexpeted derese of the medin etween the originl prolem with 25 nd the one with 30 deision vriles. B. Different Kinds of Plteus In ddition to plteus of indifferent solutions whih our frequently in single-ojetive prolems, multiojetive prolems my exhiit plteus of inomprle solutions s well. In this setion, we investigte the running time hnges of Glol SEMO for oth kinds of plteus if n ojetive is dded. 1) Plteus of Indifferent Solutions: The sis i-ojetive prolem we use for the investigtion of plteus of indifferent solutions is the originl LOTZ of [19]. All solutions with the sme numer of leding 1s nd triling 0s re mpped to the sme ojetive funtion vlues yielding plteu of indifferent solutions. In the following, we will refer to the deision vriles tht neither elong to the leding 1s nor to the triling 0s of solution x s the middle lok x M. In ddition, x denotes the length of the itstring x. Adding 3 As implemented in the PISA performne ssessment toolox [1] nd desried in [6, p.290].

10 600 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 mking indifferent solutions omprle mking indifferent solutions omprle runtime [genertions] runtime [genertions] numer of deision vriles numer of deision vriles Fig. 6. How prolem n eome either esier or hrder if indifferent solutions re mde omprle, i.e., if plteu is removed y n dditionl ojetive. Here, the oxplots for the running times until the first Preto-optiml point is rehed re shown. Note, tht the runs re orted if no Preto-optiml point hs een found in the first genertions. For lrity, the three oxplots orresponding to speifi numer of deision vriles hve een slightly shifted horizontlly. Fig. 7. How prolem n eome either esier or hrder if omprle solutions re mde inomprle, i.e., if plteu is introdued y n dditionl ojetive. Here, the oxplots for the running times until the first Preto-optiml point is rehed re shown. Note tht the runs re orted if no Preto-optiml point hs een found in the first genertions. For lrity, the three oxplots orresponding to speifi numer of deision vriles hve een slightly shifted horizontlly. ojetives tht tke into ount only the its in the middle loks of solutions will give diretion to these plteus of indifferent solutions. Depending on whether this diretion is deeptive or not, the running times of Glol SEMO on the orresponding three-ojetive prolem will e higher or lower thn for the originl i-ojetive prolem. Minimizing the ojetive g (i) in ddition to LOTZ = (g 1, g 2 ) (s defined in Fig. 8) inreses the running time due to its deeptive ehvior. On the one hnd, the numer of leding 1s nd triling 0s hs to e mximized to reh the Pretooptiml front; on the other hnd, the dditionl ojetive g (i) rewrds higher numer of leding 0s in the middle lok s well s higher numer of triling 1s. This fores Glol SEMO to flip more or less ll its in the middle lok t lest one insted of enefiting from lredy orretly set its, i.e., the running time inreses. In ontrst, the dditionl minimiztion of ojetive g (ii) = x M 1, or in other words the mximiztion of zeros in the middle lok, will flip its of the middle lok to zeros lso if they do not ontriute diretly to the mximiztion of g 1 nd g 2. However, with the middle lok s its tht re lredy set to zero, Glol SEMO is le to perform ig jumps in the ojetive funtion vlue of g 2 in future steps, i.e., the running time dereses. To support the ove-mentioned hypothesis tht the ddition of g (i) inreses nd the ddition of g (ii) dereses the running time of Glol SEMO in omprison with the originl prolem (g 1, g 2 ), 31 independent runs of Glol SEMO were performed for different numers of deision vriles (n {25, 50, 75, 100, 125, 150}). Note tht we mesured the numer of genertions until the first Preto-optiml point hs een found y Glol SEMO insted of the norml running time. The reson for tht is the lredy high numer of genertions tht re needed to find the first Preto-optiml point whih fored us to restrit the numer of genertions to : if Glol SEMO did not find ny Preto-optiml point within the first genertions, we stopped the run nd noted s the run s optimiztion time. Fig. 6 shows the orresponding ox plots. The nonprmetri Kruskl Wllis test for multiple omprisons of [6] gin rejets the null hypothesis of equl distriutions for ll omprisons t signifine level of 0.01 exept for the omprison etween the running times for (g 1, g 2, g (i) ) nd (g 1, g 2 ) with 25 deision vriles where the p-vlue is pproximtely ) Plteus of Inomprle Solutions: It remins to show tht lso for prolems with plteus of inomprle solutions, the ddition of ojetives n hnge the running time of n EA in oth wys. To this end, the prolem (h 1, h 2 ) s defined in Fig. 8 is investigted. The ojetive spe of (h 1, h 2 ) n e illustrted s the ojetive spe of the originl LOTZ prolem where the levels 2, 3, 6, 7, 10, 11, nd so forth re mirrored t the origin nd then trnslted. Fig. 9 illustrtes this prolem exemplry for smll numer of deision vriles. The hnge of the originl LOTZ prolem to (h 1, h 2 ) turns round the Preto dominne reltion etween the mirrored levels: where the Preto dominne reltion is inditing the diretion to the optimum in LOTZ, the new serh spe diretion is deeptive. Glol SEMO hs to jump out of the newly introdued lol optim y t lest two-it flip. This is where third ojetive n help. By mking the solutions within the region with deeptive Preto dominne reltion inomprle, Glol SEMO is le to perform rndom wlk on newly introdued plteus of inomprle solutions. If on the other hnd, solutions where the Preto dominne reltion points in diretion to the Preto-optiml front re mde inomprle, Glol SEMO needs more time to find the Preto-optiml front thn for the originl prolem (h 1, h 2 ).

11 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS 601 mx g se funtion g [19] 1 (x) = LEADINGONES(x) mx g 2 (x) = TRAILINGZEROS(x) slower with min g (i) (x) = x M LEADINGZEROS(x M ) TRAILINGONES(x M ) fster with min g (ii) (x) = x M 1 se funtion h mx mx h 1 (x) = h 2 (x) = LEADINGONES(x) n xm 2 LEADINGONES(x) n 2 TRAILINGZEROS(x) n xm 2 TRAILINGZEROS(x) n 2 iff 0 (n x M ) mod 4 or 1 (n x M ) mod 4 else iff 0 (n x M ) mod 4 or 1 (n x M ) mod 4 else slower with min h (i) (x) = n 2 n 2 x M fster with min h (ii) (x) = x M Fig. 8. Definitions of the funtions illustrting the hnges of running time with respet to mking indifferent solutions omprle [prolems (g 1, g 2, g ( ) )] nd mking omprle solutions inomprle [prolems (h 1, h 2, h ( ) )]. x M = 0 h 2 running time on (h 1, h 2, h (ii) ), whih is supported y the sme Kruskl Wllis test s mentioned efore t signifine level of 0.01 for ll tested deision spe sizes. x M = 4 x M = **1 0******1 111**** VII. CONCLUSION 1111**00 111**000 11**0000 x M = 5 x M = 6 111***00 x M = 3 x M = 2 Fig. 9. Illustrtion of the ojetive spe for the modified LOTZ prolem (h 1, h 2 ) nd n = 8 deision vriles. For some ojetive vetors, the orresponding solutions in deision spe re indited, where * denotes either 1 or 0 on the orresponding it string position. The ojetives h (i) nd h (ii) defined in Fig. 8 re introduing these inomprilities either on the mirrored levels of LOTZ only (h (ii) ) or in oth the first nd third qudrnt (h (i) ). The expeted ehvior is tht the ddition of h (i) will inrese nd the ddition of h (ii) will derese the running time of Glol SEMO in omprison to the i-ojetive prolem. Fig. 7 shows the oxplots of 31 independent Glol SEMO runs for different numers of deision vriles (n {25, 50, 75, 100, 125, 150}). As efore, we ount the numer of genertions until the first Preto-optiml point is found or ount if no Preto-optiml point is found within the first genertions. The visul inspetion of the oxplots in Fig. 7 indites tht Glol SEMO hs higher verge running time on (h 1, h 2, h (i) ) nd lower verge h 1 We hve investigted the question of how dditionl ojetives ffet 1) the struture, i.e., the dominne reltion, of given optimiztion prolem nd 2) the serh ehvior of evolutionry lgorithms pplied to this prolem. Motivted y previous studies on the reltionship etween single-ojetive nd multiojetive versions of prtiulr prolems ([25], [29]), we hve shown tht one nd the sme prolem n e mde oth esier nd more diffiult solely y dding different ojetives, i.e., without hnging the serh spe or one of the existing ojetives. In prtiulr, we hve provided rigorous running time nlyses in order to prove tht in the extreme se the effet of dding ojetives n mke the differene etween polynomil nd n exponentil running time. The hnges in the running times re due to hnges in the dominne struture: whenever new ojetive interferes with the existing ones, plteus of indifferent or inomprle solutions my emerge or vnish, respetively, nd my e enlrged or redued. Therefore, useful or misleding informtion my e implnted or removed. So, dditionl ojetives n derese the omplexity of prolem, lthough in generl this effet is less likely if more ojetives re involved s indited in [17] nd [31]. The presented results hve different implitions. On the one hnd, they n help with the design nd the lssifition of multiojetive enhmrk prolems ording to different tegories of hrdness. On the other hnd, they indite tht domin knowledge my not only e inorported in terms of prolem-speifi lgorithmi omponents, ut lso in the form of dditionl ojetive funtions. Finlly, the insights my e led to generl guidelines on prolem trnsformtions from M to N ojetives.

12 602 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 13, NO. 3, JUNE 2009 REFERENCES [1] S. Bleuler, M. Lumnns, L. Thiele, nd E. Zitzler, Pis A pltform nd progrmming lnguge independent interfe for serh lgorithms, in Pro. 2nd Int. Conf. Evol. Multi-Criterion Optimiztion (EMO), LNCS vol New York: Springer-Verlg, 2003, pp [2] D. Brokhoff, T. Friedrih, N. Heinghus, C. Klein, F. Neumnn, nd E. Zitzler, Do dditionl ojetives mke prolem hrder? in Pro. 9th Annu. Conf. Geneti Evol. Comput. (GECCO), NewYork:ACM, 2007, pp [3] D. Brokhoff nd E. Zitzler, Are ll ojetives neessry? On dimensionlity redution in evolutionry multiojetive optimiztion, in Pro. 9th Int. Conf. Prllel Prolem Solving From Nture (PPSN), LNCS vol New York: Springer-Verlg, 2006, pp [4] D. Brokhoff nd E. Zitzler, Dimensionlity redution in multiojetive optimiztion: The minimum ojetive suset prolem, in Pro. Opertions Res. 2006, New York: Springer-Verlg, pp [5] C. A. Coello Coello, G. B. Lmont, nd D. A. Vn Veldhuizen, Evolutionry Algorithms for Solving Multiojetive Prolems (Geneti nd Evolutionry Comput.) New York: Kluwer Ademi, [6] W.J.Conover,Prtil Nonprmetri Sttistis. 3rd ed. New York: Wiley, [7] K. De, Multiojetive Optimiztion Using Evolutionry Algorithms. New York: Wiley, [8] B. Doerr, N. Heinghus, nd F. Neumnn, Speeding up evolutionry lgorithms through symmetri muttion opertors, Evol. Comput., vol. 15, no. 4, pp , Nov [9] S. Droste, T. Jnsen, nd I. Wegener, On the nlysis of the (1 + 1) evolutionry lgorithm, Theoretil Comput, Si., vol. 276, no. 1 2, pp , Apr [10] P. J. Fleming, R. C. Purshouse, nd R. J. Lygoe, Mny-ojetive optimiztion: An engineering design perspetive, in Pro. 3rd Int. Conf. Evol. Multi-Criterion Optimiztion, LNCS vol New York: Springer-Verlg, 2005, pp [11] C. M. Fonse nd P. J. Fleming, An overview of evolutionry lgorithms in multiojetive optimiztion, Evol. Comput., vol. 3, no. 1, pp. 1 16, [12] O. Giel, Expeted runtimes of simple multiojetive evolutionry lgorithm, in Pro IEEE Congr. Evol. Comput. (CEC), Pistwy, NJ: IEEE Press, pp [13] O. Giel nd I. Wegener, Evolutionry lgorithms nd the mximum mthing prolem, in Pro. 20th Annu. Symp. Theoretil Aspets Comput. Si. (STACS), LNCS vol New York: Springer-Verlg, 2003, pp [14] J. Horn, Multiriterion deision mking, in Hndook Evol. Comput., Oxford, U.K. Oxford Univ. Press, [15] T. Jnsen nd D. Sudholt, Design nd nlysis of n symmetri muttion opertor, in Pro IEEE Congr. Evol. Comput. (CEC), Pistwy, NJ: IEEE Press, pp [16] T. Jnsen nd I. Wegener, Evolutionry lgorithms How to ope with plteus of onstnt fitness nd when to rejet strings of the sme fitness, IEEE Trns. Evol. Comput., vol. 5, no. 6, pp , De [17] M. T. Jensen, Helper-ojetives: Using multiojetive evolutionry lgorithms for single-ojetive optimistion, J. Mth. Modelling nd Algorithms, vol. 3, no. 4, pp , [18] J. D. Knowles, R. A. Wtson, nd D. W. Corne, Reduing lol optim in single-ojetive prolems y multiojetiviztion, in Pro. 1st Int. Conf. Evol. Multi-Criterion Optimiztion, LNCS vol New York: Springer-Verlg, 2001, pp [19] M. Lumnns, L. Thiele, nd E. Zitzler, Running time nlysis of evolutionry lgorithms on simplified multiojetive knpsk prolem, Nturl Computing, vol. 3, no. 1, pp , Mr [20] M. Lumnns, L. Thiele, E. Zitzler, E. Welzl, nd K. De, Running time nlysis of multiojetive evolutionry lgorithms on simple disrete optimiztion prolem, in Pro. 7th Int. Conf. Prllel Prolem Solving from Nture, LNCS vol New York: Springer-Verlg, 2002, pp [21] E. Mezur-Montes nd C. A. Coello Coello, Constrined optimiztion vi multiojetive evolutionry lgorithms, in Multiojetive Prolem Solving from Nture: From Conepts to Applitions, Berlin, Germny: Springer-Verlg, [22] R. Motwni nd P. Rghvn, Rndomized Algorithms. Cmridge, U.K.: Cmridge Univ. Press, [23] F. Neumnn, Expeted runtimes of simple evolutionry lgorithm for the multiojetive minimum spnning tree prolem, Eur. J. Oper. Res., vol. 181, no. 3, pp , [24] F. Neumnn, Expeted runtimes of evolutionry lgorithms for the Eulerin yle prolem, Comput. Opertions Res., vol. 35, no. 9, pp , Sep [25] F. Neumnn nd I. Wegener, Minimum spnning trees mde esier vi multiojetive optimiztion, Nturl Computing, vol. 5, no. 3, pp , Sep [26] F. Neumnn nd I. Wegener, Rndomized lol serh, evolutionry lgorithms, nd the minimum spnning tree prolem, Theoretil Comput. Si., vol. 378, no. 1, pp , Jun [27] R. C. Purshouse nd P. J. Fleming, On the evolutionry optimiztion of mny onfliting ojetives, IEEE Trns. Evol. Comput., vol. 11, no. 6, pp , De [28] T. P. Runrsson nd X. Yo, Serh ises in onstrined evolutionry optimiztion, IEEE Trns. Syst., Mn, Cyern., Prt C: Applit. nd Rev., vol. 35, no. 2, pp , My [29] J. Shrnow, K. Tinnefeld, nd I. Wegener, The nlysis of evolutionry lgorithms on sorting nd shortest pths prolems, J. Mth. Modelling nd Algorithms, vol. 3, no. 4, pp , [30] T. Wgner, N. Beume, nd B. Nujoks, Preto-, ggregtion-, nd inditor-sed methods in mny-ojetive optimiztion, in Pro. 4th Int. Conf. Evol. Multi-Criterion Optimiztion (EMO), LNCS vol New York: Springer-Verlg, 2007, pp [31] P. Winkler, Rndom orders, Order, vol. 1, no. 4, pp , De [32] C. Witt, Worst-se nd verge-se pproximtions y simple rndomized serh heuristis, in Pro. 22nd Annu. Symp. Theoretil Aspets Comput. Si. (STACS), LNCS vol New York: Springer- Verlg, 2005, pp nd evolutionry). mthemtil finne. Dimo Brokhoff reeived the Diplom in omputer siene from the University of Dortmund, Germny, in He hs een working towrd Ph.D. degree under Prof. Zitzler nd with the Computer Engineering nd Networks Lortory, Elgenössihe Tehnishe Hohshule Zurih, Zurih, Switzerlnd. His reserh interests re evolutionry omputtion nd, in prtiulr, multiojetive optimiztion. Tois Friedrih reeived the M.S. degree in omputer siene from the University of Sheffield, U.K. in 2003, the Diplom in mthemtis from the University of Jen, Germny, in 2005, nd the Ph.D. degree in omputer siene from the Srlnd University, Germny, in Sine 2008, he is with the Algorithm Group t the Interntionl Computer Siene Institute, Berkeley, CA. The entrl topis of his work re rndomized methods in mthemtis nd omputer siene nd rndomized lgorithms (oth lssil Nils Heinghus reeived the Diplom nd the Ph.D. degree in mthemtis from the University of Kiel in 2002 nd 2005, respetively. From Otoer 2005 to Septemer 2007, he ws working s Postdotorl Fellow with Deprtment 1: Algorithms nd Complexity t Mx-Plnk- Institut für Informtik, Srrüken, Germny. Sine Otoer 2007, he hs een Consultnt for mthemtil finne. His reserh interests re minly in the field of disrete mthemtis. Sine Otoer 2007, he hs een working s onsultnt for

13 BROCKHOFF et l.: ON THE EFFECTS OF ADDING OBJECTIVES TO PLATEAU FUNCTIONS 603 Christin Klein reeived the Diplom in omputer siene from the Srlnd University in Srrüken, Germny in Sine 2004, he hs een Ph.D. student with the Deprtment 1: Algorithms nd Complexity t Mx- Plnk-Institut für Informtik, Srrüken, Germny. His reserh interests inlude rndomized lgorithms nd omputtionl geometry. Frnk Neumnn reeived the Diplom nd the Ph.D. degrees in omputer siene from the University of Kiel in 2002 nd 2006, respetively. Sine Novemer 2006, he hs een Reserher of the Deprtment 1: Algorithms nd Complexity Group t the Mx-Plnk-Institut für Informtik in Srrüken, Germny. In his work, he onsiders theoretil spets of io-inspired omputtion methods, in prtiulr for prolems from omintoril optimiztion. Ekrt Zitzler reeived the Diplom in omputer siene from the University of Dortmund, Germny, nd the Dotor of Tehnil Sienes from Elgenössihe Tehnishe Hohshule (ETH) Zurih, Switzerlnd. Sine 2003, he hs een Assistnt Professor for Systems Optimiztion t the Computer Engineering nd Networks Lortory t the Deprtment of Informtion Tehnology nd Eletril Engineering, ETH Zurih, Switzerlnd. His reserh fouses on ioinspired omputtion, multiojetive optimiztion, omputtionl iology, nd omputer engineering pplitions. Prof. Zitzler ws Generl Co-Chirmn of the first three interntionl onferenes on evolutionry multiriterion optimiztion (EMO 2001, EMO 2003, nd EMO 2005).