Structure and Metaheuristics

Transcription

1 Structure ad Metaheurstcs Yoss Boreste Uversty of Essex Colchester, U.K Rccardo Pol Uversty of Essex Colchester, U.K ABSTRACT Metaheurstcs have ofte bee show to be effectve for dffcult combatoral optmzato problems. The reaso for that, however, remas uclear. A framework for a theory of metaheurstcs crucally depeds o a formal represetatve model of such algorthms. Ths paper ufes/recocles a sgle framework the model of a black box algorthm comg from the o-free-luch research (e.g. Wolpert et al. [25], Wegeer [23]) wth the study of ftess ladscape. Both are mportat to the uderstadg of meta-heurstcs, but they have so far bee studed separately. The ew model s a atural evromet to study meta-heurstcs. Categores ad Subject Descrptors F.2 [Theory of Computato]: ANALYSIS OF ALGO- RITHMS AND PROBLEM COMPLEXITY Geeral Terms Algorthms,Theory Keywords Theory, Represetato, No Free Luch, Heurstcs 1. INTRODUCTION Durg the last 20 years may algorthms (metaheurstcs) have bee proposed to effcetly explore search spaces o whch o kowledge s avalable [2]. Usually a search algorthm tres to fer the posto of good ew solutos the search space based o prevously sampled solutos. May metaheurstcs are spred by powerful atural or physcal processes. At Coloy Optmzato (ACO), Evolutoary Algorthms (EAs) ad Smulated Aealg (SA) are examples of such algorthms. ACO ad EAs are spred by ature; SA s spred by the aealg process of metals. These ad other metaheurstcs have bee appled successfully to a ever creasg umber of hard combatoral optmzato problems such as TSP, vehcle routg, job Permsso to make dgtal or hard copes of all or part of ths work for persoal or classroom use s grated wthout fee provded that copes are ot made or dstrbuted for proft or commercal advatage ad that copes bear ths otce ad the full ctato o the frst page. To copy otherwse, to republsh, to post o servers or to redstrbute to lsts, requres pror specfc permsso ad/or a fee. GECCO 06, July 8 12, 2006, Seattle, Washgto, USA. Copyrght 2006 ACM /06/ $5.00. shop schedulg, ad b packg. However, may cases, ther remarkable emprcal success s ot assocated to correspodgly robust theoretcal foudatos. It s of a partcular terest to study the dfferet search algorthms oe, sgle framework. The motvatos for ths are: Eve though dfferet metaheurstcs are spred by dfferet processes they are all desged to perform well the black box scearo, ad, deed, results obtaed for the black box scearo have mplcatos for all metaheurstcs at the same tme. Despte the apparet dffereces, metaheurstcs have a lot commo. Ths has led to the developmet of hybrd metaheurstcs whch are ofte prove to be more powerful tha the tradtoal oes. Furthermore, there are geeral commo propertes whch are expected to affect performace of all metaheurstcs as detfed [2]. Wolpert ad Macready were perhaps the frst to cosder the geeral propertes of the black box scearo. Wth ther No-free-luch theorem (NFLT) [25] they put a ed to the hope of developg a geeral-purpose robust optmzato algorthm by provg that such a algorthm does ot exst. Despte the vast mpact of the NFLT, ts mplcato o real-world problems ad real-world algorthms s ot clear. I partcular, t s argued that the set of problems that the NFLTs cosder s ot related to real-world problems. Culberso [5] showed that the umber of problems cosdered by the NFLTs s much hgher tha the oe cosdered by the class of NP or eve PSPACE problems. Eglsh [9, 8] made smlar argumets usg the oto of Kolmogorov complexty [10]. Droste, Jase ad Wegeer [6] descrbed the NFLTs scearo as o-realstc ad, fally, Igel ad Toussat [11] made smlar argumets for the sharper verso of the NFL [21]. It seems clear that, order to obta meagful results, t s essetal to cosder more realstc scearos. Wegeer et al. [23, 24, 7] descrbe black-box algorthms as radomzed decso trees ad use Yao s mmax prcple to derve lower bouds for the black-box complexty of partcular, more realstc, classes of problems (e.g., NIAH ad umodal fuctos). It s well kow that search spaces whch correspods to real world problems clude may symmetres (e.g., automorphsms of a TSP graph). Rowe, Vose ad Wrght [19, 20] vestgate ths scearo for GA. They formally defe a oto of structure the search space, ad study the cod- 1087

2 tos for whch mutato ad crossover respect the symmetres duced by the structure. I secto 3 we cosder the symmetres of the search space a smlar way, albet, usg dfferet otato. I secto 4 we geeralze ths oto (.e., symmetres the search space) for the case of ftess ladscapes. We defe the orbt of a fucto f w.r.t. the symmetres of the search space as a structural class of problems. We argue that ths s a atural way to group problems the black box scearo. The effcecy of metaheurstcs strogly depeds o the choce of represetato ad operators. Respectg the symmetres of the search space s a mportat property of a effcet search-operators fact, desgg such operators s, perhaps, the ma motvato for [19, 20, 13]. I secto 5 we defe a geerc model of radom search heurstcs whch explctly assumes that the search operators respect the structure of the search space. The motvatos for ths are twofold, frstly, ths s true for may metaheurstcs ( partcular, wheever the search operators are defed over the structure, they respect the structure). Moreover, eve f for some metaheurstcs (or represetatos) ths s ot the case at the preset, arguably, gve the actve, frutful research o prcpled desg, ths wll be the case the ear future. Our model s costructed such that t has detcal performace over all fuctos whch belog to the same structural class. The mplcatos of ths are twofold. Frstly, thaks to our way of groupg problems for the black box scearo o the bass of structure, t s ow possble to explore how classes of real-world problems coect to the black box scearo. We follow ths approach [3]. Secodly, a ufed realstc model for metaheurstcs ca be a mportat startg pot for the developmet of a meagful theoretcal framework. We dscuss some of the possble mplcatos of our model for future research secto 6. It s mportat to stress that by realstc we mea that we am to descrbe detal the commo propertes of exstg metaheurstcs. We do ot cosder, at ths stage, other mportat aspects of realstc search algorthm lke space restrctos. 2. MOTIVATIONS There s a gap betwee the theoretcal study of metaheurstcs ad the way that they are used practce. O oe had, metaheurstcs strogly deped o the structure of the search space, yet, formal models of black-box heurstcs make ot attempt to model ad use ths structure explctly, hdg t, stead, wth some probablty dstrbuto. O the other had, the extesve research doe the feld of ftess ladscapes (e.g., [16]) focuses o the structure aloe, gorg that t s meagful to study the features of a ladscape oly as log as the search operators are matched to the ladscape structure. I ths paper, we combe the otos of formal model of black-box heurstcs ad ftess ladscapes to oe tegrated framework. Ths provdes a atural settg where to defe ad study meta-heurstcs. As we argue secto 5, may pre-exstg metaheurstcs ft our framework. Furthermore, we gve some formal results that although easy ad tutve are very powerful ad geeral ad ca be exteded to much less obvous results. The cotrbuto of ths paper s, therefore, a chage perspectve o how to formally model meta-heurstcs. 3. STRUCTURE It s a commo cojecture (that has bee elaborated the lterature o NFLTs) that real-world problems have structure. Metaheurstcs explot ths structure to fd solutos effcetly (that s, wthout usg problem-specfc kowledge). The oto of structural search space was formally defed [19, 20]. Row, Vose ad Wrght cosder the stuato where there s a group of permutatos whch act o the search space. They defe, ths way, the may symmetres ofte arses combatoral search spaces ad study the codtos for whch mutato ad crossover respect these symmetres. I ths secto we follow a smlar approach, however, rather tha cosderg the automorphsms of a graphs we use the oto of metrc-spaces ad sometry trasformato. The eghborhood structure s ofte defed by a metrc fucto d : X X R that gves the dstace betwee ay two pots the search space X. The tuple (X, d) s also called a metrc space. Cosder all the trasformatos σ : X X. Some of them (the trval oe beg σ(x) =x) preserve the dstace relato betwee ay two pots the search space. That s, the dstace betwee the pots x, y for example equals the dstace betwee σ(x),σ(y). More formally: Defto 1. Let (X, d) be a metrc space. The trasformato σ d : X X s a dstace preservg trasformato (or a sometry) f: x, y X d(x, y) =d(σ d (x),σ d (y)) The sometry group s the set of all sometres uder fucto composto. We gve two examples of dstace preservg trasformatos. I the frst, we cosder the Eucldea N-space whch X s fte. I the secod, we cosder the Hammg dstace defed over a bary strg of sze. Note that the trasformato o a fte space ca be see as a permutato of X. Cosder the Eucldea N-space, whch s real N-space R N wth the Euclda dstace metrc, whch the dstace s: d(x, y) = x y,x,y R N. The traslato σ a(x) =x + a where a R N,sasometry over the metrc space (d, R N ). Naturally: x, y d(x, y) = x y = x + a y a = (x + a) (y + a) = d(σ a(x),σ a(y)) Ay cotuous search space, whe represeted o dgtal computer, s essetally fte. Sce we are terested to vestgate search algorthms whch ru o a dgtal computer we ca assume that the search space X, although qute large, as well as the space of all possble ftess values Y, s fte. For ths reaso we restrct our atteto to fte search spaces. Sce for a fte search space, traslato s essetally a permutato (e.g., a rotato), we wll use the term dstace preservg permutato. The Hammg dstace s defed for the bt-strg represetato the followg way: d H(x, y) = P δ(x y) where δ(x) =1fx s true, 0 otherwse. It s easy to show that the exclusve-or (XOR) operator s a dstace preservg permutato over the metrc space defed by (d H,X). 1088

3 Lemma 1. Let x 0 X. The permutato σ x 0 (x) x x 0 s a dstace preservg permutato. Proof. We eed to show that: x, y Xd H(x, y) =d H(σ x 0 (x),σ x 0 (y)). x, y d H(x, y) = = = = = δ(x y ) (x y ) (x (x 0 x 0) y ) (x x 0) (x 0 y )) (σ x 0 (x )) (σ x 0 (y )) = d H(σ x 0 (x),σ x 0 (y)) XOR s ot the oly way to defe sometry the bary search space. A secod way to do so s to permute some of the bt postos the strg sce the Hammg dstace s computed per bt, ths trasformato s sometrc as well. Icdetly, ay sometry of a fxed legth bary strg, ca be descrbed as a composto of a permutato ad a (XOR) mask (See [18] for more detals). Naturally, structural symmetres exst other combatoral spaces as well. [19, 20] dscuss ths extesvely ad gve several examples. 4. STRUCTURE AND FITNESS LANDSCAPE Whle sometry s defed over a metrc space {X, d}, a problem the black box scearo s defed by the trple {X, d, f}, wheref : X Y s the ftess fucto. The trple {X, d, f} s also kow as a ftess ladscape [17]. May attempts have bee made to characterze the propertes of ladscapes to dscrmate dffcult oes w.r.t. easy oes. Well-kow examples are solato, multmodalty, auto-correlato ad ftess dstace correlato. These methods try to characterze, oe way or aother, the coecto (correlato) betwee the ftess fucto ad the eghborhood structure. Isolato s a case whch the eghborhood of the optmum s characterzed by low ftess values. Multmodalty measures the umber of local optma. Auto-correlato ad ftess dstace correlato measure explctly the correlato betwee the ftess ad dstace fuctos. Naturally, the structure of a ladscape does ot deped o a ftess of a partcular soluto but rather o the relato betwee the dfferet solutos. A umodal ladscape s umodal regardless of the detty of the global optmum. Cosder for example the oemax problem: f(x) = P x. The global optmum for ths fucto s the strg of all oes. However, aturally, there are X dfferet problems whch have exactly the same structural characterstcs, amely: ), where xtrgt X spec- It s possble to f xtrgt (x) = P δ(x = xtrgt fes the detty of the global optmum. geeralze the oemax problem because t s easy to defe t terms of the relato betwee the ftess fucto ad P the eghborhood structure. I partcular, the ftess, δ(x = xtrgt )ofx cabewrtteas dh(xtrgt,x), that s, terms of the dstace fucto. But s t possble to geeralze other problems where ths relato s more complcated? Based o a dstace preservg permutato of a metrc space, we ca ow defe a dstace preservg permutato of a fucto. We argue that such a permutato preserves all the structural propertes of the ftess ladscape. Defto 2. Let (X, d) beametrcspace,σ d adstace preservg permutato ad Σ the sometry group. For ay f : X Y : 1. The permutato σ d f of f s the fucto σ d f : X Y defed by σ d f(x) =f(σ d 1 (x)). 2. The set F = {g σ d Σ,g = σ d f} s the structural class (or the orbt ) of f. A dstace preservg permutato of a fucto preserves the relatos betwee the ftess values ad the eghborhood structure. It smply make explct the may symmetres whch exst the space of all possble problems. We beleve that ths s a good way to group problems the black-box scearo. Whle, by defto, ths way preserves the structure of the ladscape, the followg we show explctly that the ftess dstace correlato (FDC), for example, as defed by [12], s the same for a fucto ad ay dstace preservg permutato of the fucto. Lemma 2. Let f be a fucto, σ d f a dstace preservg permutato of the fucto ad P 1 (f(x) f)(d(x,xtrgt) d) r(f) = P (f(x) f) P (d(x,xtrgt) d) the ftess dstace correlato of f. The: r(f) =r(σ d f) Proof. We cosder the FDC for oe global optmum measured o the etre search space. The permutato of a fucto does ot chage the values of the ftess fucto ad the two fuctos are defed over the same search space. Also, f = σ d f. So, the deomator the FDC formula s uaffected by the permutato. Let x trgt1 be the optmum for f ad x trgt2 = σ d (x trgt1) the optmum for σ d f. We eed to show: (f(x ) f)(d(x,x trgt1) d) = (σ d f(x ) f)(d(x,x trgt2) d) 1089

4 r(σ d f)= (σ d f(x ) σ d f)(d(x,x trgt2) d) = (reorderg the summato) (σ d f(σ d (x )) σ d f)(d(σ d (x ),x trgt2) d) = (by defto: σ d 1 f(x) =f(σ d (x))) (f(x ) σ d f)(d(σ d (x ),x trgt2) d) = by defto x trgt2 = σ d (x trgt1) ad, d(x,x trgt1) =d(σ d (x ),σ d (x trgt1)) hece: f(x ) σ d f)(d(σ d (x ),x trgt2) d) = f(x ) f)(d(x,x trgt1) d) =r(f) To coclude, let us go back to the example of oemax. It s easy to map the orgal problem, usg the xor permutato, to all the other fuctos that have smlar structure. Thus, the structural class F,whef s oemax (.e. the structural class of oemax), gves the same famly of fuctos as we prevously obtaed by specfyg explctly the fucto f(x) = d(x trgt,x). Smlarly, regardless of the complexty of the ftess fucto or eve wthout kowg the exact formulato of the ftess fucto we ca rgorously defe a class of problems whch have exactly the same structural propertes. I ths ext secto we show that ths mples detcal performace of ay metaheurstc. 5. SEARCH ALGORITHMS AND STRUCTURE Ay fucto f mplemeted a dgtal computer ca be cosdered as a mappg betwee two fte sets [25]. That s f : X Y,whereX ad Y are fte. I the most geeral case, a radomzed search heurstc ca be represeted as a mappg from a mult-set of prevously vsted pots to a ew (ot ecessarly uvsted) pot X. Droste, Jase ad Wegeer [7], [23] [24] suggested a formal defto (table 1), smlar to [25, 22], of a black-box algorthm. Ths defto geeralzes most, f ot all, exstg radomzed search heurstcs. The umber of queres (.e., ftess evaluatos) made utl a suffcetly good f opt s foud, s usually used order to evaluate the performace of the algorthm (see [25], [7] for further dscusso). As metoed the troducto, ths defto let to very useful NFLTs ad, thaks to Yao s mmax prcple, meagful bouds for dfferet classes of problems. However, the geeralty of ths model restrcts the possble cotrbuto of ay theoretcal result: the NFLTs are crtqued for ot applyg to the real-world scearo. The bouds gve by Droste et al. focus o the dstrbuto of staces of dfferet problems they do ot take to accout the aprorbases of typcal heurstcs desged for the black box scearo. To accout for ths, we wll modfy ths defto ad wll argue that our modfed verso s stll a proper geeralzato of may metaheurstcs. Usg ths prototype, we the Table 1: Black box algorthm 1. Choose some probablty dstrbuto p o S ad produce a radom search pot x 1 S accordgtop. Compute f(x 1). 2. I step t, stop f the cosdered stoppg crtera s fulflled. Otherwse, depedg o the propertes of I(t) =(x 1,f(x 1),..., x t 1,f(x t 1)) choose some probablty dstrbuto p I(t) o S ad produce a radom search pot x t S accordg to p I(t). Compute f(x t). prove that the expected performace of such algorthms o ay fucto take from a sometry group s the same. Iformally, smlarly to Droste et al. [6], we assume that ay reasoable search algorthm has o a pror preferece for specfc search regos ad t selects pots accordg to ther ftess value. However, ulke prevous approaches we also explctly assume that the algorthm decdes whch ew pots to sample o the bass of ther dstace from exstg pots. Before troducg the formal model, we beg by presetg some termology: Let X, Y be fte. A fucto s represeted by f : X Y. The mult-set I(t) = {(x,f(x ))} represets the pots that the algorthm has sampled (.e., the formato that the algorthm has) by tme t. We defe the fucto S(x, I(t)) = {(d(x, x ),f(x ))}. Ths fucto, gve x ad the multset I(t) returs a multset wth exactly the same umber of elemets as I(t) (ad wth same cardalty), but where each pot prevously sampled by the algorthm, x,sreplacedby the dstace d(x, x ) betwee x ad x.iotherwords, S(x, I(t)) descrbes the structural relatoshp betwee x ad the sets I(t). Iformally, the search algorthm has three phases: a selecto phase (select promsg pots from I(t)), a explorato phase (based o the selected pots - geerate ew pots) ad a updatg phase (decdg whch pots to keep I(t)). Table 2 gves a formal defto of a structural black-box algorthm - ote that for the sake of smplcty - we deote I(t) asi. Ths defto dffers, compared to [24, 23, 7], oe mportat aspect: the probablty dstrbuto used to geerate ew solutos p t s a fucto of tme ad, more mportatly, of S(x, I(t)) rather tha I(t). As a result t depeds solely o the ftess values ad structural/eghbourhood relatoshp betwee ew potetal samples ad the pots already sampled. Naturally, the questo ow s: s ths defto stll geeral eough to accout for exstg metaheurstcs? Blum ad Rol [2] gve a survey o the most mportat moder metaheurstcs. They dstgush betwee populato-based metaheurstcs ad metaheurstcs that use trajectory methods. We choose a represetatve metaheurstc of each class (geetc algorthm ad smulated aealg respectvely) ad show that our defto s a approprate geeralzato of t. As prevously metoed, our 1090

5 Table 2: Structural black box algorthm 1. Italze I(0) by choosg m radom search pots x 1,,x m X usg some pror probablty dstrbuto p 0 ad the computg the correspodg ftess values f(x ). 2. I step t, stop f the stoppg crtera are fulflled. Otherwse: (a) Choose some probablty dstrbuto p t = p(s(x, I(t)),t)o. (b) Produce radom search pots x 1,,x X by samplg the probablty dstrbuto p t. (c) Compute the correspodg ftess values f(x ). (d) Set I(t +1)=merge(I(t), {(x,f(x ))},t). Table 3: Smulated Aealg (SA) s GeerateItalSoluto() T T 0 whle termato codtos ot met do s PckAtRadom(N(s)) f (f(s ) <f(s)) the s s else Accept s as ew soluto wth probablty p(t,s,s) edf Update(T ) edwhle model s oly approprate for metaheurstcs wth search operators that respect the structure of the search space. As prevously metoed, gve the actve research doe prcpled desg, we beleve that operators whch do ot respect the structure (ad hece, are ot captured by our model) wll be replaced, evetually, wth operators whch do [19]. Let us start from trajectory methods. These are characterzed by a trajectory the search space. The search process ca be see as the evoluto dscrete tme of a dscrete dyamcal system. The algorthm starts from a tal state ad descrbes a trajectory the state space. A successor soluto s ether amog the eghborhood of the curret soluto or chose radomly [2]. Ths geeral descrpto makes t clear that at ubased trajectory methods are specfc staces of our structural black box search algorthm. As a example, we wll show explctly that Smulate aealg (SA) fts our defto. Smulate aealg (table 3) s perhaps oe of the most studed metaheurstc. It was oe of the frst to use a explct strategy to escape local mma. The fudametal dea s to allow the selecto solutos of worse qualty tha the curret soluto, wth a probablty whch decreases over tme. Let us see how SA ca be descrbed by our algorthm: the sze of the mult-set I s 1. The procedure GeerateItal- Soluto() s modelled step 1 of our algorthm. I order to see how the temperature parameter T s effectvely modelled, Table 4: Evolutoary Algorthm P GeerateItalPopulato() Evaluate(P) whle termato codtos ot met do P Select(P ) P Recombe(P ) P Mutate(P ) Evaluate(P ) P Select(P P ) edwhle we just eed to thk of t as a fucto of tme,.e., T (t). So, although the probablty of acceptg solutos ( the merge phase of our algorthm) depeds o the temperature, sce the temperature s a fucto of tme, the acceptace probablty s effectvely a fucto of tme too. The samplg probablty dstrbuto p(s(x, I(t)),t) s zero for all x such that the dstace betwee x ad the pot curretly I(t) s greater tha 1 (.e., x s ot a drect eghbour of the curret search pot). Otherwse, p(s(x, I(t)),t)sa costat. Populato based methods deal every terato wth a set of solutos rather tha a sgle oe. It s more dffcult to gve a geerc descrpto (lke the oe gve for trajectory methods) for ths kd of methods. However, for our argumet to hold, t s suffcet to demad that all the search operators are defed over the eghborhood structure. I partcular, we focus o perhaps the most studed populato based method the evolutoary algorthm (EA). There are several varatos of EAs (e.g., Geetc Algorthms, Geetc Programmg, Evolutoary Strateges, etc.). The algorthm gve table 4 s a typcal oe. I EAs the sze of the mult-set I s defed accordg to the populato sze. Smlarly to SA, GeerateItalPopulato() s modelled our frst step. The actual mplemetato of Select(P ) dffers from oe EA to aother. The dfferet selecto mechasms clude: touramet selecto (wth varous touramet szes), ftess proportoal selecto, rak selecto ad more. Despte the dffereces, all of them deped solely o the ftess of the solutos f, ot the correspodg x s. The probablty dstrbuto p(s(x, I(t)),t) s suffcet, therefore, to descrbe all of them. As opposed to SA, EAs there are two search operators: recombato ad mutato. Sce the two operators are appled sequece, t s suffcet to show that each of them ca be descrbed by a probablty dstrbuto based o S(x, I(t)) aloe. The mutato operator s defed smlarly to the search operator SA. That s, for each x X a mutato of a gve value s the probablty dstrbuto defed over the eghborhood B r(x). As we already argued for SA ths depeds oly o S(x, I(t)). It s more dffcult to aalyze the recombato operators. They are defed dfferetly for dfferet represetatos (e.g., bary-strg, permutato, sytactc-trees), ad eve for a gve represetato, usually, there s more tha oe way to defe recombato (e.g., for bary strgs we have oe-pot, two-pots, uform crossover, etc.). Moraglo ad Pol [13] troduced the oto of topologcal crossovers as a class of represetato depedet operators 1091

6 that are well-defed oce a oto of dstace over the soluto set s defed. Smply stated, topologcal crossovers produce offsprg o the le segmet betwee ther parets. Moraglo ad Pol showed how topologcal crossover geeralzes the oto of crossover for bary strgs [13], permutatos [14] ad sytactc trees [15]. Formally, a le segmet betwee x ad y s defed as: [x; y] ={z d(x, z) +d(z,y) =d(x, y)} A topologcal uform crossover UX s defed as the probablty dstrbuto that a soluto z wll be the offsprg of the parets x ad y: δ(z [x; y]) p UX(z x, y) = [x; y] The crossover s defed as a fucto of the segmet [x; y] whch tur, s a fucto of the dstace. Therefore, by defto, the samplg probablty dstrbuto s a fucto of dstaces oly. So, aga p(s(x, I(t)),t) ssuffcetto model ay kd of uform topologcal crossover. It follows that evolutoary algorthms that use uform topologcal crossover are staces of our structural black box algorthm as well. We showed that our algorthm ca descrbe explctly SA a represetatve trajectory algorthm ad a famly of EAs represetatve populato based algorthms. The ever creasg umber of dfferet metaheurstcs does ot allow us to prove that our model s applcable to all of them. However, we strogly beleve that, a smlar way, the algorthm defed table 2 ca accout for may other metaheurstcs. For example, t must deftely apply to local search. Also, we beleve t may apply to tabu search ad partcle swarm optmsers. But what s the advatage of usg ths defto? It follows from our model that ay algorthm that ca be descrbed by our defto has exactly the same performace o ay fucto whch belogs to the same sometry group. Our argumet s true by the defto of the structuralsearch-algorthm. However, we wll make t eve more geeral: Theorem 1. Let (X, d) be a metrc space, σ d adstace preservg permutato, f : X Y afuctoada, a structural search algorthm. Let P (A f) deotes ay performace measure defed as a fucto of I Y. The followg holds: σ dp (A f) =P (A σ d (f)) Proof. A problem the black box scearo s defed as a fucto of the trple {X, d, f}. However, by costructo, the mult-set S(x, I(t)) depeds oly o dstace ad ftess. Sce f ad σ d f are sometrcally somorphc, the expected performace (.e., the expected set of f s) of a structural search algorthm for both s the same. 6. DISCUSSION We am at gvg the most precse defto of a black box algorthm that s geeral eough to accout for as may metaheurstcs as possble. Naturally, t s possble to cosder alteratve models partcular, some mght clam that our model s ot geeral eough. We caot possbly prove that t s (there are smply too may metaheurstcs aroud). However, our model fts the vew of may researches who explore the propertes of the ftess ladscapes rather tha ay partcular metaheurstcs. Furthermore, the oly restrcto that our model has w.r.t. exstg models s the requremet for t to be ubased to ay rego of the search space. We beleve that ths restrcto, gve that a black box algorthm does ot have a pror kowledge about the problem, s qute reasoable. Ths paper has formalzed some tutve results whch may mght be aware of. The objectve, however, s to provde a ew perspectve whch search algorthms the black box scearo ca be studed. I ths secto we summarze two possble drectos whch we ted to follow future research. 6.1 Classes of problems for black box algorthms Classc algorthm theory s cocered wth algorthms whch are (1) desged to solve partcular problems ad (2) are proved to be effcet o these problems. Outsde the cotext of a problem, a problem-specfc algorthm has o meag. Black box algorthms, o the other had, are beleved to be geerc. They are defed out of the cotext of ay partcular problem. Ths s deed oe of the ma obstacles a theory for black-box algorthms has to overcome. The basc oto of a problem (or at least, the target problems) s ot well defed. As a cosequece, the NFLTs cosder for example, what may beleve to be rrelevat problems. O the other had, sce black-box algorthms are ot desged for ay specfc problem, t s temptg to thk about what Droste et al [6] call the oe-shot-scearo: real-lfe, a black box algorthm eeds to solve oly oe stace (or oe ftess fucto) o whch, however, t has o kowledge about. From the perspectve of classcal algorthm theory, however [6], ths s absurd o theory, t s possble to desg a algorthm whch sample, the frst step, the global optmum. The ma pot that we would lke to make s that classes of problems for the black box scearo are well defed, albet, ot the usual way: ay fucto f ca be assocated wth a structural class of fuctos o whch the algorthm s expected to perform the same. Whle ths mght have bee kow for some tme, t had o real mpact. For example, havg ths md, the oe-shot-scearo, does ot exst. Usg the xor operator as a sometry permutato (assumg a bary-hammg metrc space), the global optmum, for dfferet fuctos whch belog to the same structural class, ca be ay soluto X. Sce the algorthm s expected to have the same performace over the etre class, t has to be more sophstcated tha just pckg, o the frst tral, the global optmum. Ths has some mplcato eve whe tryg to aalyze the performace of a black-box algorthm o well-studed problems. It s commo computatoal complexty to dstgush betwee a problem (e.g., TSP) ad a stace of a problem (e.g., ay cofgurato of ctes ad dstaces). The theory s cocered wth the expected performace of the algorthm o staces of the problem. Its performace, aturally, o ay other problems s of o relevace. Whle ths s certaly the case for problem-specfc algorthms t s ot the case for black box algorthms. For each (relevat) stace of a problem, a black box algorthm s desged to perform exactly the same o the structural group of ths stace. Ths s the case rrespec- 1092

7 tvely of whether the fuctos from the structural class are staces of the problem or ot. Whe aalyzg the performace of a black box algorthm o a certa problem, oe must take to accout, ot oly all the possble staces, but also, the set of all possble structural classes of these staces. As metoed before, dfferet approaches to the study of black-box algorthms focus o dfferet aspects. The ftess ladscape approach studes the structural propertes of a problem, wthout cosderg ay partcular algorthm. O the other had, formal models ted to hde the structure sde a probablty dstrbuto. Our model ca brdge betwee the two approaches. For example, Droste at el.[7] used Yao s mmax prcple to prove lower bouds for black box algorthms. The basc dea s to use the expected ru tme (w.r.t. a probablty dstrbuto o problem staces) of a determstc search algorthm order to derve lower bouds for radomzed algorthms. Usg our model, t s possble to study the effect of the oto of structural classes (whch coects to the study of ftess ladscape) o the probablty dstrbuto defed over the problem staces. We beleve that ths ca gve tghter bouds to the performace of black box algorthms. We pla to vestgate ths future research. As a fal remark, for the NFLTs, ay algorthm, the so called black-box algorthms cluded, has to make a a pror assumpto about the problem order to perform better tha radom search. It s ofte clamed that black box algorthms make the rght assumpto about real-world problems. We would lke to emphasze, that we do ot make ay such clam. That s, we do ot argue that black box algorthms are partculary good for the real world scearo. Moreover, whle we argue that the performace over ay structurally detcal fucto s smlar we do ot, at ths stage, dstgush betwee easy structure ad hard oe. We do ot kow whether the assumpto black box algorthms make s good or bad we smply state that they make a assumpto, ad that they should be studed accordgly 1. To gve addtoal examples, oe ca ask how the sze of the sometry group chages for dfferet represetatos. Ths s equvalet to the umber of staces (.e., sometry permutatos of a fucto) a problem (.e., a fucto wth uque structural propertes) the black box scearo has. Smlarly, oe ca vestgate how ths sze chages as the sze of the search space creases. 6.2 Etropy ad Search The smple tuto whch we formalzed ths paper suggests that the key strategy of ay black box algorthm s (a) focus o solutos wth hgh ftess ad (b) whe samplg a ew pot, try to select oe accordg to ts dstace from such solutos. Ths strategy wll fal ether of the two scearos (1) solutos, more ofte tha ot, have smlar ftess (e.g., the NIAH) ad hece most of the choces, as to whch solutos to select, are doe radomly, ad (2) solutos have dfferet ftess values, however, the oto of proxmty s ot selectve eough (e.g., fully coected graph all the solutos the search space have the same dstace from ay pot. Proxmty that case s ot a good method to select whch pot to sample ext). 1 Ths s aturally studed the cotext of ftess ladscapes. However, ths paper we suggest a dfferet approach, outsde of that cotext. It s possble to characterze, formally, how radom the search s expected to be w.r.t. these two scearos. At ths stage we caot go to full, formal, detals (t s stll work progress), however, ma tuto s ths: Let us restrct our atteto to search algorthms that, at each stage, have a populato of sze k, of whch, the selecto phase, s are selected, ad accordgly k ew solutos (.e., a ew populato) are geerated. The etropy, or ucertaty the way the algorthm searches ca be defed, separately for the two stages. I the frst stage, gve the ftess of the k solutos (.e., {f(x 1),..., f(x k )}), let the probablty dstrbuto Pf k defe the probablty of each soluto (x ) to be selected. The etropy of the partcular set ca be defed as follows: S s = P k P f k (x )log(pf k (x )). If the solutos are selected uformly at radom, the etropy s maxmal ad the search wll be essetally radom. The expectato of ths for all possble sets of k ftess values, wll gve us the etropy of the selecto phase (that s, the extet to whch solutos are pcked radomly.e., rrespectvely of ther ftess value). Smlarly, oce s pots are selected, oe ca calculate the probablty dstrbuto of pckg ay other pot the search space. The probablty of a pot x to be selected depeds o ts dstaces from the s selected pots, ad hece, smlarly to the prevous step, the etropy for a partcular set of s pots s defed (assumg that I(t) represets our set of s selected pots): S e = P X p S(x,I(t))(x )log(p S(x,I(t)) (x )). The etropy of the explorato phase s, smlarly, the expectato of ths for all possble sets of s pots (that s, the extet to whch the algorthm chooses to sample ew solutos a radom way). It s ot trval to defe a geerc way such sets or, moreover, gve such sets, to fd the correspodg probablty dstrbutos (Pf k,p S(x,I(t)) ). However, oce these ssues are resolved t wll be possble to compare dfferet algorthms accordgly. Moreover, whle S e depeds solely o the metrc-space (.e. s the same rrespectvely of a partcular ftess fucto), S s depeds also o the ftessdstrbuto of the fucto. Cosderg the NIAH, for example, rrespectvely of the selecto mechasm (or, selecto testy [1]) the etropy wll be maxmal (.e. gve a set of solutos wth the same ftess the algorthm wll select oe of them, uformly at radom). A smlar approach,.e. measurg the formato cotet of a ftess-fucto was suggested [4]. 7. CONCLUSION The oto of sometry, sometrc somorphsm or cogruece mappg was studed as a atural cosequece, mathematcs ad geometry to the defto of metrc spaces. The otos of sometrc trasformato captures the may symmetry propertes of all the ways to combe a metrc space wth a ftess fuctos. Whle the NFLTs cosder Y X possble ftess fuctos t s clear that there are much less fuctos wth a uque structure. Ths s the oly reasoable way to thk of structure. I ths paper we formally defed a oto of a structural class of problems. We defed accordgly a model of structural search algorthms. We proved, by costructo, that ay structural search algorthm s expected to have detcal performace o ay fucto whch belog to the same sometry group (.e. the same structural class). We showed 1093

8 (usg smulated aealg ad evolutoary algorthms as examples) that our model s lkely to represet may exstg metaheurstcs. Thus, we have ufed the owadays ma approaches to the study of meta-heurstcs: o the oe had, the model cosdered the NFLTs [25] ad smlarly by Droste, Jase ad Wegeer [23, 24, 7] ad o the other had, the oe mplctly cosdered by the study of ftess ladscapes [17, 16]. Establshg a oe-trval model of metaheurstcs allows a extesve explorato (whch s oe of our mmedate objectves) of the geerc behavor of such algorthms. Some of the mmedate areas of future research were cosdered secto 6. The class of structural problems s of partcular terest the cotext of combatoral optmzato (CO). The represetatos of may CO problems s relatvely well uderstood. Drawg the le betwee classes of CO problems ad the class of structural problems defed here, may shed mportat lght o the potetal of metaheurstcs for combatoral optmzato. We make a frst step towards ths drecto [3]. 8. ACKNOWLEDGMENT Ths work was supported by the Aglo-Jewsh-Assocato ad the Harold Hyam Wgate Foudato. The authors would lke to thak Alberto Moraglo ad Joatha Row for useful commets. 9. REFERENCES [1] T. Blckle ad L. Thele. A comparso of selecto schemes used evolutoary algorthms. Evolutoary Computato, 4(4): , [2] C. Blum ad A. Rol. Metaheurstcs combatoral optmzato: Overvew ad coceptual comparso. ACM Comput. Surv., 35(3): , [3] Y. Boreste ad R. Pol. Structure ad metaheurstcs. GECCO [4] Y. Boreste ad R. Pol. Iformato ladscapes. I GECCO 05: Proceedgs of the 2005 coferece o Geetc ad evolutoary computato, pages , New York, NY, USA, ACM Press. [5] J. C. Culberso. O the futlty of bld search: A algorthmc vew of o free luch. Evolutoary Computato, 6(2): , [6] S. Droste, T. Jase, ad I. Wegeer. Optmzato wth radomzed search heurstcs - the (a)fl theorem, realstc scearos, ad dffcult fuctos. Theor. Comput. Sc., 287(1): , [7] S. Droste, T. Jase, ad I. Wegeer. Upper ad lower bouds for radomzed search heurstcs black-box optmzato. Electroc Colloquum o Computatoal Complexty (ECCC), (048), [8] T. M. Eglsh. Evaluato of evolutoary ad geetc optmzers: No free luch. I Evolutoary Programmg, pages , [9] T. M. Eglsh. O the structure of sequetal search: Beyod o free luch. I J. Gottleb ad G. R. Radl, edtors, EvoCOP 2004, Combra, Portugal, Aprl 5-7, 2004, Proceedgs, volume 3004 of Lecture Notes Computer Scece, pages Sprger, [10] P. Gruwald ad P. Vtay. Shao formato ad kolmogorov complexty. IEEE Trasactos o Iformato Theory, I Revew. [11] C. Igel ad M. Toussat. A o-free-luch theorem for o-uform dstrbutos of target fuctos. Joural of Mathematcal Modellg ad Algorthms, 3(4): , [12] T. Joes ad S. Forrest. Ftess dstace correlato as a measure of problem dffculty for geetc algorthms. I Proceedgs of the 6th Iteratoal Coferece o Geetc Algorthms, pages , Sa Fracsco, CA, USA, Morga Kaufma Publshers Ic. [13] A. Moraglo ad R. Pol. Topologcal terpretato of crossover. I K. Deb et al. edtors, Geetc ad Evolutoary Computato - GECCO 2004, Geetc ad Evolutoary Computato Coferece, Seattle, WA, USA, Jue 26-30, 2004, Proceedgs, Part I, volume 3102 of Lecture Notes Computer Scece, pages Sprger, [14] A. Moraglo ad R. Pol. Abstract geometrc crossover for the permutato represetato. IEEE Tras. Evolutoary Computato, page submtted, [15] A. Moraglo ad R. Pol. Geometrc ladscape of homologous crossover for sytactc trees. I CEC (1), pages , [16] P.F.Stadler ad C.R.Stephes. Ladscapes ad effectve ftess. Commets Theor. Bol., 8: , [17] C. M. Redys ad P. F. Stadler. Combatoral ladscapes. SIAM Rev., 44(1):3 54, [18] J. Rowe, D. Whtley, L. Barbulescu, ad J.-P. Watso. Propertes of gray ad bary represetatos. Evol. Comput., 12(1):47 76, [19] J. E. Rowe, M. D. Vose, ad A. H. Wrght. Group propertes of crossover ad mutato. Evol. Comput., 10(2): , [20] J. E. Rowe, M. D. Vose, ad A. H. Wrght. Structural search spaces ad geetc operators. Evol. Comput., 12(4): , [21] C.Schumacher,M.D.Vose,adL.D.Whtley.The o free luch ad problem descrpto legth. I L. Spector et al. edtors, Proceedgs of the Geetc ad Evolutoary Computato Coferece (GECCO-2001), pages , Sa Fracsco, Calfora, USA, 7-11 July Morga Kaufma. [22] M. D. Vose. The Smple Geetc Algorthm: Foudatos ad Theory. MIT Press, Cambrdge, MA, USA, [23] I. Wegeer. Towards a theory of radomzed search heurstcs. I B. Rova ad P. Vojtás, edtors, MFCS, volume 2747 of Lecture Notes Computer Scece, pages Sprger, [24] I. Wegeer. Radomzed search heurstcs as a alteratve to exact optmzato. I W. Lesk, edtor, Logc versus Approxmato, Essays Dedcated to Mchael M. Rchter o the Occaso of hs 65th Brthday, volume 3075 of Lecture Notes Computer Scece, pages Sprger, [25] D. Wolpert ad W. G. Macready. No free luch theorems for optmzato. IEEE Tras. Evolutoary Computato, 1(1):67 82,