Understanding Elementary Landscapes

Transcription

1 Understanding Elementary Landsapes L. Darrell Whitley Andrew M. Sutton Adele E. Howe Department of Computer Siene Colorado State University Fort Collins, CO 853 ABSTRACT The landsape formalism unites a finite andidate solution set to a neighborhood topology and an objetive funtion. This onstrut an be used to model the behavior of loal searh on ombinatorial optimization problems. A landsape is elementary when it possesses a unique property that results in a relative smoothness and deomposability to its struture. In this paper we explain elementary landsapes in terms of the expeted value of solution omponents whih are transformed in the proess of moving from an inumbent solution to a neighboring solution. We introdue new results about the properties of elementary landsapes and disuss the pratial impliations for searh algorithms. Categories and Subjet Desriptors I..8 [Artifiial Intelligene]: Problem Solving, Control Methods, and Searh General Terms Theory Keywords Combinatorial Optimization, Loal Searh. INTRODUCTION Grover [9] originally made the observation that the behavior of loal searh on some NP-Hard problems ould be modeled using a differene equation: a disrete analogue of the ontinuous wave equation that desribes wave propagation in physis. This disrete wave equation an be expressed as an eigendeomposition of the neighborhood transition matrix under a partiular loal searh operator for a ertain lass of problems. Stadler [] named this lass of problems elementary landsapes and energetially explored properties of these Permission to make digital or hard opies of all or part of this work for personal or lassroom use is granted without fee provided that opies are not made or distributed for profit or ommerial advantage and that opies bear this notie and the full itation on the first page. To opy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speifi permission and/or a fee. GECCO 8, July 6, 8, Atlanta, Georgia, USA. Copyright 8 ACM /8/7...$5.. landsapes. Exept for several notable examples, little researh and exploration has been onduted for some time on this lass. There is also a pereption that elementary landsapes are in fat quite omplex, or at least that the tools needed to understand elementary landsapes are reondite. This paper has two goals. One goal is to explain and understand elementary landsapes from a simpler perspetive. In pratie, the wave equation disovered by Grover is a onsequene of the deomposability of omponents that make up a solution. In many ases, onstraints in the relationships among these omponents within the neighborhood of an inumbent solution fore Grover s equation to hold. The other goal of this paper is to show that this simplified view permits new insights and intuitions about elementary landsapes. We illustrate this by introduing several new proofs about properties of elementary landsapes. The remainder of the paper is organized as follows. In the next setion, we explain the relationship between objetive funtion omponents that hold on many well-studied problems that ause the wave equation to be obeyed. We introdue this relationship in terms of expeted value of the evaluation of solutions seleted uniformly at random from the neighborhood of a fixed point. We derive the wave equation for symmetri TSP using -opt. In Setion 3, we disuss some salability impliations of the wave equation and show that the expeted hange in value an always be bounded. In Setion 4 we prove some properties about plateaus and loal optima on elementary landsapes and disuss the impliations for searh algorithms. In Setion 5 we onlude the paper.. ELEMENTARY LANDSCAPES We introdue the wave equation in terms of the expeted value of the objetive funtion evaluation at a solution drawn randomly from the loal searh neighborhood. For a given instane to a ombinatorial problem, we have a set of all possible solutions whih we will refer to as the set of andidate solutions. The nature of the andidate solution set is dependent on the problem to solve. We define the objetive funtion f as a map f : R and without loss of generality assume we must find the element of that minimizes f. We define a neighborhood operator as a funtion N that maps elements of to elements of its power set N : P(). The neighborhood operator is determined by the heuristi searh method in question. We will only examine neighborhood operators that apply to searh proesses that keep a single andidate solution. Population based searhes

2 that allow reombination, i.e. sharing partial omponents of solutions, admit more omplex strutures [4]. In this paper we shall onern ourselves only with neighborhoods that are symmetri (i.e. y N(x) x N(y)) and regular (i.e. d = N(x) for all x and some onstant d). Barnes et al. [] have extended the notion of elementary landsapes to non-symmetri and non-regular neighborhoods. The landsape for a ombinatorial problem instane is defined by a triple (, N, f): its andidate solution set, the neighborhood operator whih imposes a onnetive struture on the andidate solution points, and the objetive funtion that assigns a value to eah point. This landsape formalism was originally introdued by Wright [6] in the ontext of evolutionary dynamis but an be extended to any funtion over a disrete domain with a orresponding neighborhood : the ontext in whih we are interested here. A landsape is elementary when the objetive funtion f is an eigenfuntion of the Laplaian of the graph indued by the neighborhood operator [9, ]. This property an be expressed more onretely as an equation relating the expeted value of the objetive funtion at a uniformly drawn random element from a andidate solution s set of neighbors. Throughout this paper, we will denote x as some fixed but arbitrary element of and y as a element of N(x) drawn uniformly at random. We will also denote as f the mean value of f over all solutions in. On an elementary landsape we have E[f(y)] = f(x) + k d ( f f(x)) () for some k whih is fixed for the entire landsape. Sine y is drawn uniformly at random, we also have E[f(y)] = f(z) () d z N(x) In other words, the expeted value of the objetive funtion evaluation of a neighbor y is always equal to the average value of the evaluation over all solutions z in the neighborhood.. Intraomponents and interomponents Several interesting onsequenes arise from Equation (). First, we investigate what relationships exist in pratie that allow this wave equation to hold. In most pratial elementary landsapes studied, the objetive funtion for a partiular andidate solution an be written as a linear ombination of a subset of a olletion of omponents. Examples of solution omponents are edge weights in graph problems suh as TSP or Min-Cut partitioning, or ollisions in onstraint satisfation problems suh as graph oloring. In suh problems, a andidate solution x speifies this subset for inlusion in the sum. Let C be a set of real valued omponents ( C is polynomial in problem size). Then there exists C x C suh that f(x) = C x We refer to the set C x as the intraomponents of a solution x, and the set C C x as the interomponents of x. When a loal searh algorithm moves from an inumbent solution x to a neighboring solution y N(x), an exhange of these omponents is made. In partiular, a subset of the intraomponents out C x is removed and a subset of the interomponents in C C x is added. In other words f(y) = f(x) + out in If we fix x arbitrary and let y be a uniform random move or mutation obtained by one appliation of the neighborhood operator ating on x, we an ompute an expeted value for the objetive funtion f(y) at y. " E[f(y)] = E f(x) # + out in " # " # = f(x) E + E out in We ompute the expeted value by uniformly sampling all the intraomponents for potential removal and all the interomponents for potential addition. A landsape thus satisfies Equation () when the following relationship holds for all x. " # " # E E = k d ( f f(x)) in out The existene of omponents that satisfy this relationship is a suffiient but not neessary ondition for a landsape to be elementary.. Case study: TSP under -opt A number of NP-hard optimization problems have been shown to satisfy Grover s wave equation. As with loal searh, the move operator is ritial. The following is an inexhaustive list of elementary landsapes.. The symmetri Traveling Salesman Problem under - exhange [9], -opt and 3-exhange, [5].. The antisymmetri Traveling Salesman Problem under -opt and -exhange [3] and the weakly-symmetri Traveling Salesman Problem [], and variants of the multiple Traveling Salesman Problem [6]. 3. The Min-Cut Graph Partitioning problem with a simple exhange of two verties aross the ut [9]. 4. Graph Coloring with the neighborhood being a hange in olor at any vertex in the graph [9]. 5. Weight Partitioning over N objets. Eah objet is assigned a sign S i (, ) with the neighborhood being a hange in sign [9]. We will derive the expeted value wave equation for the TSP under -opt and examine some of the onsequenes. Suppose G(V, E) is a omplete weighted undireted graph on n verties (ities). Every vertex pair v i, v j V has an edge e i,j E. Let w i,j be the weight (or ost or distane) assoiated with edge e i,j. The solution spae thus is omprised of all valid Hamiltonian yles in G. In partiular a given x speifies a permutation on n points (x, x,..., x n ). Without loss of generality we fix the home ity at n.! n f(x) = w xi,x i+ + w xn,n + w n,x i=

3 (,,3,4,5) (,3,,4,5) Figure : A andidate tour (left) and one of its -opt neighbors (right) obtained by deleting edges (, ) and (3, 4) and adding edges (, 3) and (, 4). The orresponding permutation representation is shown below eah graph. This means that the evaluation funtion is deomposable into the osts assoiated with every possible edge. The omponent set C is exatly the set of real-valued edge weights w i,j. The intraomponents of a solution x are those edge weights used in the alulation of f(x); the interomponents are the edge weights not in the tour speified by x. In Figure a 6 ity TSP is shown. The tour x = ( ) is shown on the left and the tour y = ( ) is shown on the right. The edges {e,, e,3, e 3,4, e 4,5, e 5,6, e 6, } are the intraomponents of x. All possible edges that are not intraomponents of x are by definition interomponents of x. These are the lighter interior dashed lines in the figure. To ompute f, the average over all possible solutions, we also need to know the probability of eah edge ourring in any partiular solution. In this ase, that probability is uniform. Sine a tour length is n and there are n(n )/ possible edges it follows that f = n w i,j n(n )/ and by simple algebra f = n w i,j Now, given an inumbent solution x we need to ompute the expeted ost over all of the neighbors of x. The deletion of intraomponents of x and the addition of interomponents of x in effet onnet x to all of its neighborhood. To alulate the expeted value of a neighbor of x we examine the subset of intraomponents that must be removed ( out ) and the subset of interomponents that must be added ( in ). All intraomponents of x are uniformly removed when onstruting the set of neighbors of x; all of the interomponents of x are then uniformly added in when onstruting the set of neighbors of x. Under -opt, exatly two edges hange in eah neighbor. The onstrution of eah neighbor of x therefore requires deleting two edges from x. Thus, omponents (edges) are deleted with probability /n. Note that the sum of all the intraomponents is simply the tour length whih is given by f(x). Then the expeted value of the sum of intraomponents removed is E " out # = n f(x) The entire ost matrix has n(n )/ omponents and x has n omponents. This means that the number of interomponents is given by (n(n )/) n = n(n 3)/ Furthermore, the sum of all the interomponents is simply the sum of all the edges in the graph G minus the sum of all the intraomponents f(x) w i,j f(x) Sine two interomponents must be added, we have the expeted value of the sum of interomponents added " # E w i,j f(x) A n(n 3)/ in So, fixing the inumbent solution at an arbitrary x, the expeted value of a randomly seleted neighbor y N(x) an be written as " # " # E[f(y)] = f(x) E + E (3) out in = f(x) n f(x) + n(n w i,j f(x) A Beause the probabilities are uniform we an look at the onstrution of eah neighbor as an independent random event. Thus, two edges are deleted from x and two edges are inserted to reate a neighbor y. In reality, the edges that are deleted determine the edges that are inserted, but this an be ignored by looking at the neighborhood as an aggregate. Now we derive Equation (). Note that w i,j = n f and therefore rewriting Equation (3): E[f(y)] = f(x) n f(x) w i,j f(x) A n(n 3)/ = f(x) «n f(x) + n f f(x) n(n 3)/ = f(x) n f(x) + n n(n 3)/ f n(n 3)/ f(x) = f(x) (n 3)/ + n(n 3)/ f(x) + n n(n 3)/ f = f(x) n n(n 3)/ f(x) + n n(n 3)/ f = f(x) + n n(n 3)/ ( f f(x)) = f(x) + k d ( f f(x))

4 Thus we have Equation () with k = n and d = n(n 3)/. Here d is, of ourse, by definition the neighborhood size for lassi -opt. Note that when f(x) < f, the seond term is always positive and E[f(y)] is always greater than f(x). On the other hand, when f(x) > f the seond term beomes negative and the expeted value of a random neighbor is less than f(x). 3. SCALABILITY Sine we an exatly ompute E[f(y)] for a randomly seleted neighbor y of x we an also ask a fundamental question about salability. How does the differene between f(x) and E[f(y)] sale with problem size? We an prove using simple algebra that the differene sales in a linear fashion. This is intuitive: for large problems, f(x) and E[f(y)] are loser to eah other relative to f and the hange dereases linearly as a funtion of problem size. In any one problem, the differene between f(x) and E[f(y)] grows smaller as we approah f and inreases as we move away from f. To understand this better, assume that a TSP instane with n verties is normalized by f/n. In effet, this normalization is suh that the average edge length is. Thus, the normalized average solution is = n. This also lets us f f/n ompute bounds on f(x) and E[f(y)] relative to a normalized edge length of. Assuming we are looking for the best (minimal) solutions, onsider the ase where f(x) < E[f(y)] < f To better understand the differene E[f(y)] f(x) we will use a normalization of the ost matrix suh that f = n = P n w i,j n(n )/ w i,j n(n )/ = Or more onisely, f = n w i,j = We an then ompute the differene between E[f(y)] f(x) and obtain the following bound. E[f(y)] f(x) = f(x) + n n(n 3)/ ( f f(x)) f(x) n = (n f(x)) n(n 3)/ n = n(n 3)/ n n n(n 3)/ f(x) = n (n 3)/ n f(x) (n 3)/ n «n = f(x) «n 3 n As f(x) approahes zero relative to f = n the quantity E[f(y)] f(x) is bounded by (whih for large n n n 3 is approximately.); there are of ourse real limits on how small f(x) an be. Around f(x) = n/, the normalized quantity E[f(y)] f(x) is approximately.. As f(x) approahes the value f = n the differene between E[f(y)] f(x) approahes. Tables and report numerial alulations of f(x) and E[f(y)] and their differenes varying both f(x) and n aording to Equation (). n f(x) E[f(y)] E[f(y)] f(x) Table : Calulations for E[f(y)] and f(x) < n varying f(x). n f(x) =.3 f E[f(y)] E[f(y)] f(x) Table : Calulations for E[f(y)] and f(x) < n varying n. To illustrate the effet using real data, we sampled solutions from symmetri instanes obtained from the TSPLIB library maintained by Gerhard Reinelt at Heidelberg University. On eah instane we omputed a normalization fator q as follows. q = n(n ) P w i,j We define the normalized objetive funtion as the weighted tour length sum multiplied by the normalization fator q. Clearly, the mean value of the normalized instanes is n. On eah normalized instane we sampled points below f and alulated the objetive funtion value of the sampled point, the theoretial expeted value as given by Equation () and exatly one uniform random sample of the neighborhood of eah sampled point. We report the satter plot of the theoretial predition vs. the atual sampled neighbor value for five seleted instanes in Figure. Eah seleted instane has a different size n. In Figure 3 we report the expeted differene vs. the sampled differene on these five instanes. Figure 4 shows this differene plot for every (normalized) symmetri TSP instane in the TSPLIB with points sampled per instane. The expeted value of solution differene is bounded below by zero and above by (in the limit as n ). This is marked on the plots by dotted lines. In these experiments, we are observing a sampling proess from a distribution that follows the theoretial expeted value given by Equation (). The plots show us the variane in the random variables we are sampling. 4. DISCUSSION There are a number of interesting observations about this lass of landsapes that impat loal searh. In our TSP ase study, we saw that < k = (n ) <. Does this d n(n )/ software/tsplib95/

5 ...5 E[f(y)] f(x) 8.5. f(y) 4.5 pr7.tsp pr4.tsp pr36.tsp pr44.tsp pr5.tsp Figure : Theoretial predition of neighbor value vs. sampled neighbor value for sampled sub-mean points on five (normalized) instanes in TSPLIB. 4 Figure 4: Expeted differene vs. sampled differene of sub-mean points and neighbors on all normalized instanes in TSPLIB. Dotted lines represent bounds on expeted value. Expeted value is bounded above by and below by..5. hold in general for other problem lasses? Barnes et al. [] define two families of elementary landsapes with respet to onstraints on the possible eigenvalues of the graph Laplaian. Smooth elementary landsapes obey the above onstraint suh that the value kd lies between and. On the other hand, on rugged elementary landsapes the value of k are not onstrained to the unit interval, and some rather d surprising results are a onsequene. We are unaware of any well-studied neighborhood operator and problem lass in the NP-hard set that orresponds to a rugged landsape. Thus our results apply only to smooth landsapes.. 4. Plateaus and loal optima The following observations were made by Codenotti and Margara [5]..5 E[f(y)] f(x) f(y) f(x) E[f(y)]. If f (x) < f then f (x) < E[f (y)] < f. If f (x) = E[f (y)] then f (x) = f = f (y) If f (x) > f then f (x) > E[f (y)] > f 4 4 f(y) f(x) Figure 3: Expeted differene vs. sampled differene of sub-mean points and neighbors on five normalized instanes. Dotted lines represent bounds on expeted value. Expeted value is bounded above by and below by. These results undersore Grover s [9] observation that all loal minima lie below the average funtion value of the searh spae. Furthermore, the results state that ertain types of plateaus annot exist on elementary landsapes. A plateau is a set P of andidate solutions in suh that for all a, b P, f (a) = f (b) and there is a path (a = x, x,..., xk = b) suh that xi+ N (xi ). Plateaus (also known as neutral networks) are strutural features that arise in many ombinatorial problems [8,,, 3]. Plateaus pose an interesting hallenge to loal searh sine they prelude gradient information about improving movement.

6 As Codenotti and Margara [5] originally pointed out, on an elementary landsape, if all neighbors of a andidate solution share the evaluation of that solution, then we say the landsape is flat: every andidate solution belongs to the same plateau (ondition above). However, the wave equation imposes some onstraints on plateaus in non-degenerate ases as well. Consider a solution x that belongs to a plateau P. Then every solution on P will have the same neighborhood expeted value. This is easy to see sine all solutions in P have an objetive funtion evaluation equal to f(x) and obey Equation (). Lemma. For a plateau P on a (non-flat) elementary landsape, if x P has only equal and disimproving neighbors, then there annot exist a solution z P with only equal and improving neighbors. Proof. Let x, z P for some plateau P. Suppose for ontradition that x has only (both) equal and disimproving neighbors and z has only (both) equal and improving neighbors. Let y x and y z be neighbors drawn uniformly at random from the neighborhood of x and z respetively. Sine f(x) = f(z) (by definition of plateau), we must have E[f(y x)] = E[f(y z)] sine Equation () holds. If we assume minimization, the average value of the neighbors of x must be stritly greater than than the average value of the neighbors of z whih ontradits the expeted value of a uniform random seletion is equal for both neighborhoods. Note that Lemma does not prevent a solution on the same plateau as x from having both improving and disimproving neighbors. However, in this ase, the evaluation of these neighbors must have the same expeted value as those in the neighborhood of x. We define a loal minimum as a solution x min suh that f(y) f(x min ) for all y N(x min ). A loal maximum is defined analogously. It is important to point out that this definition is distint from other definitions of loal extrema in landsapes that ontain plateaus (e.g. Frank et al. [8]). Grover [9] showed that, on elementary landsapes, if x min is a loal minimum and x max is a loal maximum, then f(x min ) f f(x max ). We assume that, for all x, all f(x) >. If this does not hold, the values of the objetive funtion an be shifted by a onstant value without violating Equation (). Let x min be a loal minimum. If we sample y N(x min ) uniformly at random, we an express the expeted value of y in terms of an average over the elements of N(x min) E[f(y)] = d (S eq + S gr ) (4) where S eq is the sum of the value of equal or neutral moves: those y N(x min) suh that f(y) = f(x min). S gr is the sum of the value of the neighbors of x min that are stritly greater in objetive evaluation: those y N(x min ) suh that f(y) > f(x). Lemma. Let a and b be two solutions on a plateau P with no improving neighbors. Let n eq (a) and n eq (b) denote the number of equal neighbors of a and b respetively. Without loss of generality, suppose n eq (a) n eq (b). Then, assuming minimization, the sum of all disimproving moves in the neighborhood of b is greater than or equal to the sum of all disimproving moves in the neighborhood of a. Proof. We first derive an expression for the number of equal moves in an arbitrary solution x min. Note that at x min, S eq = n eq(x min)f(x min). By Equations () and (4) we have d (Seq + Sgr) = f(xmin) + k d ( f f(x min)) S eq + S gr = df(x min ) + k( f f(x min )) n eq(x min)f(x min) + S gr = df(x min) + k( f f(x min)) n eq (x min ) = d + k( f f(x min )) f(x min ) n eq(a) n eq(b) = " S gr f(x min ) Now onsider the differene of neutral moves among a and b.» d + k( f f(a)) Sa gr f(a) f(a) d + k( f f(b)) f(b) = Sb gr S a gr f(a) Sb gr f(b) Where S a gr and S b gr denote the sum over all neighbors with stritly greater objetive evaluation for a and b respetively. Sine we assumed that f(x) for all x is positive and that n eq(a) n eq(b) is non-negative, we have and this proves the result. S b gr S a gr A similar result holds for all loal maxima on plateaus. 4. Impliations for searh An interesting onsequene of the expeted value version of the wave equation is that a loal searh algorithm on an elementary landsape an always ompute the expeted value of a random solution in its neighborhood without expanding a single solution in this neighborhood. Furthermore, we an expand a partial neighborhood and predit the expeted value of the remaining neighborhood. Suppose M N(x) is a subset of the neighborhood of a point x. Then the expeted value of a point r uniformly drawn from the remaining neighbors in N(x) an be written as E[f(r)] f(z) f(z) A d M z N(x) z M = d M d f(x) + k d ( f f(x)) # «! f(z) z M by Equations () and (). Let the expeted improvement be D(r; x) = E[f(r)] f(x) = d M f(x) d f(x) + k «d ( f f(x)) z M f(z) This is the expeted differene in value if we move (or mutate) at uniform random to an element in the remaining neighbor set N(x) M. Now suppose x and x are two solutions in. Furthermore, suppose M and M are two partial expansions!

7 (.977)n (.894)n x f(x) = (.895)n (.835)n (.8965)n (.8)n Expeted value over entire neighborhood Average value of partial neighborhood x Expeted value of remaining neighborhood f(x) = (.884)n (.96)n n = 7 Figure 5: A stylized sketh of partial and whole neighborhood relationships for two solutions x and x. The average evaluation of the partial neighborhood of x is worse than x ; the expeted evaluation of remaining neighbors of x are better than those of x. Numerial data were generated using two random solutions from (normalized) 7-ity TSP gr7.tsp instane. of the neighborhoods of x and x. If r and r are randomly drawn elements from the remaining neighborhoods N(x ) M and N(x ) M, then we an ompute the expeted values of r and r above and the expeted improvement D(r ; x ) and D(r ; x ) for eah. It is learly more promising to ontinue expanding the neighborhood orresponding to the better of the two expeted improvements (i.e. smallest, if we are minimizing): this is more likely to yield a better hange in value. This idea is illustrated in Figure 5 where the partial neighborhood that results in an average value worse than the expeted value of the entire neighborhood yields a better expeted value over the remaining neighborhood. On an elementary landsape, neighborhoods an be onsidered rather loalized. From the omponent point of view, neighbors are very similar in struture to the urrent inumbent solution. Neighbors also have similar objetive funtion values (on average). The neighborhood does not support any mehanism that allows sampling in more distant parts of the searh spae. This makes loal searh on these landsapes fundamentally myopi. It should be pointed out this is not true of all forms of loal searh. The neighborhoods used by Gray- and binary-oded representation of parameter optimization problems do a better job of systematially globally sampling in distant parts of the searh spae. These neighbors are used by many forms of geneti algorithms. Pattern Searh is also better at systematially globally sampling in distane parts of the searh spae. Stadler [3] showed that a landsape (with a symmetri neighborhood operator) is elementary if and only the time series generated by a random walk on the landsape using transitions defined by the neighborhood operator is an AR() proess: an observation that was later generalized by Dimova et al. [7]. This observation, along with Grover s results about loal extrema lying above or below the mean solution value, imply that elementary landsapes desribe a lass of relatively smooth and strutured problems. When ompared to other onventional ombinatorial problems, the TSP will be omparatively well-behaved. As Equation () predits, andidate solutions in the TSP will have a onstant relationship with the average neighborhood value (with respet to the mean). This is not neessarily a pervasive feature in ommonly studied ombinatorial problems. Consider the mean-entered ratio between the evaluation of an arbitrary solution x and its average neighborhood value. f(x) R(x) = f (/d) P y N(x) f(y) f Suppose x lies on an elementary landsape. Equations () and () give us R elem (x) = f(x) f f(x) + k d ( f f(x)) f = d d k Thus the ratio is onstant for landsapes that obey Equation (). To ontrast this relationship empirially with other ombinatorial problems, we seleted an instane of the permutation flowshop sheduling problem (FSP) under the shift neighborhood [5], and the asymmetri (ATSP) under the exhange neighborhood. On eah landsape, we generated random solutions and alulated the value of R(x), Equation (5), for eah solution x. The values are plotted in Figure 6 for ar3.fsp: a 5 flowshop problem instane originally published by Carlier [4] and ontained in the online OR-LIBRARY. the pr5.tsp and ali535.tsp (symmetri) TSP instanes and the ft53.atsp ATSP instane from TSPLIB. The ratio is onstant for the TSP problems, but varies dramatially for the ATSP and FSP problems. We an therefore assume that searh algorithms that perform ompetitively on TSP are likely to be exploiting the partial deomposability of the landsape. On the one hand, this implies that suh an algorithm may easily be adaptable to other well known elementary problems. On the other hand, searh algorithms that are benhmarked on elementary problem lasses may not generalize well to omposite, non-elementary landsapes. 5. CONCLUSION The elementary property of ertain landsapes introdues several interesting onstraints on the behavior of loal searh algorithms. Landsapes with this property tend to be relatively smooth when ontrasted to other ombinatorial optimization problems with well-studied loal move operators. In this paper we have explained the elementary property in terms of neighborhood sampling and observed some of its onsequenes on real world instanes. The onstraints imposed by the wave equation preludes the existene of ertain plateau strutures, and fores ertain relationships between loal optima with the same evaluation. The ele- (5)

8 R(x) pr5.tsp ali535.tsp ft53.atsp ar3.fsp samples Figure 6: Ratio R(x) of mean-entered objetive and mean-entered average of neighborhood objetive for random solutions eah on two elementary TSP instanes, an ATSP instane, and a flowshop sheduling instane. mentary property also allows us to make preditions about the value of partial or full neighborhoods during searh. The Traveling Salesman Problem is widely used as a benhmark to testing new heuristi searh algorithms. But, there are many properties of TSP that make it unlike many other ombinatorial optimization problems. The wave equation and the assoiated haraterization by onstituent solution omponents presented in this paper make it lear that the TSP is in some sense partially deomposable. This allows for the partial update of the ost funtion. It also means that near optimal solutions an be found by methods that exploit the partial deomposability of the problem. Any method that heuristially exploits information about the individual omponents of the ost matrix may be a risk of over speialization. The results may not generalize well beyond the TSP, or at least beyond the family of elementary landsapes. 6. ACKNOWLEDGMENTS This researh was sponsored by the Air Fore Offie of Sientifi Researh, Air Fore Materiel Command, USAF, under grant number FA The U.S. Government is authorized to reprodue and distribute reprints for Governmental purposes notwithstanding any opyright notation thereon. [3] L. Barnett. Netrawling optimal evolutionary searh with neutral networks. In Proeedings of the Congress of Evolutionary Computation (CEC), pages 3 37, Seoul, Korea, May. [4] J. Carlier. Ordonnanements a ontraintes disjontives. R.A.I.R.O. Reherhe operationelle/operations Researh, :333 35, 978. [5] B. Codenotti and L. Margara. Loal properties of some NP-omplete problems. Tehnial Report TR 9-, International Computer Siene Institute, Berkeley, CA, 99. [6] B. Colletti and J. W. Barnes. Linearity in the traveling salesman problem. Applied Mathematis Letters, 3(3):7 3, April. [7] B. Dimova, J. Wesley Barnes, and E. Popova. Arbitrary elementary landsapes & ar() proesses. Appl. Math. Lett., 8(3):87 9, 5. [8] J. Frank, P. Cheeseman, and J. Stutz. When gravity fails: Loal searh topology. Journal of Artifiial Intelligene Researh, 7:49 8, 997. [9] L. K. Grover. Loal searh and the loal struture of NP-omplete problems. Operations Researh Letters, :35 43, 99. [] C. M. Reidys and P. F. Stadler. Neutrality in fitness landsapes. Applied Mathematis and Computation, 7:3 35,. [] A. Solomon, J. W. Barnes, S. P. Dokov, and R. Aevedo. Weakly symmetri graphs, elementary landsapes, and the tsp. Appl. Math. Lett., 6(3):4 47, 3. [] P. F. Stadler. Toward a theory of landsapes. In R. Lopéz-Peña, R. Capovilla, R. Garía-Pelayo, H. Waelbroek, and F. Zertruhe, editors, Complex Systems and Binary Networks, pages Springer Verlag, 995. [3] P. F. Stadler. Landsapes and their orrelation funtions. Journal of Mathematial Chemistry, : 45, 996. [4] P. F. Stadler and G. P. Wagner. Algebrai theory of reombination spaes. Evolutionary Computation, 5(3):4 75, 997. [5] É. Taillard. Some effiient heuristi methods for the flow shop sequening problem. European Journal of Operations Researh, 47:65 74, 99. [6] S. Wright. The roles of mutation, inbreeding, rossbreeding, and seletion in evolution. In Proeedings of the Sixth Congress of Genetis, volume, REFERENCES [] L. Barbulesu, A. E. Howe, L. D. Whitley, and M. Roberts. Understanding algorithm performane on an oversubsribed sheduling appliation. Journal of Artifiial Intelligene Researh, 7:577 65, De 6. [] J. W. Barnes, B. Dimova, and S. P. Dokov. The theory of elementary landsapes. Applied Mathematis Letters, 6(3): , April 3.