Criticality and Parallelism in Combinatorial Optimization

Size: px

Start display at page:

Download "Criticality and Parallelism in Combinatorial Optimization"

Douglas Dean
5 years ago
Views:

Criticality and Parallelism in Combinatorial Optimization William G. Macready Athanassios G. Siapas Stuart A.

represent the views of the Santa Fe Institute.

1 Criticality and Parallelism in Combinatorial Optimization William G. Macready Athanassios G. Siapas Stuart A. Kauffman SFI WORKING PAPER: SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder. SANTA FE INSTITUTE

2 Criticality and Parallelism in Combinatorial Optimization William G. Macready Athanassios G. Siapas y Stuart A. Kauman Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM y Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA June 2, 1995 Abstract We demonstrate the existence of a phase transition in combinatorial optimization problems. For many of these problems, as local search algorithms are parallelized, the quality of solutions rst improves and then sharply degrades to no better than random search. This transition can be successfully characterized using nite-size scaling, a technique borrowed from statistical physics. We demonstrate our results for a family of generalized spin-glass models and the Traveling Salesman Problem. We determine critical exponents, investigate the eects of noise, and discuss conditions for the existence of the phase transition. 1

3 Optimization tasks are common and very often uncommonly dicult. In most areas of science and engineering, from free energy minimization in physics to prot maximization in economics, the need to optimize is ubiquitous. In light of the importance and diculty of optimization, much eort has gone into developing eective algorithms for nding good optima. The methods of simulated annealing [1], genetic algorithms [2], and taboo search [3] are three of the most popular techniques, inspired by ideas from statistical mechanics, evolutionary biology, and articial intelligence respectively. All of these methods rely in part on constructing improved solutions by applying a local operator to a population of candidate solutions. Good solutions result from the accumulation of many benecial local modications applied one after another. An obvious speed up in the algorithm's performance can be obtained if we apply many local modications in parallel. Despite the promise that parallel algorithms hold, they have received less attention and little is known about them [4]. In this report we investigate the eects of parallelizing local search for combinatorial optimization. We demonstrate that for a wide class of search techniques increasing parallelism leads to better solutions faster. But only up to a certain point. At some degree of parallelism the quality of solutions abruptly degrades to that of random sampling. This transition is sharp and will be shown to share many ofthecharacteristics of thermodynamic phase transitions. We compute critical exponents that characterize this phase transition using nite-size scaling, a technique borrowed from statistical physics, and demonstrate our results on two important problems: energy minimization on NK energy functions [5], and tour length minimization for the traveling salesman problem (TSP) [6]. The NK model, a generalization of spinglass models [7], was chosen as one of the few general models capable of generating tunably dicult optimization tasks. The TSP is perhaps the most famous and well studied combinatorial optimization task and is often used as a test-bed for new ideas. Both problems, though quite dierent, show remarkably similar behavior. The NK model [5] denes a family of energy functions over a discrete search space. The search space consists of all possible congurations s fs 1 ::: s n g of N variables. If eachofthevariables can takeanyofapossible values, the search space is of size A N.We conne our investigation to the case A =2,we call our variables spins, and let them take onthevalues 1 due to their similarity with Ising spins which arise in models of magnetism. Our 2

4 local modications for the NK model are single spin ips where s i!;s i. Each spin, s i,makes a contribution to the total energy dependent on its own state and the state of K other spins. These K other spins may be selected at random or according to some specied topology. The total energy of a conguration, fsg, is dened by: Efsg = 1 N NX i=1 E i (s i s i1 s ik ) (1) By analogy with spin glasses [7] the local energy contributions, E i (s i s i1 s ik ), for each ofthe2 K+1 local spin congurations, fs i s i1 s ik g, are chosen at random from a uniform distribution over [0,1). However, by specializing E i and the selection of the K neighbors we can investigate specic optimization problems, including spin-glasses, graph-coloring, number partitioning etc. The number of other spins, K, that each spin interacts with varies from 0 to N ; 1 and controls the ruggedness of the energy landscape. In the K = 0 limit the spins are independent and there is a single minimum. As K increases the number of conicting constraints increases leading to multiple local minima that can trap local search algorithms. At thek = N ; 1 extreme, every spin aects every other spin, energies of adjacent congurations are uncorrelated, and there are exponentially many local minima. This K 1 limit is analytically tractable and has been studied by many authors [8]. We focus on simulated annealing [1] as our representative local search algorithm. In simulated annealing, local modications are accepted according to the Metropolis criterion [9], a method for simulating the evolution of a physical system in a heat bath to thermal equilibrium. A modication is applied and the resulting change in energy, E, is computed. The modication is always accepted if it lowers the energy, and accepted with probability p(e) = exp(;e=t) if E>0. T is a temperature parameter and controls the fraction of uphill moves that are accepted at zero temperature, T = 0, only downhill moves are accepted. It is important to occasionally accept uphill moves (i.e. T >0) to prevent trapping on poor local minima where all local modications raise the energy. Simulated annealing derives its name from the annealing, or gradual lowering, of the temperature parameter, so that uphill moves are accepted with decreasing frequency as the temperature lowers. 3

5 0.50 K=30 K=12 K=8 K=6 K=5 Energy K= τ Figure 1: Energy reached at the end of 500 generations versus for N = 5000 T = 0 and several dierent K values. The search space is vast, of size Ageneration is dened to be N update attempts, an average N of which are simultaneous. The average over 30 randomly generated landscapes is plotted for each value. Beyond the obvious computational speedups of applying many local modications simultaneously, parallel local search also allows escape from poor local optima. Even when only greedy moves are accepted (T = 0), allowing for simultaneous spin updates introduces the possibility of \mistakes": if the energy of s i depends on s j and both spins simultaneously attempt to ip, s i may conclude that E <0. However, such a decision may be based on outdated information, since s i made its decision assuming s j didn't ip. If s j did ip the energy of s i may actually go up causing E >0. We parameterize the degree of parallelism by 0 < 1 which denotes the probability that a spin attempts to ip. If there are N variables, then on the average N of them are updating under the local operator at the same time. The! 0 limit corresponds to a sequential algorithm, which at 4

6 T = 0 will result in the system eventually trapping into (usually poor) local optima. Maximal parallelism is obtained for = 1whenallvariables update simultaneously. At this extreme, the system typically cannot converge since too many of the spins use outdated information. The transition between these two extremes is surprisingly sharp [10]. As the parallelism is increased from 0 the system nds better solutions because it becomes increasingly easier to escape poor local optima. If the energy landscape is smooth enough (K <5) the asymptotic energy decreases monotonically with achieving a minimum at complete parallelism (for example the K = 2 curve in gure 1). However, if K > 5 this benet is realized up to a certain degree of parallelism c,beyond which there is an abrupt degradation in performance (Figure 1). This abrupt degradation is associated with an order-disorder transition in the update dynamics. Above the transition where the energy remains high, spins continue to ip indenitely. Below the transition, reaching low energy congurations is accompanied by a freezing of the spins (at T = 0) to particular values. This motivates the denition of the following order parameter which distinguishes the behavior on the two sides of the transition: Let p i be the probability that at equilibrium the i th spin will ip if asked. This is 0 if the spin is frozen and 0.5 if the spin is ipping randomly. We dene the order parameter for the i th spin to be its entropy, S i = ;p i log 2 p i ; (1 ; p i )log 2 (1 ; p i ). The order parameter, S, fortheentire P system is dened N as the average entropy per spin, S = (1=N) i=1 S i. Plots of the order parameter for K = 6 near the transition can be found in Figure 2(b). The order parameter is close to 0 below c and rises sharply towards 1 above c. The sharpness of the transition, as well as the location of c, depend upon the size N of the system. We use nite-size scaling [11], a method from statistical physics in which the observation of how the critical point c (N) changes with the size of the system gives direct evidence for critical behavior at the transition. As the size of the system N increases, the transition sharpens and the transition point shifts according to: c (N) ; c (1) N ;1= (2) Figure 2(a) shows the results of the nite-size scaling analysis for K = 6. The empirical observation behind this analysis is that suciently close to the critical point, systems of all sizes are indistinguishable except for an 5

7 (a) (b) τ c (N) S N=200 N=400 N= N -1/ν τ (c) S(y) N=200 N=400 N= y Figure 2: (a) c (N) vs. N ;1=, where 1:300:04, c (1) 0:7230:001, computed by a nonlinear least squares t from a collection of (N c (N)) values. Here we dene c (N) tobethe value at which the entropy is0:5. (b) order parameter of K = 6 energy6functions with dierent N values. (c) rescaled order parameter curves plotted with respect to y = N 1= ( ; c )= c instead of. Note that all curves collapse onto a universal (N-independent) curve.

8 0.50 Energy T=0.0 T= T= T= T= T= τ Figure 3: Dependence of the transition on temperature. Here N =800 K = 12, the temperature T is xed for each curve, and the average over 50 runs of the energy found at the end of G = 150 generations is plotted. overall change of scale. By dening a rescaled parameter as y = N 1= ( ; c (1))= c (1) (3) the rescaled curves fall on a universal (N-independent) curve (Fig. 2(c)). The existence, position, and sharpness of the transition shown in Figures 1,2 depend not only on N and K, but also on the temperature, T,and running time, G, of the algorithm. Figure 3 demonstrates that the transition persists for small temperatures and broadens as we move to higher temperatures. The length of time, G, weallow the algorithm to run also aects the transition [12]. Deep in the ordered regime ( c ) convergence is extremely fast as low entropy (frozen) regions quickly come to dominate and rapidly spread throughout the system (Figure 4(a)). In the other extreme ( c ) asymptotic convergence is also fast as high entropy regions quickly come to 7

9 dominate (Figure 4(d)). Close to the transition, there is a ght between the low and high entropy domains, neither of which quickly dominates (Figures 4(b) and (c)). This phenomenon is reminiscent of the critical slowing down seen in Monte Carlo simulations of physical phase transitions. Near the transition both low entropy (violet) and high entropy (yellow-red) clusters alternately come to percolate across the lattice. We gain insight into the transition by making an annealed approximation in which the energy function is randomly assigned anew after each local local move. The transition persists in the annealed model and occurs at the same point, c (K). It is interesting that a further simplication, obtained by making a mean-eld approximation where uctuations in the energy between sites are ignored (i.e. all E i = E), completely eliminates the transition. The energy increases smoothly with when uctuations are ignored. The uctuations drive the transition in the language of physics the transition is noise-induced. This can be understood simply. As noted previously, performance degrades due to spins making decisions with outdated information when their local environment changes. If, in spite of the outdated information, the environment doesn't change, spin ips will decrease the energy. Such cases are likely to arise in locally low energy congurations. Even if numerous spins in the local environment are given the chance to ip, it is unlikely many of them will since they are already at such alow energy. For those spins that do ip, it is likely they will have acted benecially lowering the energy further. Low energy uctuations thus tend to nucleate regions of lower energy. Similarly, high energy uctuations nucleate higher energy regions. This behavior is evident in the clusters of Figure 4 where high entropy/energy regions are yellow andlowentropy/energy regions are blue. Below c low energy uctuations dominate, crystallizing larger and larger low energy regions so that all spins eventually \freeze" into a very low energy state. Above c high energy uctuations win out and the order seen below c melts. Thus far we have described an abrupt transition in a family of optimization problems as the degree of parallelism is varied. Qualitatively, at least, the forces driving the transition are not unique to the NK model. The transition is driven by overlapping applications of the local operator. The degree to which one application undoes the benecial eects of the other will determine how much parallelism can be aorded. We address the genericity of the transition by investigating a very dierent optimization task. 8

(a) (b) (c) (d) Figure 4: Rendering of the entropy elds for

The energy of each spin depends on the states of the

zero. The color of each spin represents the entropy

A rainbow color map is used with the violet end of the

10 (a) (b) (c) (d) Figure 4: Rendering of the entropy elds for a lattice of spins. The energy of each spin depends on the states of the nearest K = 12 spins in the lattice and the temperature is zero. The color of each spin represents the entropy computed over the last 40 of a total of 80 generations. A rainbow color map is used with the violet end of the spectrum corresponding to low entropy. The images correspond to (a)9 = 0:43, (b) = 0:46, (c) = 0:47, (d) = 0:51.

11 The second optimization task we consider is the classic traveling salesman problem. Though part of its importance lies in its role as benchmark for new theory and optimization techniques, the TSP and its variants have many practical applications ranging from printed circuit board design to X-ray crystallography toscheduling. The task is simple: nd the shortest tour passing through a set of cities, visiting each city only once. One of the most eective solution techniques for the TSP is due to Lin and Kernighan [14]. Their method relies on a local operator, k-opt, to improve solutions. The k-opt operator removes k edges between cities and replaces them with k new edges such that the new edges still form a valid tour. At each improvement, the Lin-Kernighan heuristic intelligently chooses an appropriate value of k to consider. Here we investigate the quality of solutions as a particular k-opt move is applied with increasing parallelism. The results we present do not depend on the exact form of the operator, but only on thefactthatithasalocalinteraction range. The operator we use is dened as follows: for city i we select k ; 1 other cities at random yielding a set fc i c i1 ::: c ik;1 g of k cities. This subset of cities are then cyclicly permuted within the tour. For example, if the tour starting at city 1 and visiting cities 2 3 ::: 10 1 in sequence is represented as f g and the subset of cities considered for a 3-opt move isf3 5 7g, then the resulting tour after the move isf g. The results for a set of N = 439 cities and various interaction ranges k are presented in gure 5. The results are exactly analogous to the ones presented for the NK model with k playing the role of K. For k 3 there is no transition, and better solutions are obtained as increases. This monotonic improvement arises from the fact that as increases it becomes easier to escape local optima. As we increase the range of the operator by increasing k, weeventually reach apoint where it is no longer possible to achieve locally optimal solutions for high values and an abrupt degradation of performance is observed after a certain degree c of parallelism. The location of c decreases with the range of the operator, being approximately c 0:85 for k = 4 and c 0:65 for k =5. We have also looked at inversions where the direction of the tour is reversed between two randomly chosen cities. This operator can make great changes especially if the selected cities are far apart in the tour. The transition for inversions islessthan c < 0:1. A study of the order parameter for TSP revealed behavior exactly analogous to that described for the NK model. 10

12 opt 4-opt 5-opt 6-opt Tour length τ Figure 5: Expected tour lengths under 3-opt, 4-opt and 5-opt local moves as a function of for a set of N = 439 cities called pr439.tsp. This tour is supplied in the TSPLIB package, which is available by anonymous ftp through elib.zib-berlin.de. The results are averaged over 30 initial starting points after having run 100 generations. 11

13 Conclusions We have discovered and characterized a phase transition that arises as local search operators on complicated spaces are parallelized. The existence of this phase transition places a sharp upper bound on the amount of useful parallelism in local search. Moreover, the results presented here provide the foundation for a general theory that would predict which problems could or could not be successfully parallelized. Such a theory would be based on an analysis of the synergistic/antagonistic eects of overlapping applications of local operators. These eects may result from the inherent ruggedness of the particular landscape representation of the problem (the NK case) or from the range of interaction of local operators (k-opt in the TSP case). A phase transition seen in the satisability of constraint satisfaction problems [15] has previously shown the importance of critical phenomena in arti- cial intelligence. The phase transition described here is quite distinct from this satisability transition and further demonstrates the importance critical behavior in many elds of optimization and articial intelligence. It is interesting to consider whether the critical behavior presented here arises in more general distributed systems of optimizing agents. Any time the decision of one agent relies on information contained in the state of another agent, the possibility exists for an abrupt degradation in performance as more agents act in parallel and the amount of \stale" information that exists due to nonzero information propagation delays increases. Phase transitions such as the one described in this report may be lurking in many human organizations. We would like to thank Dan Stein for useful discussions and comments, particularly his insights into the energy decrease leading up to the phase transition. We would also like to thank J. Tsitsiklis, N. Berker, G.J. Sussman, H. Abelson, K. Dahmen, and E. Hung for many useful suggestions. References [1] S. Kirkpatrick, C. D. Gelatt Jr., M. P. Vecchi, Science, 220, 671, (1983). [2] S. Forrest, Science, 261, 872, (1993). 12

14 [3] F. Glover, ORSA J. Comput., 1, 190, (1989), F. Glover, ibid. 2, 4, (1990), D. Cvijovic, J. Klinowski, Science, 267, 664, (1995). [4] R. Azencott, ed., Simulated Annealing Parallelization Techniques (John Wiley & Sons, New York, 1992). [5] S. A. Kauman, The Origins of Order, (Oxford University Press, 1993). [6] Gerhard Reinelt, The Traveling Salesman, computational solutions for TSP applications, (Springer-Verlag, Berlin, Heidelberg, 1994). [7] K. H. Fischer, J. A. Hertz, Spin Glasses, (Cambridge University Press, 1991). [8] B. Derrida, Phys. Rev. B, 24, 2613, (1981), C. A. Macken, A. S. Perelson, Proc. Natl. Acad. Sci. USA, 86, 6191, (1989), E. D. Weinberger, Phys. Rev. A, 44, , (1991). [9] N. A. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller ande. J. Teller, J. Chem. Phys. 21, 1087, (1953). [10] We stress that no such transition occurs for sequential dynamics when we investigate how the asymptotic energy depends upon search at a quenched, i.e. xed, temperature. The dependence of the asymptotic energy on T is markedly dierent from the dependence on parallelism. [11] R. J. Creswick et al, Introduction to Renormalization Group Methods in Physics, (Wiley & Sons, 1992). J.J. Binnet, et. al, The Theory of Critical Phenomena (Oxford University Press, Oxford, 1992). [12] In the numerical experiments reported in gures 1,2,3,5 we have waited suciently long that further increases of G will not noticeably aect the transition. [13] G. Weisbuch, D. Stauer, J. Physique. 48, 11, (1987). E.D. Weinberger, Phys. Rev. A. 44, 6399, (1991). [14] S. Lin, B.W. Kernighan, Operations Research, 21, , (1973). 13

15 [15] S. Kirkpatrick, B. Selman, Science, 264, 1297, (1994), P. Cheeseman, B. Kanefsky, W.M Taylor, Proc. of the 12th IJCAI, (1991). 14

5. Simulated Annealing 5.1 Basic Concepts. Fall 2010 Instructor: Dr. Masoud Yaghini

5. Simulated Annealing 5.1 Basic Concepts Fall 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Real Annealing and Simulated Annealing Metropolis Algorithm Template of SA A Simple Example References