( ) ( ) ( ) ( ) Simulated Annealing. Introduction. Pseudotemperature, Free Energy and Entropy. A Short Detour into Statistical Mechanics.

Transcription

1 Aims Reference Keywords Plan Simulated Annealing to obtain a mathematical framework for stochastic machines to study simulated annealing Parts of chapter of Haykin, S., Neural Networks: A Comprehensive Foundation, Prentice-Hall, 999, and Neural Networks and Learning Machines, Prentice-Hall, 009. temperature, annealing schedule, Metropolis algorithm, combinatorial optimization, energy function, move set, mean-field annealing, quadratic assignment problem, cost function, critical temperature statistical mechanics Metropolis algorithm annealing schedule travelling salesperson problem energy function move sets for TSP mean-field annealing critical temperature Introduction Because (industrial strength) neural networks may have thousands of degrees of freedom (e.g. weights) it is possible to get inspiration from the theory of statistical mechanics. Statistical mechanics deals with the macroscopic equilibrium properties of large systems of elements that are subject to the microscopic laws of mechanics. Simulated annealing is an optimization technique that uses a thermodynamic metaphor. Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Notes Bill Wilson, 03 A Short Detour into Statistical Mechanics Consider a system with many degrees of freedom that can be in many possible states. Let p i denote the probability of state i. Then i p i =. Let E i denote the energy of the system in state i. Statistical mechanics says that when a physical system is in thermal equilibrium with its environment, state i occurs with probability E p = exp i i Z kbt where T is temperature (Kelvin scale), k B is Boltzmann's constant, and Z is a constant. As i p i =, we can derive E Z = exp i i kbt The probability distribution of.3 is called the Gibbs distribution and the factor exp( E i /(k B T)) is called the Boltzmann factor. Note from.3 that lower energy or higher temperature higher probability. As T is reduced, the probability is concentrated in a smaller subset of low-energy states. (.3) (.4) Pseudotemperature, Free Energy and Entropy We can mimic this setup in a neural net context using a concept of pseudotemperature T. As T has no scale (like Kelvin) we don't need an analog of k B, so we can write Ei p = exp i Z T Ei and Z = exp i T Notice that if T = Z =, then E i = log e p i, so log e p i measures something like energy. The Helmholtz free energy, F, of a system, is defined as (.5/) F = T log e Z (.) The average energy E of the system is defined by E = i p i E i (.8) Simulated Annealing Notes Bill Wilson, 03 3 Simulated Annealing Notes Bill Wilson, 03 4

2 Entropy Using.5 to.8, we derive E F = T i p i log e p i (.9) The RHS (except for the factor T) is called the entropy H of the system: H = i p i log e p i (.) so E F = TH or F = E TH (.) Entropy and Thermal Equilibrium When two systems A and A' come into contact, the total entropy of the two systems tends to increase: H + H' 0 In view of., (F =! TH) this means that the free energy of the system, F, tends to decrease, reaching a minimum when the two systems reach thermal equilibrium: The minimum of the free energy of a stochastic system with respect to the variables of the system is achieved at thermal equilibrium, at which point the system is governed by Gibbs' distribution. To see this, turn.5 inside out to obtain E i = T(log e (p i )+log e Z). Thus ( + log e Z) E = i p i E i = T i p i log e p i = T i p i log e ( p i ) T log e Z i p i = T i p i log e ( p i ) T log e Z = T i p i log e ( p i ) + F, so E F = T i p i log e ( p i ), as claimed. Simulated Annealing Notes Bill Wilson, 03 5 Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Simulated annealing is an optimization technique. In Hopfield nets, local minima are used in a positive way, but in optimization problems, local minima get in the way: one must have a way to escape from them. The two ideas of simulated annealing are as follows:. When optimizing a very large and complex system (i.e., a system with many degrees of freedom), instead of always going downhill, try to go downhill most of the time.. Initially, the probability of going uphill should be relatively high ( high temperature ) but as time (iterations) go on, this probability should decrease (the temperature decreases according to an annealing schedule). The term annealing comes from the technique of hardening a metal (i.e. finding a state of its crystalline lattice that is highly packed) by hammering it while initially very hot and then at a succession of decreasing temperatures. Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. (983) Optimization by simulated annealing, Science 0, -80. Metropolis Algorithm The algorithm for simulated annealing is a variant (with time-dependent temperature) of the Metropolis 3 algorithm. In each step of this algorithm, a unit of the system is subjected to a small random displacement (or transition or flip), and the resulting change E in the energy of the system is computed. If E 0, the displacement is accepted. If E > 0, the algorithm proceeds in a probabilistic manner: the probability that the displacement will be accepted is p.exp( E/T) where p is a constant and T is the temperature. If T is large, exp( E/T) approaches. Thus p is the probability that a transition to a higher energy state will be accepted when the temperature is infinite. The use of the expression exp( E/T) ensures that at thermal equilibrium the Boltzmann distribution of states prevails 4. This in turn ensures that, at high temperatures, all states have equal probability of occurring, while as T 0, only the states with minimum energy have a non-zero probability of occurrence. 3 Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, E. (953) Equations of state calculations by fast computing machines, Journal of Chemical Physics, Boltzmann, L. (8) Weitere studien über das Wärmegleichgewicht unter gasmolekülen, Sitzungsberichte der Mathematischen-Naturwissenschaftlichen Classe der Kaiserlichen Akademie der Wissenschaften Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Notes Bill Wilson, 03 8

3 Design of the Annealing Schedule Initial value of the temperature: T 0 is chosen high enough to ensure that virtually all proposed transitions are accepted by the simulated annealing algorithm; Decrement: Usually, a geometric progression of temperatures is used: T k = α.t k, where α is a constant slightly less than (e.g ). At each temperature, enough transitions are attempted that either there are accepted transitions per unit on the average, or the number of attempts exceeds 0 times the number of units; Final value of the temperature: The system is frozen and annealing stops if the desired number of acceptances is not achieved at three successive temperatures. Simulated Annealing for Combinatorial Optimization Simulated annealing is well-suited for solving combinatorial optimization problems. Solutions (or states corresponding to possible solutions) are the states of the system, and the energy function is a function giving the cost of a solution. Kirkpatrick et al. applied their methods to the Travelling Salesperson problem, finding nearoptimal solutions for up to 000 sites, using α = 0.9. In order to apply simulated annealing to such a problem, it is necessary to have a set of neurons and an energy function. The figure below illustrates the neural layout and its interpretation. sequence cities Neural configuration for a tour of 5 cities C-C5, in the order C C4 C C3 C5 C. Simulated Annealing Notes Bill Wilson, 03 9 Simulated Annealing Notes Bill Wilson, 03 Energy Function for TSP Constraints on the neural activations (for a solution) include that there should be: exactly one neuron "on" (activation ) in each row (i.e. each city visited exactly once) exactly one neuron on, in each column (salesperson only visits one city at a time!). Let us write v ij for the activation level of the neuron in row i and column j. One constraint can be expressed as saying that we want to minimize e j = (v j +v j +v 3j +v 4j +v 5j ) = ( i v ij ) That's for column j. Taking into account all rows, we want to minimize E = j e j = j ( i v ij ) Similarly, taking into account all rows, we want to minimize E = i ( j v ij ) Objective Function An optimization problem, of course, comes with an objective function to be minimized. In our case, suppose that d ij is the distance from city i to city j. Suppose that cities and are adjacent on the salesperson's tour, and that city is the m th city visited. Then city must be either m st or m+ st on the tour, and the contribution to the total distance travelled will be d, (v,m v,m + v,m v,m+ ) Remember that in a solution, if v,m =, then v,m+ = 0, and vice-versa. Generalizing this, for the total distance travelled we get the objective function E 3 = 0.5 k i j d ij v ik (v j,k + v j,k+ ) To minimize E, E, and E 3 simultaneously, we minimize E = k E + k E + k 3 E 3 where the k i are positive constants. The move set (next slide) maintains the constraint that each v ij = 0 or. Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Notes Bill Wilson, 03

4 Move Set for Simulated Annealing inversion,,8,9 is replaced by 9,8,, translation Remove a section (-8) & replace it between two other consecutive cities (in this case 4 and 5). switching Select two non-consecutive cities (in this case and ) and switch them in the tour. Mean-Field Annealing Simulated annealing can be slow, and the annealing schedule can be part of the problem. In some cases it is found that most of the crystalization of the system takes place around a particular temperature, termed the critical temperature. Fang, Wilson and Li 5 tackled the quadratic assignment problem exploiting this fact. Matlab code for simulated annealing is available in tsp.m, available at the "Other Matlab code" link under Software Availability on the class home page. Reference for parts of this material: Neuro-fuzzy and Soft Computing by JSR Jang, CT Sun, and E Mizutani, Prentice-Hall, 99, page Luyuan Fang, William H. Wilson, and Tao Li (990) Mean field annealing neural nets for quadratic assignment, INNC-90-PARIS Proceedings, : 8-8 Simulated Annealing Notes Bill Wilson, 03 3 Simulated Annealing Notes Bill Wilson, 03 4 The Quadratic Assignment Problem Consider the optimal location of m plants at n possible sites, where n m, in the following situation: The amount of goods to be transported between plants is given. There is a cost associated with moving goods between sites. 3 3 goods flows The goal is to allocate plants to sites so as to minimise total cost. Terminology: x ik = if plant k is located at site i - only one plant per site. 0 x ik c ij is the cost of transporting unit of goods from site i to site j. c ij 0. transport cost/unit 3 d kl is the amount of goods to be transported from plant k to plant l. d kl 0. Cost Function: f(x) = i=..n j i k=..m l k c ij d kl x ik x jl Minimising this function is an NP-hard problem. Simulated Annealing Notes Bill Wilson, 03 5 Simulated Annealing Notes Bill Wilson, 03

5 Neural architecture for this problem We choose a two-dimensional array of neurons, of dimension m n. x ij represents the state of the i,j-th neuron: x ij = plant j is assigned to site i. Two constraints on the x ij in a solution: n one plant per site x ij = for each plant j i= m every plant must be located at exactly one site x ij for every i j= Energy Function: n m n m m $ E = ( A ) c ij d kl x ik x jl + ( B ) x ik x il + C & i=j i k=l k i=k= l k k=% n x ik where A, B, and C are constants. The energy function is minimised if the constraints are satisfied & the cost function is minimised. i= ' ) ( Critical Temperature Phenomenon: when temperature T is very high, the network reaches an equilibrium point where all the neurons have similar activation values near 0.5; as T is decreased, this point is also lowered; at a certain temperature T c, (the critical temperature), this point drops down to δ which depends on the parameters A, B, C and t. This is the lowest equilibrium point at which all neurons have similar activation values. Behaviour below T c as the temperature drops below T c, the neuron activations diverge rapidly towards 0 and ; when the temperature becomes very low, the network settles into a stable state which represents a feasible solution to the problem. The neuron activation values do not diverge until the critical temperature is reached. Near the critical temperature, the neuron activations rapidly diverge towards the two extreme points 0 and. Below T c, neuron activations again remain relatively stable. Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Notes Bill Wilson, 03 8 Critical Temperature Estimate Let m c and m d be the mean values of c aj and d bl. The lowest equilibrium point δ is estimated as δ C/(A(n )(m )m c m d + B(m ) + nc) The expected critical temperature is estimated as T c = 0.5* t ( δ)[a(m )(n )m c m d δ + B(m )δ + C(nδ )] Annealing Schedule for Mean-Field Annealing: At T = t( δ)[a(m )(n )c max d max δ + B(m )δ + C(nδ )], simulate until equilibrium. Around T c, (between T max and T min ) the temperature changes according to ΔT = K T T c ( T max T min ) +ε where T is the present temperature and ε and K are constants. At each temperature, iterate s steps (where s is large enough to guarantee reaching equilibrium at a temperature above the actual critical temperature). At a low temperature near 0, simulate until equilibrium. Simulated Annealing Notes Bill Wilson, 03 9 Simulated Annealing Notes Bill Wilson, 03 0

6 Simulation Results run number optimal mean field meanfield/optimal Results for 5 5 Data: Mean Field Network. Based on randomly generated 5 5 quadratic assignment problems. Critical Temperature Plot for 5 by 5 Mean Field Network Simulated Annealing Notes Bill Wilson, 03 Simulated Annealing Notes Bill Wilson, 03