Evolutionary Computation. Algorytmy Niedeterministyczne II. Evolutionary Computation. Evolutionary Computation. Evolutionary Computation

Transcription

1 lgorytmy Niedeterministyczne Krzysztof Trojanowski WMP. SNS UKSW sem. zimowy 5/ uring the sixties three different implementations of the idea of evolutionary computation have been developed at three different places:. n the US ogel introduced evolutionary programming,. n the US olland called his method a genetic algorithm. 3. n ermany Rechenberg and Schwefel invented evolution strategies. or about 5 years these areas developed separately. t is since the early nineties that they are envisioned as different representatives ( dialects ) of one technology. t was also in the early nineties that a fourth stream following the general ideas has emerged genetic programming. The contemporary terminology denotes the whole field by evolutionary computing and considers evolutionary programming, evolution strategies, genetic algorithms, and genetic programming as sub-areas. Two classes of problems:. real valued search space problems the search space is R n.. combinatorial problems (raph Theory, Network esign, Storage and Retrieval, Sequencing and Scheduling, Logic and others), Main components of the evolutionary algorithm:. Representation of solution and its initialisation (problem class dependent).. Selection 3. Recombination (representation dependent). Mutation (representation dependent) 5. Replacement volutionary lgorithm : P initial population problem class dependent : evaluation(p) 3: repeat : P recombination(selection(p)) representation dependent 5: P mutation(p ) representation dependent : evaluation(p ) 7: P replacement(p, P ) 8: until termination condition met Representation of solution (problem class dependent) Representation of solution for combinatorial problems: permutations (e.g., list of nodes in the graph for TSP) combination (e.g., subset of nodes for minimum vertex cover in the graph) variation (e.g., shortest common subsequence for a set of strings) bit sequence (e.g., the - knapsack problem) for real valued search space problems: bit sequences with binary or ray coding (genotype phenotype) vector of floating point numbers (genotype = phenotype)

2 Representation of solution for real valued search space problems: number inary ray lgorithms converting ray inary and inary ray can be found in the blog. Representation of solution for real valued search space problems: amming distance number of bit-wise differences. amming distance between 3 and for inary coding is 3 ( and ). amming distance between 3 and for ray coding is ( and ). t is for any neighbour values! amming cliff when a one-bit mutation can make a large (or a small) jump and a multi-bit mutation can make a small (or large) jump. int: f you encode your individual such that small changes in the genotype (like one bit flip) are somewhat more likely to result in small changes in the phenotype and in the fitness as well, you can help your optimizer. Representation of solution initialisation: The initial population of candidate solutions is usually generated randomly across the search space. f you have some knowledge about your problem, however, you could bias the system by tending to pick values in certain regions of the space. dditionally, you can seed the initial population with pre-chosen individuals of your own design. Selection Selection stochastic universal sampling: Selection: itness Proportionate Selection (roulette wheel already presented) Stochastic Universal Sampling Ranking Tournament Selection litist selection magine that the roulette wheel has an equally spaced multi-armed spinner. ssume that the population is laid out in random order on the wheel as in a pie graph where each individual is assigned space on the pie graph in proportion to fitness. SUS (stochastic universal sampling):. make a single spin of the roulette wheel. This provides a starting position and the first selected individual.. Then proceed by advancing all the way around the wheel in equal sized steps from one arm to the next one of the spinner. Step-size equals 3 /m where m is the total number of individuals to be selected. This does not mean that every candidate on the wheel will be selected; individuals with very thin slices of the wheel might be stepped over completely depending on the random starting position.

3 Selection ranking: Rank selection establishes the arbitrary way to designate the sampling probabilities sel(x) by defining their ranks R(x) from the best to the worst. The rank assignment:. or the individual x with the lowest quality R(x ) = α where α >. f y is the immediate predecessor of x and (for minimization context): if f (x) > f (y) then R(x) = R(y) + β, β >, and if f (x) = f (y) then R(x) = R(y). n general: P(x k ) = α + β k where α and β are constants and: P (α + β k) = k= Selection tournament selection: Single sampling:. Selecting in some way the tournament mate (a subset of k individuals) from the population P. Selecting one of the best fitted individuals from the tournament mate. Typical size of the tournament mate is. The usual methods of mate selection is the simple k-time sampling (without returning) according to the uniform probability distribution on the whole P. dvantage: tournament can cope with situations where there is no formal objective function at all. Selection elitist selection: The main idea is to ensure the passage of the best fitted individuals from the current population P t to the next steps of the algorithm. Two steps:. We select in a deterministic way the sub-multiset lite P t which gathers the best fitted individuals from the current population. lite is passed to P t+ with the probability.. Other individuals are selected according to other, probabilistic rules. Recombination representation dependent:. binary representation. real-valued representation 3. combinatorial representation Recombination binary representation: One-point crossover Two-point crossover Uniform crossover Recombination binary representation: binary crossover operator itself can be written as a binary string or mask. This means that for allele origins from the first parent, whereas for from the second one. xample masks: one point crossover, two point crossover, uniform crossover. Questions: ow often do we apply it Some always do, others use a stochastic approach, applying crossover with a probability χ <. o we generate one offspring or two n many cases there are natural twin offspring resulting, but in more sophisticated problems it may be that only one offspring arises. When we choose only one from two, how do we do it

4 parents ominant (discrete) recombination a a n a a n iscrete recombination ntermediate recombination lobal intermediate recombination Weighted recombination population a... a.... a a n a n a n randomly select 3 parents a... a a n a n offspring a... a n ntermediate recombination where: offspring i = ρ ρ parent j i j= offspring i i-th coordinate of the offspring, parent j i i-th coordinate of the j-th parent, ρ number of parents. lobal intermediate recombination averages the whole population to generate one offspring individual. n the global intermediate recombination, every individual in the population takes the same probability in generating offspring. ntermediate recombination calculates the center of mass (centroid) of the ρ parent vectors. Weighted recombination: efore recombination:. rank the solutions according to its fitness (highest i corresponds to best individual).. assign weights to the solutions: w i = log(µ+) log(i); i {,..., µ} weights for weighted recombination 8 or the case of evolution strategies a pair of real vectors, v = (x, σ): a vector of standard deviation: ( σ i µ ) = µ w i σ i exp(τ N(, ) + τn i (, )) i= wi i= a point coordinate: ( x µ ) i = µ w i x i + σ i= wi i N i (, ) i=

5 Naive, that is, for example one-point crossover of the solutions of combinatorial problems may produce infeasible offspring solutions: parents offspring f this represents a TSP, the first offspring visits cities and twice, and never gets to cities 3 or. ombinatorial problem crossovers: PMX (partially mapped crossover) OX X (cycle crossover) PMX (partially mapped crossover) uild an offspring by choosing a subsequence of a tour from one parent and preserving the order and position of as many cities as possible from the other parent. subsequence of a tour is selected by choosing two random cut points (boundaries for swapping operation). n example: p = [ ] p = [ ] irst, the segments between the cut points are swapped: o = [x x x 8 7 x x] o = [x x x 5 7 x x] PMX (partially mapped crossover) an example: p = [ ] o = [x x x 8 7 x x] p = [ ] o = [x x x 5 7 x x] Then we fill no conflict cities: o = [x x 9] o = [x x ] The swap defines also a series of mappings:, 8 5, 7, 7 inally, for conflict cities we use mappings: o = [ ] o = [ ] OX uilds offspring by choosing a subsequence of a tour from one parent and preserving the relative order of cities from the other parent. n example: p = [ ] p = [ ] irst, the segments between the cut points are copied into offspring: o = [x x x 5 7 x x] o = [x x x 8 7 x x] OX an example: p = [ ] o = [x x x 5 7 x x] p = [ ] o = [x x x 8 7 x x] Next starting from the second cut point of parent p, the cities from the parent p are copied in the same order, omitting symbols already present. Reaching the end of the string, we continue from the first place of the string.. The sequence of the cities in p : after removal of, 5, and 7: This sequence is placed in the first offspring: o = [ 8 finish of the sequence 5 7 start 9 3] Similarly, we get the other offspring: o = [ ].

6 X (cycle crossover) builds offspring in such a way that each city (and its position) comes from one of the parents. n example: p = [ ] p = [ ] would produce the first offspring by taking the first city from the first parent: o = [ x x x x x x x x] X (cycle crossover) an example: p = [ ] p = [ ] and the offspring o = [ x x x x x x x x] n p, is just below the selected, so: o = [ x x x x x x x] n p, 8 is just below the selected, so: o = [ x x x x x 8 x] n p, 3 is just below the selected 8, so: o = [ x 3 x x x 8 x] n p, is just below the selected 3, so: o = [ 3 x x x 8 x] The selection of requires selection of, which is already on the list the cycle is completed. the remaining cities are filled from the other parent, that is, p : o = [ ]. Similarly: o = [ ]. Mutation Mutation binary representation: produces the binary string p from the string p by using the mutation mask i according to the formula p = p i representation dependent:. binary representation. real-valued representation 3. combinatorial representation The point-wise, modulo addition of the mask to the parental code the bit inversion in p on positions that coincide with the positions of ones in the mask code i. The mask i = [a,,..., a l ] is sampled independently for the mutation of each particular parental code p and independently of the value of this code. ach bit a i, i =,..., l of the mask string is sampled independently from the other a j, j i during the single mask sampling procedure. Mutation # real valued representation: n individual consists of a vector and a scalar: v = (x, σ). oth are mutated (mutation order is important). Uncorrelated mutation with one σ:. σ σ exp(τ N(, )) (self-adaptation step). x t+ i = x t i + N(, σ ) (mutation step) Typically: the learning rate : τ n a boundary rule: σ < ε σ = ε igure: Mutation #: distribution of offsprings for the parent [,] in -dimensional search space.

7 Mutation # real valued representation: Mutation # real valued representation: omments: dvantages:. Simple adaptation mechanism.. Self-adaptation usually fast and precise. n individual consists of two vectors: v = (x, ~σ ). oth are mutated (mutation order is important). Uncorrelated mutation with n σ s:. σi σi exp(τ N(, ) + τ Ni (, )) (self-adaptation step). xt+ = xt + N(, ~ σ ) (mutation step) isadvantages:. ad adaptation in case of complicated contour lines.. ad adaptation in case of very differently scaled object variables, for example, < xi < and < xj <. Typically: Thomas a ck, Lecture at O the overall learning rate: τ a boundary rule: σi < ε σi = ε Mutation # real valued representation: omments: dvantages:. ndividual scaling of object variables.. ncreased global convergence reliability. isadvantages:. Slower convergence due to increased learning effort.. No rotation of coordinate system possible (could be required for badly conditioned objective function). igure: Mutation #: istribution of offsprings for the parent [,] in -dimensional search space: [σx =.5, σy = ] on the left, and [σx =, σy =.5] on the right Thomas a ck, Lecture at O Mutation real valued representation: isadvantages of the mutation #: n - n the coordinate wise learning rate: τ No rotation of coordinate system possible (could be required for badly conditioned objective function).. method for a rotation of a point in the -dimensional search space by a counterclockwise angle θ: R b = c, where cos(θ) sin(θ) b c R= b= c= sin(θ) cos(θ) b c The set of points (circles): (,) (,) (3,) (,) (.5,) (.,) The set of points after counterclockwise rotation by (squares): (.5,.8) (.5,.33) (9.5,.87) (-.35,.3) (.35,.3) (.3, 3.98)

8 Mutation real valued representation: or more dimensional search space, for example -dimensional, the rotation procedure of a point by the plane [3, 5] is: R= cos(θ) sin(θ) sin(θ) cos(θ) b. b =.. b c. c =.. c igure: istribution of offsprings for the parent [,] in -dimensional search space for [σx =, σy =.5] before the rotation on the left, and after the rotation on the right Mutation real valued representation: or θ[3,5] =, that is, when there is no rotation for the plane [3, 5], cos(θ) = and sin(θ) =, so R[3,5] is the identity matrix: R= Mutation real valued representation: n n-dimensional search space there exist n(n )/ planes: [, ], [, 3], [, ],... [, n], [, 3], [, ],... [n, n], thus... The full definition of the rotation procedure in n-dimensional space requires n(n )/ angles θij where i = {,,..., n } and j = {i +,..., n}. or each plane [i, j] we create its rotation matrix R(θij ). orrelation matrix: R = n Y n Y R(θij ) i= j=i+ orrelation matrix is a good starting point for a new mutation... Mutation #3 real valued representation: ~ n individual X consists of three vectors (x, ~σ, θ): X = ((x,..., xn )(σ,..., σn )(θ,..., θn(n )/ )) ll the three are mutated (mutation order is important). Mutation produces X = ((x,..., xn )(σ,..., σn )(θ,..., θn(n )/ )) according to covariance matrix where:. σi = σi exp(τ N(, ) + τ Ni (, )) (self-adaptation step). θj = θj + β Nj (, ) and if θj > π θj = θj π sign(θj ) (self-adaptation step; β.873 in radians, that is, 5 ) ~ ~, ) (mutation step) 3. xt+ = xt + N( ~ ~, )... ow to create the perturbation vector N( ~ ~, ): Mutation perturbation vector N( Multiplication of uncorrelated mutation vector σ~u with n(n )/ rotational matrices R(θj ): σ~c = n Y n Y i= j=i+ R(θij ) σ~u

9 Mutation #3 real valued representation: : n q = N(N )/ total number of rotation planes : for i = to N do 3: sigmau[i] = sigma[i] N i (, ) the uncorrelated σ u : end for 5: for k = to N do : n = N k index of the st axis of the plane 7: n = N index of the nd axis of the plane 8: for i = to k do rotations 9: d = sigmau[n ]; d = sigmau[n ] : sigmau[n ] = d sin(theta[n q]) + d cos(theta[n q]) : sigmau[n ] = d cos(theta[n q]) d sin(theta[n q]) : n = n increment the index for nd axis of the plane 3: n q = n q increment the index of the angle θ q : end for 5: end for Mutation #3 real valued representation: for (int i=; i<n; i++) // generation of the uncorrelated mutation vector sigmau[i] = sigma[i] * N(,); nq = N*(N-)/-; for (int k=; k<(n-); k++) { // rotations // ata structures: n = N--k; n = N-; int nq; for (int i=; i<=k; i++) { int n, n; d = sigmau[n]; double d, d; d = sigmau[n]; double sigma[n]; sigmau[n] = d*sin(theta[nq])+ d*cos(theta[nq]); double sigmau[n]; sigmau[n] = d*cos(theta[nq])- d*sin(theta[nq]); double theta[m]; n = n-; double x[n]; nq = nq-; double mut_x[n]; } } for (int i=; i<n; i++) // mutation mut_x[i] = x[i] + sigmau[i]; Mutation #3 real valued representation: omments: dvantages:. ndividual scaling of object variables.. Rotation of coordinate system possible. 3. ncreased global convergence reliability. isadvantages:. Much slower convergence.. ffort for mutations scales quadratically. 3. Self-adaptation very inefficient. Thomas äck, Lecture at O Unified view of mutation for the real valued representation Perturb each element in the population by adding a random vector: x = x + z, where z is a random vector with mean and covariance matrix = (c ij ) i, j =,..., n.. The components of z are independent random variables having the same distribution. is a diagonal matrix: = diag(s, s,..., s ) with s the only control parameter.. The components of z are independent random variables having different distributions. is a diagonal matrix: = diag(s, s,..., s n ) and the control parameters are s, s,..., s n. 3. The components of z are dependent random variables. There are n(n + )/ control parameters of the mutation: s, s,..., s n mutation steps θ, θ,..., θ k rotation angles (k = n(n )/) is a matrix where c ii = si and c ij = / (si sj ) tan( θij ) if i and j are correlated (otherwise, c ij = ). Unified view of mutation for the real valued representation The perturbation vector z: #: z N(, s ) #: z N(, ) where = diag(s, s,..., s n ) #3: z N(, ) where N(, ) is a realization of a multi-variate normal distribution with zero mean and unity covariance matrix. Probability density function for the mutations: p z(x) = exp( xt x) (π) n det() where is any positive definite and symmetric correlation matrix. (cfr. -dimensional aussian p.d.f.: f (x) = /( πσ ) exp( (x µ) σ ).) Mutation combinatorial problems: Mutation operators for combinatorial problems are problem specific. Neighborhood functions in Local Search methods the mutation operators. xample: neighborhood structures for job shop scheduling (VNS): xchange any two randomly selected operations are simply swapped. or instance, for [,,,,, 3,, 3, 3] and the two random numbers and 8 the new state will be [,, 3,,, 3,, 3, ]. nsert inserts a randomly chosen operation number in front or back of another randomly chosen operation number. or instance, for [,,,,, 3,, 3, 3] and the two random numbers 3 and the new state will be [,,,,, 3,, 3, 3] (the operation 3 is moved in front of the operation ).

10 Mutation combinatorial problems: Mutation combinatorial problems: xample: neighborhood structures for TSP: nversion selects two points along the length of the chromosome, which is cut at these points, and the substring between these points is reversed. nsertion selects a city and inserts it in a random place. isplacement selects a subtour and inserts it in a random place. Reciprocal exchange swaps two cities. () () igure: -opt modification of a solution for TSP. Mutation combinatorial problems: () () (3) Replacement () (5) () igure: 3-opt modification of a solution for TSP. (3)-() results of modification (note, that the case (3) represents different approach to modifcation that cases () ()). Replacement elete-all (generational, (µ, λ)) deletes all the members of the current population and replaces them with the same number of chromosomes that have just been created. Steady-state or (µ + λ) deletes n old members and replaces them with n new members (n is a parameter). Question: which members to delete from the current population the worst individuals, randomly selected or those chromosomes that were used as parents... (this is also a parameter). Steady-state-no-duplicates the same as the steady-state technique but the algorithm checks that no duplicate chromosomes are added to the population. This adds to the computational overhead but can mean that more of the search space is explored. rief summary

11 Summary two crucial issues enetic diversity of the population should be preserved as long as possible. f populations starts to concentrate in a very narrow region of the search space, all advantages of handling many different individuals vanish. This phenomenon is known as premature convergence. Two main directions to prevent this:. a priori ensuring creation of new material,. a posteriori manipulating the fitnesses of all individuals to create a bias against being similar, or close to, existing candidates. Summary two crucial issues xploration and exploitation The dilemma is: whether to search around the best-so-far solutions (as their neighborhood hopefully contains even better points) or explore some totally different regions of the search space (as the best-so-far solutions might only be local optima). Two driving forces of exploration and exploitation. Selection represents a push toward quality and is reducing the genetic diversity of the population.. Variation implemented by recombination and mutation operators, represents a push toward novelty and is increasing genetic diversity. n appropriate balance between these two forces has to be maintained. Thank you.