how should the GA proceed?

Size: px

Start display at page:

Download "how should the GA proceed?"

Jack Miller
5 years ago
Views:

1 how should the GA proceed? string fitness which new string would be better than any of the above? (the GA does not know the mapping between strings and fitness values!) recombine 'surface' similarities among the fittest strings combine 'good ideas' from different good strings because certain substructures in good strings cause their high fitness, and recombining such 'good ideas' may lead to better strings nb: fitness measure only says what needs to be done, not how to do it!! 1

2 how GAs work: population - variability - differential effect on survival find best strategy for running 4 hamburger joints strategy:! price: 50 or 5$? drink: wine or coke? service: slow or fast? restaurant 1: price=hi drink=coke service=fast 011 restaurant 2: hi wine fast 001 restaurant 3: low coke slow 110 restaurant 4: hi coke slow 010 unknown: which vbl most important? max profit? max loss? gradient info about fitness landscape (which single vbl, changed alone, has most impact)? can vbls be optimized separately or are they nonlinearly related? does environment itself change? available feedback: only weekly performance; no training period! since nothing is known about environment, test different random strategy in each restaurant for 1. week (hope you're not unlucky!). Favoring diversity expect payoff to approximate average payoff over whole search space 2

3 from generation 0 to generation 1 i string x i fitness f(x i ) f(x i ) / f(x i ) pool after reproduction 1 011! 3! f = total worst average! 1! 3.0!!!!!!! best!! 6!!!! 6 fitness-proportional reproduction: low-fit inds eliminated; expect ind 3 to be selected with prob.5 for each of the 4 positions in population, i.e. expect it to be selected for reproduction twice better average, better worst, less diversity crossing over strings 1 and 3: 01 1 xo 11 0: children 010 & 111. generation 1: 111 (7), 010 (2), 110 (6), 010 (2).. better best 3

4 fitness-proportional reproduction, xo, mut prob (select i ) = f i / f i string 1: 0.14 string 2: 0.49 string 3: 0.06 string 4: % 14% 6% 49% spin biased roulette wheel once for each string; copies of selected strings go into mating pool crossover: sexual reproduction mutation: 4

generic GA initialize population with random candidate solutions; evaluate fitness of each individual; while termination condition not satisfied do 1. select parents; 2. recombine pairs of parents; 3.

5 generic GA initialize population with random candidate solutions; evaluate fitness of each individual; while termination condition not satisfied do 1. select parents; 2. recombine pairs of parents; 3. mutate resulting offspring; 4. evaluate fitness of offspring; 5. select individuals for next generation; 1. fitness proportional, stochastic 2. whether to crossover, which parts, how to combine parts: stochastic. All higher organisms use xover; basis of breeding guarantees that genotype space is connected (eventual convergence) 5. based on fitness and/or age; sometimes deterministic: rank & select best n. 5

6 global EA, local HC memetic... individuals are improved by hill climbing during fitness assessment. * Lamarckian or Baldwin Effect would be better than Memetic. 1: t number of iterations to Hill-Climb 2: P Build Initial Population 3: Best 4: repeat 5: AssessFitness(P) 6: for each individual P i P do 7: P i Hill-Climb(P i ) for t iterations Replace P i in P 8: if Best = or Fitness(P i ) > Fitness(Best) then 9: Best P i 10: P Join(P, Breed(P)) 11: until Best is the ideal solution or we have run out of time 12: return Best 6

7 initializing population enforce diversity: make sure all individuals in initial population are unique -- when a new individual is made, check if it's already there, if so, throw it away, make another one of course not by O(n 2 ) scanning through entire population but: create hash table store individuals as keys in hash table, with arbitrary values (O(n)) if individual is not in hash table, hash it and add to population, else throw away 7

from one generation to the next j := 1; while j popsize do select mate1 from oldpop; select mate2 from oldpop; crossover (mate1, mate2); /*also mutate*/ child1 and child2 go into newpop; maintain #

8 from one generation to the next j := 1; while j popsize do select mate1 from oldpop; select mate2 from oldpop; crossover (mate1, mate2); /*also mutate*/ child1 and child2 go into newpop; maintain # of xovers, parentage, other statistics; fitness child1 := objfunction(decode(child1)); fitness child2 := objfunction(decode(child2)); j := j + 2; /*assuming even popsize*/. generational replacement: oldpop := newpop. 8

9 no run = N? yes end run := 0 gen := 0 make initial random pop run++ termination sat for run? no i := 0 measure individual fitness no i := 0 yes i++ i = M? yes designate result for run Generic Generational Genetic Algorithm select genetic operation Pr Px Pm select one individual based on fitness select two individuals based on fitness select one individual based on fitness gen++ yes perform reproduction perform crossover perform mutation no i = M? i++ insert copy in new pop insert offspring in new pop insert mutant in new pop i++ pick 1 of the 3+ genetic operations (Pr+Px+Pm=1), then select 1 or 2 individuals based on fitness; reselection is allowed. 9

10 generic GA Algorithm 20 The Genetic Algorithm (GA) 1: popsize desired population size 2: P {} 3: for popsize times do 4: P P {new random individual} 5: Best 6: repeat 7: for each individual P i P do 8: AssessFitness(P i ) 9: if Best = or Fitness(P i ) > Fitness(Best) then 10: Best P i 11: Q {} Here s 12: for popsize/2 times do 13: Parent P a SelectWithReplacement(P) 14: Parent P b SelectWithReplacement(P) 15: Children C a, C b Crossover(Copy(P a ), Copy(P b )) 16: Q P {Mutate(C a ), Mutate(C b )} 17: P Q 18: until Best is the ideal solution or we have run out of time 19: return Best 10

11 making random bit strings and vectors 1: v a new vector v 1, v 2,...v l 2: for i from 1 to l do 3: if 0.5 > a random number chosen uniformly between 0.0 and 1.0 inclusive then 4: v i true 5: else 6: v i false 7: return v 1: min minimum desired vector element value 2: max maximum desired vector element value 3: v a new vector v 1, v 2,...v l 4: for i from 1 to l do 5: v i random number chosen uniformly between min and max inclusive 6: return v 11

12 mutating bit strings and vectors 1: p probability of flipping a bit Often p is set to 1/l 2: v boolean vector v 1, v 2,...v l to be mutated 3: for i from 1 to l do 4: if p random number chosen uniformly from 0.0 to 1.0 inclusive then 5: v i (v i ) 6: return v 1: v vector v 1, v 2,...v l to be convolved 2: p probability of adding noise to an element in the vector 3: r half-range of uniform noise 4: min minimum desired vector element value 5: max maximum desired vector element value 6: for i from 1 to l do 7: if p random number chosen uniformly from 0.0 to 1.0 then 8: repeat 9: n random number chosen uniformly from r to r inclusive 10: until min v i + n max 11: v i v i + n 12: return v p often = 1 12

13 1 and 2 point crossover 1: v first vector v 1, v 2,...v L to be crossed over 2: w second vector w 1, w 2,...w L to be crossed over 3: c random integer chosen uniformly from 1 to L inclusive 4: for i from c to L do 5: Swap the values of v i and w i 6: return v and w if c=1 or c = L, no xover happens 1: v first vector v 1, v 2,...v l to be crossed over 2: w second vector w 1, w 2,...w l to be crossed over 3: c random integer chosen uniformly from 1 to l inclusive 4: d random integer chosen uniformly from 1 to l inclusive 5: if c > d then 6: Swap c and d 7: for i from c to d 1 do 8: Swap the values of v i and w i 9: return v and w 13

14 integers to binary... in binary, different bits have different significance; so Hamming distance between consecutive integers is often 1. e.g. chance of changing 7 (0111) to 8 (1000) chance of changing 7 to 6 (0110). thus Gray coding: int[] bintogray (int[] b) { g[1] = b[1]; for (k=2; k m; k++) {g[k] = xor (b[k-1], b[k]);} return g; } integer standard Gray int[] graytobin (int[] g) { b[1] = value = g[1]; for (k=2; k m; k++) {if (g[k] == 1) value =!value; b[k]) = value;} return b; }

15 selection decreasing selection pressure: truncation tournament, ranking (size 2 tournament = linear ranking) fitness proportional uniform 2 rounds of selection: 1. who gets to mate? 2. who gets to survive? one of these rounds should use uniform selection or selection pressure is too high - high selection pressure greedy. 15

16 roulette wheel selection chromosome v i has fitness f i o population fitness F = k=1 popsize f k o selection probability p i for chromosome v i = f i / F o cumulative probability q i for chromosome v i =!! j=1 i p j, i=1,,popsize roulette wheel selection:! 1. r := random number from [0, 1]! 2. if r q 1, select first chromosome v 1 ; otherwise, select the! k th chromosome v k (2 k popsize) such that q k-1 < r q k 16

17 roulette wheel selection, cont. assume the population is sorted by fitness from 1 to popsize μ; calculate [q 1,..., q μ ] where q i = j=1 i p j ; thus q μ = 1.0. current_member := 1; while (current_member μ) do r := random value uniformly from [0,1]; i := 1; while (q i < r) do i := i + 1; mating_pool[current_member] := population[i]; current_member++; fitness proportional selection, esp. roulette wheel, is highly susceptible to fitness function transposition (e.g. adding a constant to all fitness values).

18 roulette wheel selection 1: perform once per generation 2: global p population copied into a vector of individuals p 1, p 2,..., p l 3: global f f1, f 2,..., f l fitnesses of individuals in p in the same order as p 4: if f is all 0.0s then Deal with all 0 fitnesses gracefully 5: Convert f to all 1.0s 6: for i from 2 to l do Convert f to a CDF. This will also cause fl = s, the sum of fitnesses. 7: f i f i + f i 1 8: perform each time 9: n random number from 0 to f l inclusive 10: for i from 2 to l do This could be done more efficiently with binary search 11: if f i 1 < n f i then 12: return p i 13: return p 1 CDF: cumulative distribution function 18

19 roulette wheel selection and 1-point xover as a linear (slow!) search through the roulette wheel with fitness-weighted slots: partialsum := 0.0; j := 0; /*population index*/ wheelposition := random * total fitness F; repeat j++; partialsum += fitness j until partialsum wheelposition or j = popsize; return j /*index of selected individual*/ select one individual cossover parent1, parent2; return child1, child2 (and mutate at the same time): if random p xover then pick xpoint between 1 & length -1 else xpoint := length; for j:= 1 to xpoint do child1[j] := mutate(parent1[j] with p mut ); child2[j] := mutate(parent2[j] with p mut ); if xpoint length then for j:= xpoint + 1 to length do child1[j] := mutate(parent2[j] with p mut ); child2[j] := mutate(parent1[j] with p mut ); 19

20 stochastic universal sampling SUS: another form of fitness-proportional selection SUS spins the wheel only once, with popsize µ equally spaced pointers: r = random number in [0, 1]; for (sum = i = 0; i < µ; i++) for (sum += ExpVal(i, t); sum > r; r++) select(i); ExpVal(i, t): expected # of times individual i is selected to reproduce, i.e. fitness(i) / average pop fitness. Each i reproduces at least ExpVal(i, t) times and at most ExpVal(i, t) times. drawbacks of fitness-proportional selection schemes (SUS, roulette wheel) - early, large fitness differences stifle exploration: premature convergence - later, when many individuals are similar, evolution slows down: random search - need 2 passes through population: 1. mean fitness ; 2. ExpVal 20

21 stochastic universal sampling 1: perform once per n individuals produced Usually n = l, that is, once per generation 2: global p copy of vector of individuals (our population) p 1, p 2,..., p l, shuffled randomly To shuffle a vector randomly, see Algorithm 26 3: global f f1, f 2,..., f l fitnesses of individuals in p in the same order as p 4: global index 0 5: if f is all 0.0s then 6: Convert f to all 1.0s 7: for i from 2 to l do Convert f to a CDF. This will also cause fl = s, the sum of fitnesses. 8: f i f i + f i 1 9: global value random number from 0 to f l /n inclusive 10: perform each time 11: while f index < value do 12: index index : value value + f l /n 14: return p index SUS is O(n), not O(n lg n) to select n individuals; more importantly, unlike fitness-proportional selection, it guarantees that very fit individuals (over s/n) are chosen at least once! 21

Chapter 8: Introduction to Evolutionary Computation

Chapter 8: Introduction to Evolutionary Computation Computational Intelligence: Second Edition Contents Some Theories about Evolution Evolution is an optimization process: the aim is to improve the ability of an organism to survive in dynamically changing