Genetic Algorithms. t=0 initialise [P(t)] evaluate [P(t)] do while (not termination-condition) Lecture 9: Paradigms of Evolutionary Computing

Transcription

1 Genetic Algorithms Genetic lgorithms (GA) re explortory serch nd optimistion methods tht re bsed on Drwinin-type survivl of the fittest strtegy with reproduction, where stronger individuls in the popultion hve higher chnce of creting offspring. Ech individul in the popultion represents potentil solution to the problem. The individuls re represented in the GA by mens of string similr to the wy genetic informtion is coded in orgnisms s chromosomes (Hollnd, 975). Unlike other optimistion techniques, GA does not require mthemticl descriptions of the optimistion problem, but insted relies on cost function, in order to ssess the fitness of prticulr solution to the problem in question (Goldberg, 989). The GA then itertively cretes new popultions from the old by rnking the strings nd interbreeding the fittest to crete new strings, which re (hopefully) closer to the optimum solution to the problem in question. So in ech genertion, the GA cretes set of strings from the bits nd pieces of the previous strings, occsionlly dding rndom new dt to keep the popultion from stgnting. The end result is serch strtegy tht is tilored for vst, complex, multimodl serch spces. A genetic dptive pln then cn be defined s qudruple s N { Σ, Π, Φ Ω} Λ =, N where Σ is the coding formt, Π is popultion of sie N, Φ is fitness re-scling lgorithm nd Ω = [ ω, ω 2, K, ω m ] is the set of genetic opertors. The most common genetic opertors re reproduction crossover nd muttion. Genetic pln refers to the process through which successive popultions re generted ug evlution, selection, mting nd deletion. Let Ψ be probbility distribution over Π which is derived from the fitness of ech tril, µ ( A Π). A genetic pln cn then be formlly expressed s the Λ : Ψ Π Ω. mpping ( ) Π The structure of the genetic lgorithm cn be stted s follows Structure of the genetic lgorithm t=0 initilise [P(t)] evlute [P(t)] do while (not termintion-condition) P M (t) reproduce [P N (t)] evlute[ P M (t) ] Q select[ P M (t) ] P N (t) replce[q] P(t) [P N (t)] t = t enddo ()

2 Where P N (t) denotes popultion of N individuls t genertion t, P M (t) denotes offspring popultion of sie M generted by mens of reproduction, Reproduction opertors re such s crossover nd muttion nd Q is n intermedite popultion in the mting pool. Chromosome representtion The clssicl chromosome representtion scheme for GA is binry vectors of fixed length. In the cse of n-dimensionl serch spce, ech individul consists of n vribles with ech vrible encoded s bit string. In the cse of nominl-vlued vribles, ech nominl vlue cn be encoded s D-dimensionl bit vector, i.e., the vrible cn hve 2 D nominl vlues. In the cse of continuous-vlued vribles, ech vrible should be mpped to D-dimensionl bit vector, i.e. Φ : R { 0,} D The rnge of continuous spce needs to be restricted to finite rnge of [ α, β ]. Ug stndrd binry decoding, ech continuous vrible of chromosome C is encoded ug fixed length bit string. For exmple, if 30-bit representtion, following conversion cn be used: 30 ( 2 ) C n, i [ ], min mx needs to be converted to Binry coding is frequently used, but it hs the disdvntge of Hmming Cliffs - formed when two numericlly djcent vlues hve bit representtions tht hve lrge Hmming distnce. This cuses problem when smll chnge in vribles should result in smll chnge in the fitness. An lterntive bit representtion is to use Gry coding. Gry coding hs the dvntge over binry coding in tht the Hmming distnce between two successive numericl vlues is one. GAs hve lso been developed tht use integer or rel-vlued representtions. The dvntge of rel-vlued coding over the binry coding includes incresed precision nd chromosome string becomes shorter. Also rel-vlued coding gives greter freedom to use specil crossover nd muttion techniques. Selection The selection (reproduction) opertor llows individul (strings) to be copied for possible inclusion in the next genertion. The chnce tht string will be copied is bsed on the individul s fitness vlue, clculted from fitness function. the most commonly used selection methods re rndom, proportionl, tournment, rnked-bsed nd elitism. Those re discussed in the previous lecture on evolutionry computing. Cross-over The im of crossover is to produce offspring from two prents selected ug reproduction opertor, which tkes plce t certin probbility clled the crossover probbility (C p ). The vlue of C p is set by the user, nd the suggested vlue rnges between 0.6 nd 0.8, lthough mx min min n 2

3 this vlue cn be domin dependnt. Severl crossover opertors re in wide use such s gle-point, uniform, nd multi-point crossover. Muttion There my not be enough vriety of strings in the initil popultion generted rndomly, which should be uniformly distributed within the entire problem spce. The im of muttion is to produce new genetic mteril into n existing individul nd mintin genetic diversity t ll genertions of GA. The muttion probbility should be kept very low (usully between 0.0 nd 0.02). Widely used muttion opertors re: bit inversion minly used for binry or Gry coding nd dding smll number usully used for rel vlue coding. An Exmple of GA: Simple Optimistion Problem The GA lgorithm used in the following exmple is bsed lmost exctly on the description given on the previous pge. The popultion sie will be 4, nd strings of bits of length 5 will be used. A crossover probbility of 0.6 is ssumed nd muttion probbility of With such low chnce of muttion, it does not occur in the following exmple. The problem is simply stted. Find the mximum vlue of the following function: 2 y = f ( x) = x 8x 5 0 x 25 In order to mke things esy for us, we will ssume tht the mximum is between 0 nd 25 (the ctul mximum is t x=4) nd tht the mximum is n integer vlue. y=f(x) <= X <= 25 Figure : vlues of f(x) 3

4 Coding scheme: In binry coding, we cn represent integer vlues in the rnge [0..3] with 5 bit string. e.g String Decoded Vlue Fitness function: which will give the reltive fitness vlues. Simplest method to employ here is to use the decoded x vlue to clculte the y coordinte nd use the y coordinte s the fitness vlue. 2 f ( x) = x 8x 5 Selection: fitness vlue for string i, s percentge, will be the y vlue t i divided by the sum of ll the y vlues for every string. Fitness function for string i in the string popultion is s follows Fitness vlue For exmple, sy y=x 2 nd we re trying to find the mximum vlue of the function between [0..3]. Then the following strings would hve the reltive fitness indicted below: i = f String x Vlue f(x) Reltive Fitness i f In relity, ce the vlue of the function we wnt to minimise cn tke on negtive vlues, the fitness function is slightly more complex thn the one used bove. However, in essence, the two remin equivlent. Running the GA The first Itertion: Firstly, we need to crete rndom popultion of strings. Sy we strt with the following: String Popultion

5 Now we perform selection. The fitness vlue of ech string is clculted nd the strings re selected the following number of times: String x Vlue f(x) Reltive Fitness Vlue No of times Selections With these selections, our mting pool now looks like this: String Popultion Finlly, the crossover probbilities need to be clculted (two crossovers need to be performed to crete new popultion of two). The GA clcultes tht it should perform splitting twice on two sets of rndomly selected genes. Crossover performs the following to crete the new popultion: Mting Pool Strings New Popultion So, t the end of the first itertion, our new popultion looks like the following: String Popultion x Vlue So, even fter one itertion, with no knowledge except for the reltive fitness vlue, the GA hs begun to quickly converge on the optimum vlue of 4. This is strtling, considering the GA knows nothing bout the problem spce in which it serches. It is effectively blind. Yet, just by exmining mesure of goodness, hving lrge number of points to exmine simultneously nd hving lrge mount of rndomistion thrown in, the GA efficiently serches the problem spce for possible nswers. 5

6 Genetic Progrmming A Genetic progrmming (GP) is n ppliction of the GA pproch designed to perform n utomtic derivtion of equtions, logicl rules or progrm functions developed by John R Ko (Ko, 992). Rther thn representing the solution to the problem s string of prmeters s in conventionl GA, the GP uses tree encoding scheme or structure. The leves of the tree, clled terminls, represent input vribles or numericl constnts. Their vlues re pssed to nodes, t the junctions of brnches in the tree, which perform some rithmeticl or progrm function before psg on the result further towrds the root of the tree. Genetic progrmming is much more powerful thn genetic lgorithms in tht the output of the genetic lgorithm is quntity, while the output of the genetic progrmming is nother computer progrm. Genetic progrmming works best for severl types of problems. The first type is where there is no idel solution, (for exmple, progrm tht drives cr). Furthermore, genetic progrmming is useful in finding solutions where the vribles re constntly chnging. In the previous cr exmple, the progrm will find one solution for smooth concrete highwy, while it will find totlly different solution for rough unpved rod. There re five mjor preprtory steps in ug GP for prticulr problem. These re required to be specified by the user set of terminls (e.g., the independent vribles of the problem, ero-rgument functions, nd rndom constnts) for ech brnch of the to-be-evolved progrm, set of functions for ech brnch of the to-be-evolved progrm, fitness mesure (for explicitly or implicitly mesuring the fitness of individuls in the popultion), selection of certin prmeters for controlling the run, nd selection of termintion criterion nd method for designting the result of the run. Genetic progrmming typiclly strts with popultion of rndomly generted computer progrms composed of the vilble progrmmtic ingredients within the hyperspce of vlid progrms, which cn be view s spce of rooted trees. Genetic progrmming itertively trnsforms popultion of computer progrms into new genertion of the popultion by pplying nlogs of nturlly occurring genetic opertions. These opertions re pplied to individul(s) selected from the popultion. The individuls re probbilisticlly selected to prticipte in the genetic opertions bsed on their fitness (s mesured by the fitness mesure provided by the humn user in the third preprtory step). The itertive trnsformtion of the popultion is executed inside the min genertionl loop of the run of genetic progrmming. 6

7 A generl GP lgorithm is given below g 0 ; genertion 0 C g {C g,n n=,2,, N} ; crete n initilise popultion rndomly Do while no(convergence) { F GP evlute (C g,n ) ;scertin fitness vlue C g,n select (C g,n, O g,n ) ; crete new popultion by selecting from old ;popultion nd offspring O g,n crossover (C g,n ) ; generte offspring by crossover O g,n mutte (O g,n ) ; generte offspring by muttion g g } Single best progrm in the popultion produced during the run - the best-so-fr individul is hrvested nd designted s the result of the run. If the run is successful, the result my be solution (or pproximte solution) to the problem. Fitness vlue: must determine how good the individuls re t solving the given problem. And s with genetic lgorithms, the crossover nd reproduction opertions re seprte from the ctul evlution of the fitness, mking the genetic progrmming opertors problemindependent. The mesurement of fitness is rther nebulous subject. Since, it is highly problemdependent, we consider mssging the results to mke fitness evlution much esier, through process known s scling. Simply put, scling stndrdies the mesurement of how fit prticulr individul is with respect to the rest of the popultion. Bsed on the fitness vlue, we go bout this selection for survivl in one of two wys: to choose the individuls with the highest fitness for reproduction. ``Only the strong survive.'' to ssign probbility tht prticulr individul will be selected for either reproduction or crossover. This method of choice, becuse it llows for more diversity. Some wek individuls my contin brnches of code, which re strong. The fitness function is determined subjectively. For exmple, we could include the depth of the tree s potentil qulity we wish to control, nd therefore we could develop fitness function, which tkes this into ccount. 7

8 Genetic progrmming opertors Crossover - Ko considers crossover, long with reproduction, to be the two foremost genetic opertions. It is minly responsible for the genetic diversity in the popultion of progrms. Similr to its performnce under genetic lgorithms, crossover opertes on two progrms ( binry opertor), nd produces two child progrms. Two rndom nodes re selected from within ech progrm nd then the resultnt sub-trees re swpped, generting two new progrms. These new progrms become prt of the next genertion of progrms to be evluted. A crossover opertion with different prents is shown in Figure 2 nd crossover opertion with identicl prents is shown in Figure 3. exp / - x exp / Figure 2: Crossover opertion with two different prents. / /

9 / / 3.4 Figure 3: Crossover opertion with identicl prents. 3.4 Muttion Severl muttion opertors hve been developed for GP. The most frequently used opertors re Function node muttion, Terminl node muttion, Swp muttion, Grow muttion, Gus muttion, Trunc muttion. Figure 4 explins the concept of muttion in tree structure encoding: ln exp - / x ln ln - / x 3 4 Figure 4: Different types of muttion. 9

10 Evolutionry Progrmming Evolutionry Progrmming (EP), originlly conceived by Lwrence J. Fogel in 960, is stochstic optimistion strtegy similr to GA. EP differs substntilly from GA nd GP in tht EP emphsises the development of behviourl models nd not the genetic models. EP is derived from simultion of dptive behviour in evolution. Tht is, EP considers phenotypic evolution. The evolutionry process consists of finding set of optiml behviours from spce of observble behviours. For this purpose, the fitness function mesures the behviour error of n individul with respect to the environment tht individul. EP further differs from Gs nd GP in tht no crossover opertion is implemented. Only muttion is used to produce the new popultion. For EP, like GAs, there is n underlying ssumption tht fitness lndscpe cn be chrcteried in terms of vribles, nd tht there is n optimum solution (or multiple such optim) in terms of those vribles. For exmple, if one were trying to find the shortest pth in Trveling Slesmn Problem, ech solution would be pth. The length of the pth could be expressed s number, which would serve s the solution's fitness. The fitness lndscpe for this problem could be chrcteried s hypersurfce proportionl to the pth lengths in spce of possible pths. The gol would be to find the globlly shortest pth in tht spce, or more prcticlly, to find very short tours very quickly. The bsic EP method involves 3 steps (Repet until threshold for itertion is exceeded or n dequte solution is obtined): () Choose n initil popultion of tril solutions t rndom. The number of solutions in popultion is highly relevnt to the speed of optimition, but no definite nswers re vilble s to how mny solutions re pproprite (other thn >) nd how mny solutions re just wsteful. (2) Ech solution is replicted into new popultion. Ech of these offspring solutions re mutted ccording to distribution of muttion types, rnging from minor to extreme with continuum of muttion types between. The severity of muttion is judged on the bsis of the functionl chnge imposed on the prents. (3) Ech offspring solution is ssessed by computing its fitness. Typiclly, stochstic tournment is held to determine N solutions to be retined for the popultion of solutions, lthough this is occsionlly performed deterministiclly. There is no requirement tht the popultion sie be held constnt, however, nor tht only gle offspring be generted from ech prent. 0

11 A generl EP lgorithm is given below g 0 ; genertion 0 C g {C g,n n=,2,, N} ; initilise popultion Do while no(convergence) { F EP evlute (C g,n ) ;fitness vlue O g,n mutte (C g,n ) ; generte offspring by muttion C g,n select (C g,n, O g,n ) ; crete new popultion by selecting from old ;popultion nd offsprin } g g Evolutionry opertors Muttion Muttion is pplied to ech of the individuls t certin probbility. The muttion opertor to be used depends on the specific ppliction. Selection After muttion, the new popultion is selected, in one of the following wys: All the individuls prents nd offsprings hve the sme chnce to be selected. Any of the selection opertor discussed erlier (in lecture 8) cn be used to crete new popultion. An elitist trnsfer the group of best prents to the next genertion. The reminder of the popultion is selected from the reminder of the prents nd offsprings. First cull the worst prents nd offsprings re mrked nd then select the reminder good individuls. Exmple: Finite-Stte Mchine Evolutionry Progrmming (EP) ws originlly developed to evolve finite-stte mchines (FSM). A FSM is essentilly computer progrm, which represents sequence of ctions tht must be executed, where ech ction depends on the current stte of the mchine nd n input. A 3-stte FSM is given in Figure 5 nd response of the FSM to given input symbols is shown in Tble. Show Chromosome representtion for the FSM. Explin how muttion opertions cn be performed on chromosomes.

12 2 0/c 0/b 0/b / /b /c 3 Figure 5: A 3-stte Finite Stte Mchine Tble : Response of FSM. Present stte Input Next stte Output b c b b c 2

13 Evolutionry strtegies Evolution strtegies (ES) were developed s method to solve prmeter optimistion problems by Schwefel (Schwefel, 994). Evolution-strtegic optimistion is bsed on the hypothesis tht during the biologicl evolution the lws of heredity hve been developed for fstest phylogenetic dpttion. ES imitte, in contrst to the GAs, the effects of genetic procedures on the phenotype. The presumption for coding the vribles in the ES is the relition of sufficient strong cuslity i.e. smll chnges of the cuse must crete smll chnges of the effect. The climx of the theory of the Evolution-Strtegy is the discovery of the Evolution Window: Evolutionry progress tkes plce only within very nrrow bnd of the muttion step sie. This fct leds to the necessity for rule of self-dpttion of the muttion step sie. Figure 6: Evolution window The erliest ESs were bsed on popultion consisting of one individul only. There ws lso only one genetic opertor used in the evolution process muttion. However, the interesting v = x,σ, where ide ws to represent n individul s pir of rel-vlued vectors ( ) x represents point in the serch spce nd σ is vector of stndrd devitions. Muttions re relised by replcing x by x ( t ) = x( t) N(0, σ ) where N ( 0, σ ) is vector of independent rndom Gus numbers with ero men nd stndrd devition. This is in ccordnce with the biologicl observtion tht smller chnges occur more often thn lrger ones. The offspring (the mutted individuls) is ccepted s new member of the popultion iff it hs improved fitness nd ll constrints re stisfied. For exmple, if f is the objective function without constrints to be mximised, n offspring ( t ),σ (t),σ f x( t ) > f x( t), otherwise, the offspring is ( x ) replces its prent ( x ) iff ( ) ( ) eliminted nd the popultion remins unchnged. 3

14 When implemented to solve rel-vlued function optimistion problems, both typiclly operte on the rel vlues themselves (rther thn ny coding of the rel vlues s is often done in GAs). Multivrite ero men Gus muttions re pplied to ech prent in popultion nd selection mechnism is pplied to determine which solutions to remove (i.e.,"cull") from the popultion. The similrities extend to the use of self-dptive methods for determining the pproprite muttions to use - methods in which ech prent crries not only potentil solution to the problem t hnd, but lso informtion on how it will distribute new trils (offspring). The following pseudo code is n illustrtion of generl ES lgorithm g 0 ; genertion 0 C g {C g,n n=,2,, N} ; initilise popultion F EP evlute (C g,n ) ;evlute fitness of ech individul Do while no(convergence) { For l = to λ ( λ no of offsprings) { P g,n (n>2) select(c g,n ) ;select t rndom O g,λ crossover (P g,µ ) ;cross over O g,λ mutte (O g,µ ) ; mutte offsprings F EP evlute (O g,λ ) } C g,µ select (C g,µ, O g,λ ) ; select best µ individuls from prent nd offsprin } g g Selection opertors ES typiclly uses deterministic selection in which the worst solutions re purged from the popultion bsed directly on their function evlution. For ech genertion λ offspring re generted from µ prents nd mutted. After crossover nd muttion the individuls for the next genertion re selected. Two strtegies hve been developed: ( µ λ) -ES: In this cse the ES genertes λ offspring from µ prents, with λ µ. The next genertion consists of the µ best individuls selected from µ prents nd λ µ λ -ES implements elitism to ensure tht the fittest prents survive offspring. The ( ) to the next genertion. ( µ, λ) -ES: In this cse, the next genertion consists of the µ best individuls selected from λ offspring. The ( µ, λ) -ES requires tht µ < λ. By doing this, the life of ech individul is limited to one genertion. This llows the ( µ, λ) -ES to perform 4

15 better on problems with n optimum moving over time, or on problems where the objective function is noisy. The nottions ()-ES, (λ)-es, (, λ)-es, (m/µ, λ)-es chrcterie Evolution-Strtegies with n increg level of imittion of biologicl evolution. The letter m mens the totl number of prents, µ mrks the number of prents, which will be recombined, nd λ stnds for the number of offspring. Recombintion opertors ES is n bstrction of evolution t the level of individul behviour. Mny forms of recombintion hve been implemented within ES. Agin, the effectiveness of such opertors depends on the problem t hnd. The opertors used in (µλ)-es nd (µ, λ)-es incorportes two-level lerning: their control prmeter σ is no longer constnt, nor it is chnged by some deterministic lgorithm but it is incorported in the structure of the individul nd undergoes the evolution process. To produce n offspring, the system cts in severl stges: Select two individuls ( x, σ ) = ( x, L, ),( σ, L, σ )) x n ( x, σ ) = ( x, L, ),( σ, L, σ )) x n n n nd pply recombintion (crossover) opertor. There re two types of crossover Discrete, where the new offspring is q q q q ( x, σ ) = ( x, L, x ),( σ, L, σ )) where q i = or q i = 2 i.e. ech component comes from either from first or second preselected prent. Another opertor tht is used is intermedite recombintion, in which the vectors of two prents re verged together, element by element, to form new offspring where the new offspring is ( x, σ ) = ( ( x x ) / 2,, L,( ) / 2 ), ( σ σ ) / 2,, L,( σ σ ) / 2 ) x n x n n n n n e.g Intermedite recombintion

16 Apply muttion to the offsping ( x,σ ) obtined, the resulting new offspring is (,σ ) N (0, σ ) x, where σ = σ. e, x = x N( 0, σ ), nd where σ is prmeter of the method. The effects of these opertors reflect the behviourl s opposed to structurl interprettion of the representtion ce knowledge of the vlues of vector elements is used to derive new vector elements. Exmple: A simple neurl network with fuified input is shown in Figure 7 below. The ctivtion function f(.) is defined s f ( x) =, where defines the shpe of the x e sigmoid function. Explin how ES cn be pplied to optimise the prmeters of MFs nd sigmoid function. A O i x Σ f(.) A 2 B Σ y x 2 Σ f(.) B 2 Figure 7: Neurl Network with Fuified Inputs 6

17 References Hollnd, J.H. (975). Adpttion in Nturl nd Artificil Systems, University Michign Press, Ann Arbor. Goldberg, Dvid E. (989). Genetic Algorithms in Serch, Optimition, nd Mchine Lerning, Addison Wesley Publishing Compny. De Jong, K.A. (975). Anlysis of the behviour of clss genetic dptive systems, PhD Thesis, Dept. of Computer nd Communictions sciences, University of Michign, Ann Arbor. Crun, R.A. nd Schffer, J.D. (988). Representtion nd hidden bis: Gry vs. binry coding, Proceedings of 6th Int. Conference on Mchine Lerning, pp Michlewic, Z. (992). Genetic Algorithms Dt Structures = Evolution Progrms, Springer Verlg. Brlette, M.F. (99). Initilistion, muttion, nd selection methods in genetic lgorithms for function optimistion, Proceedings of ICGA 4, pp Fogel, Dvid, B. (995). Evolutionry Computtion - Towrd New Philosophy of Mchine Intelligence. Fogel, LJ., Owens, A.J. nd Wlsh M.J. (996). Artificil Intelligence through Simulted Evolutionry, Chichester: John Wiley. Ko, John R. (992). Genetic Progrmming: On the Progrmming of Computers by Mens of Nturl Selection. Cmbridge, MA: The MIT Press. Schwefel H.-P (995). Evolution nd Optimum Seeking, Chichester: John Wiley. 7