A simplied Genetic Algorithm, essentially a mutation-selection scheme, is analyzed

Transcription

1 The Interacton of Mutaton Rate, Selecton, and Self-Adaptaton Wthn a Genetc Algorthm Thomas Back a a Unversty of Dortmund, Department of Computer Scence, P.O. Box 5 5, 46 Dortmund 5, Germany Abstract A smpled Genetc Algorthm, essentally a mutaton-selecton scheme, s analyzed theoretcally wth respect to convergence rates and success probabltes for mutatons on a smple bt-countng objectve functon. The theoretcal results are then conrmed by expermental runs and provde a bass for the explanaton of the self-adaptaton mechansm of mutaton rates, whch turns out to enable a near-optmal schedule for the mutaton rate by means o self-organzng process. Ths way, a rst step towards a theoretcal foundaton for the usefulness of self-adaptaton wthn Genetc Algorthms s presented. 1. INTRODUCTION For Genetc Algorthms (GAs) [1], a large number of expermental nvestgatons towards dscoverng optmal settngs of the exogenous parameters, manly the mutaton rate p m, the crossover probablty p c, and the populaton sze, are reported n lterature. Some resultng settngs are p m = :1, p c = :6, 5 1 [2], p m 2 [:5; :1], p c 2 [:75; :95], 2 3 [3], p m = :1, p c = :95, = 3 [4]. The settngs of Grefenstette were obtaned by usng a meta-level GA whch optmzes the parameter settngs o GA. These nvestgatons, whch serve as startng ponts to most applcatons of GAs, are all based on the dea to nd a settng whch maxmzes the performance over a set of test functons of very derent characterstcs and s constant for all these derent functons. Although ths s n accordance wth Holland's denton of robust adaptve plans (see [1], page 27), a recent result of Hart and Belew shows that no sngle nondetermnstc algorthm exsts whch s able to approach the optmal functon values orbtrary functons f : f; 1g l Z to a certan accuracy n an ecent way,.e. n a tme polynomal w.r.t. l [5]. Ths result s extremely mportant snce t essentally means that gven an Evolutonary Algorthm the most ecent parameter settng must depend on the tness functon, at least. It s qute evdent that strategy parameters should vary durng a run for ndng an optmum n order to acheve even hgher ecences. In the followng, deas from Evoluton Strateges (ESs) [6, 7] on the self-adaptaton of strategy parameters are combned wth a smple GA n order to nd a way for the theoretcal nvestgaton o hybrd algorthm. Furthermore, some new arguments concernng the mportance of mutaton n genetc search are presented. In secton 2 a theoretcal argument for ntroducng a tme-dependent schedule of the mutaton probablty s pre-

2 sented for a smple objectve functon. Based upon ths, n secton 3 the convergence velocty s calculated for a mutaton-selecton mechansm, and the result s conrmed by experments. These results are used n secton 4 to demonstrate that the self-adaptaton mechansm for mutaton rates n GAs (rst ntroduced to GAs n [8]) provdes a selforganzng near-optmal schedule for mutaton rates. 2. SEARCH BY MUTATION Holland has ntroduced mutaton to Genetc Algorthms as a \background operator" ([1], page 111), whch assures the prncpal possblty to recover from lost alleles,.e. alleles whch are converged wthn the populaton. In contrast to ESs, whch use mutaton as the man search operator, mutatons n GAs are not manly ntended to serve as a mechansm for generatng new structures. Furthermore, GAs have n most cases been analyzed n terms of schemata (hyperplanes n l-dmensonal btspace) and the dsruptve eect of mutaton and crossover to schemata, an approach whch s absolutely ndependent ony partcular objectve functon. In contrast to schema analyss approaches ths work analyzes a partcular objectve functon dened on btstrngs whch s analogous to the corrdor model n ESs. Addtonally, n the followng mutaton s used as the only search operator n a GA, and the ecency of ths search by a mutaton-selecton algorthm s demonstrated. We focus on the smple bt-countng functon f : f; 1g l f; : : : ; lg ; f( 1 ; : : : ; l ) = l =1 (1) whch s unmodal and an easy problem for GAs. For ths functon, t s possble to calculate the success probablty for mutatons,.e. the probablty that mutaton on ndvdual ncreases the tness of the ndvdual. More formally, let m : f; 1g l f; 1g l denote the mutaton operator and a 2 f; 1g l an ndvdual. Then, we dene the followng probabltes: p + a = P(f( m (a)) > f(a)) success probablty (2) p a = P(f( m (a)) < f(a)) falure probablty (3) p a = P(f( m (a)) = f(a)) stagnaton probablty (4) These are the probabltes of mprovement, worsenng, or constancy, respectvely, of the ndvduals' tness subject to mutaton. For the bt-countng functon (1), these probabltes are gven n theorem 1. Theorem 1 Let a 2 f; 1g l be a btstrng of length l, p the bt-mutaton probablty, and := f(a) denotes the number of ones wthn a,.e. ts functon value under objectve functon (1). Then: p + a = = p (1 p) fa l j=+1 j p j (1 p) a j (5)

3 p a = p a = a = = p (1 p) a j=+1 j p j (1 p) fa j (6) p 2 (1 p) l 2 (7) Proof: To gve a proof for the expresson for p + a, two cases can be dstngushed: (1) None of the one-bts n a mutates. Then, at least one zero-bt up to all zero-bts must undergo mutaton n order to ncrease the number of ones,.e. we have between = 1 and = mutatons of zero-bts. The choce, whch of the choces to mutate bts bts s mutated, s arbtrary, such that there exst l (probablty p ) and to leave l bts unmutated (probablty (1 p) l ). Combnng ths, the rst case results n a success probablty p + 1 a = =1 p (1 p) l (8) (2) Between = 1 and = ones are mutated to zero-bts. Then, n order to ncrease the number of ones n spte of ths ntal loss, at least j = + 1 up to j = zero-bts must be mutated to ones. From the rst case of ths proof the expresson for the mutaton of the zero-bts can be taken, whch has to be multpled by the probablty of mutatng ones, summng up over all values of : p + 2 = =1 2 4 p (1 p) a j=+1 j 13 p j (1 p) a j A5 (9) Summng up the probabltes p + 1 and p + 2 and usng the fact that p + 1 s a specal case of the outer sums' argument n p + 2 (for = ), one obtans p + a = p p + 2. By an analogous calculaton, the expresson for p a s derved. Furthermore, stagnaton can be ncorporated nto both expressons for p + and a p a by startng the second sum at j = nstead j = + 1, meanng that at least the same number of zeroes and ones must be mutated. Then, by takng only the summand for j = n the nner sum, the expresson for p a results. Of course, the dentty p + a + p a + p a = 1 holds for these probabltes. However, the proof of ths dentty nvolves a longsh calculaton and s therefore omtted here. Q.E.D. From ths theorem, expressons can easly be derved for the probablty p + (k) = a P(f( m (a)) = f(a) + k) for mprovng the btstrng a by addng exactly k ones ( k l f(a)) and p a (k) = P(f( m (a)) = f(a) k) for worsenng the btstrng a by subtractng exactly k ones ( k f(a)). The expressons result from takng nto account only the summand for j = + k n the nner sum of the expressons presented n theorem 1. It should explctly be noted that the expressons for p + (k) and a p a (k) now nclude the stagnaton probablty,.e. p + a () = p a () = p a:

4 Corollary 1 k : p + a (k) = = k : p a (k) = l = + k + k p 2+k (1 p) l 2 k (1) p 2+k (1 p) l 2 k (11) Success probablty Optmal mutaton rate (1+3) (1+2) (1+15) (1+1) (1+ 5) (1+ 2) (1+ 1) Mutaton rate Number of correct bts Fgure 1: Success probabltes wth l = 1, fa 2 f3; : : : ; 95g (left) and optmal mutaton rate for derent (1+)-settngs, 2 f1; 2; 5; 1; 15; 2; 3g (rght). It s extremely nstructve to plot, gven the total strng length l as well as the number of bts whch are already correct n the btstrng a, the dependence of the success probablty p + a on the mutaton rate p. For l = 1, the resultng curves are shown n the left part of gure 1 for derent values of correct bts n the strng (.e. the curves are plotted for 2 f3; 45; 49; 5; 52; 55; 6; 65; 7; 75; 8; 85; 9; 95g and l = 1). In both gures the legend descrbes the plots n the order from top to bottom. Clearly, when less than half of the bts n a strng are correct, the optmal mutaton probablty s one (nvertng all bts guarantees mprovement). When half of the bts are correct, the optmal probablty s :5 and decreases quckly for even small ncreases of the number of correct bts. Although t s easy to calculate the expresson for the dervatve of p + a wth respect to p, the resultng equaton when settng dp+ a = can not be solved for p n a closed dp analytcal form. Calculatng the zeroes by means of numercal methods yelds the lowest curve plotted n the rght part of gure 1, where the dependence of the optmal mutaton rate on the number of correct bts s shown. The remanng curves wll be dscussed and explaned n secton 3. The curve gves a clear demonstraton of the strong dependence of the optmal mutaton rate settng on the current Hammng dstance to the optmal soluton. In other words, the mutaton rate should not be constant but should decrease over tme durng the search.

5 3. SELECTION AND PROGRESS RATES So far, nothng but a sngle ndvdual subject to mutaton was taken nto account. In order to combne the result on mutaton rates wth the concepts o populaton and of selecton, we focus here on the (+)-selecton and the (,)-selecton. In the latter case, the best ndvduals out of the osprng are selected to become parents of the next generaton, whle n the former case the best ndvduals are selected from the set of parents and osprng ndvduals. Ths way, the (+)-scheme s eltst snce t wll only accept mprovements, a behavour advantageous for convergence velocty but dsadvantageous when convergence relablty and self-adaptaton of mutaton rates are desred. Both selecton mechansms were orgnally ntroduced by Schwefel n ESs [7], later on the (,)-selecton was tested successfully also n the context of GAs [9]. Followng the theory developed for ESs by Schwefel [7] and Rechenberg [6], the convergence velocty ',.e. the expectaton of the mprovement per generaton, can be derved for a (1,)-GA and a (1+)-GA on objectve functon s dened n equaton (1). To do so, the probablty densty functon w 1 (k) s needed, whch descrbes the case that the best ndvdual among the osprng consttute an mprovement by k: w 1 (k) = =1 w k =k w k <k (12) Ths expresson results from the fact that due to the dscrete nature of the objectve functon any number of ndvduals between = 1 and = coverng the dstance k can be the best after mutaton. The probablty for ndvduals to cover the dstance k at the same tme s just wk =k, the remanng ndvduals cover a smaller dstance wth probablty w. For each choce of k <k best ndvduals out of there exst possbltes, and the sum must be calculated for all possble values of (1 ). For a (1,)-GA k may take postve as well as negatve values, whle for a (1+)-GA k may only be postve or zero (only mprovements or stagnaton are accepted). From the calculatons n secton 2, the probabltes w k =k, w k <k and w k >k (for an mprovement by more than k) can be gven ( k ): w k =k = w k >k = 8 < : p 8 >< >: + a (k) ; k p a ( k) ; k < a =k+1 1 =k+1 p + a () ; k p a ( ) + a = p + a () ; k < w k <k = 1 w k =k w k >k (15) Altogether, an expresson for ' can be formulated, whch s gven n theorem 2. Theorem 2 Let w 1 (k) be the probablty densty functon as dened n (12) and (13). For s n equaton (1), the expectaton of the convergence velocty ' for a (1+)-GA or (1,)-GA (13) (14)

6 s '(l; ; p; ) = a k=k mn k w 1 (k) = a k=k mn k =1 w k =k w k <k (16) where k mn = for a (1,)-GA and k mn = 1 for a (1+)-GA (strctly speakng, k mn =, but ths summand does not contrbute anythng to the sum n equaton (16)). In the followng, we wll wrte ' (1;) n the former case and ' (1+) n the latter case. Proof: The expresson for ' s clear by denton of the expectaton value and the remarks on the range allowed for k as gven before. Q.E.D. Progress per generaton (1+ 1) (1+ 5) (1+1) (1+2) (1, 5) (1,1) (1,2) Progress per generaton (1+ 1) (1+ 5) (1+1) (1+2) (1, 5) (1,1) (1,2) Mutaton rate Mutaton rate Fgure 2: Progress plots for fa = 6 (left) and fa = 8 (rght). The convergence rates are always absolute, local values,.e. they depend on the current qualty of the best ndvdual and they gve the mprovement w.r.t. the qualty functon value whch s expected for the best ndvdual. In gure 2 the resultng plots o (1+)- GA and a (1,)-GA are shown for derent values of 2 f1; 5; 1; 2g, agan for strngs of length l = 1 and assumng actual functon values of = 6 (left plot) and = 8 (rght plot). It s obvous and ntutvely clear that, when the number of osprng ncreases, the expected progress ncreases for both the (1+)-GA and the (1,)-GA. Whle there s no derence between the (1+)-selecton scheme and the (1,)-selecton scheme for small mutaton rates (and values of not too small), progress decreases very rapdly for the (1,)-strateges above the optmal mutaton rate, becomng even negatve as soon as the mutaton rate s too large. The (1+)-strategy always yelds a postve progress approachng zero only from above for growng mutaton rate. Furthermore, the progress curves are narrower for larger than for smaller,.e. the range of mutaton rates for whch progress s acheved becomes smaller and smaller as the optmum s approached. Once agan, t s nterestng to have a look at the dependence of the optmal mutaton rate on the number of correct bts, whch can be calculated by numercal methods. Ths

7 s done for a (1+)-GA wth 2 f1; 2; 5; 1; 15; 2; 3g n the rght part of gure 1. The plot for the (1+1)-case s of course dentcal to the result obtaned by maxmzng p + a n secton 2, snce p + a = P l k=1 p+ a (k) and ' (1+1) = P l k=1 k p+ a (k). Whle the values of p (l=2) = :5 and p (l 1) = 1=l reman constant when ncreasng, the optmal mutaton rate n between these boundares grows slowler and slowler when s ncreased. Most remarkable s the strong ncrease of p as changes from 1 to 2,.e. when the \redundancy" of osprng allows for the rsk of producng garbage, whch s at the same tme a chance of producng larger mprovements. Best objectve functon value (1+2), Optmal s.-dev., Optmal Standard s.-dev., Standard (1+2),.1 s.-dev.,.1 Standard w.o. Recomb. Standard w.o., s.-dev Generaton Progress (1+2), practcal (1+2), practcal, std.-dev. (1+2), theoretcal Number of correct bts Fgure 3: Comparson of the convergence behavour of derent varants (left) and comparson of theoretcal and expermental progress results for an optmally adjusted mutaton rate (rght) In order to perform a rst expermental test of the theory derved so far, tests were run for the case o (1+2)-GA and l = 1. It should be noted that for = 1 recombnaton has no eect, snce recombnng dentcal ndvduals always yelds the parent ndvdual. The best performance values of the followng algorthmc varants are compared n the left plot of gure 3: A (1+2)-GA whch adjusts ts mutaton rate to the optmal value wth respect to maxmzaton of the expected progress ' (1+2) (l = 1; ; p) accordng to the measured performance value of the best ndvdual n each generaton (ths varant s labeled \(1+2), Optmal" n gure 3). A standard GA wth = 2, p m = :1, p c = :6, one-pont crossover, and proportonal selecton (labeled \Standard"). A (1+2)-GA wth a constant mutaton rate of p m = :1 (labeled \(1+2),.1"). A GA wth = 2, p m = :1, p c =, and proportonal selecton,.e. a standard GA wthout recombnaton (labeled \Standard w.o. Recomb.").

8 These curves are plotted together wth ther standard devatons (results are averaged for 1 runs, each) from left to rght n the order gven above n gure 3 (left part). The resultng plot for the optmally adjusted mutaton rate has a very small standard devaton and reaches the one percent vcnty of the optmum a factor larger than sx tmes faster than both the standard GA and the (1+2)-GA wth p m = :1. Addtonally, for the optmally adjusted mutaton rate t s possble to check the dependence of the local progress ' (1+2) (l = 1; ) on the number of correct bts. The expermental progress values are calculated from the data by extractng the mprovement of the best value between generatons and plottng the mprovement f a (t + 1) f a (t) (.e. the progress per generaton) as a functon of the actual best performance f a(t). The result and ts standard devaton s plotted n the rght part of gure 3, together wth the theoretcally optmal progress ' (1+2) () = maxf' (1+2) (l = 1; ; p) j p 1g. A dscrepancy between theoretcal and expermental values at the begnnng of the run can be explaned by the fact that the GA run starts wth a populaton of 2 ndvduals ntalzed at random nstead of one as assumed n ths theory. As a consequence, the ntal expermental progress s much larger than the theoretcal one and the experment starts at a best soluton of qualty 63. However, after only two generatons the expermental progress curve approaches the theoretcal one rather well, ndcatng that the theory developed here provdes a reasonably vald mathematcal approach. 4. TOWARDS UNDERSTANDING SELF-ADAPTATION As demonstrated n the prevous sectons, the optmal schedule for the mutaton rate depends on several quanttes,.e. the length l of the btstrngs, the current objectve functon value, the populaton sze, and, of course, the actual objectve functon f. Some authors have ether emprcally observed that a tme-dependent decrease of the mutaton rate s advantageous (see [1], who does not present an exact expresson for the tme-schedule used) q or derved a smlar result by a theoretcal approach (see [11]). The expresson p m (t) = = exp( t=2) p (,, are constants) by Hesser and Manner l ntroduces a tme-dependence resultng n curves over tme whch are smlar to that one presented here. The problem s, however, that a predened determnstc tme schedule can not take nto account the partculartes of derent tness functons but s agan xed by some new exogenous parameters such that the schedule must be tuned accordng to the tness functon, resemblng the temperature schedule tunng problem n Smulated Annealng. Self-adaptaton of mutaton rates s a derent approach, ntendng a selforganzng schedule of mutaton rates durng the course of evoluton. Bascally, the dea stems from Evoluton Strateges [7]. For GAs, a rst approach towards self-adaptaton of the mutaton rates was presented recently, demonstratng the advantages of the approach expermentally [8]. The basc workng mechansm of self-adaptve mutaton wthn GAs can be descrbed brey as follows: Each ndvdual s extended by addtonal ^l bts (^l = 2 turned out to be a reasonable choce) whch are ntalzed at random. These bts are nterpreted as the encodng o real number between and :5, the ndvduals' prvate mutaton rate, whch determnes the mutaton probablty for the ndvdual. On average, half of the bts

9 can expected to be correct, such that the upper bound of :5 on possble values of the mutaton rate makes sense. Mutaton of the ndvdual works by decodng the mutaton rate of the ndvdual, resultng n mutaton probablty ^p, then mutatng wth mutaton rate ^p the part of the ndvdual whch encodes ^p, agan decodng the (now mutated) mutaton rate, whch yelds mutaton rate p, and nally mutatng the object varable part of the ndvdual by usng mutaton rate p. Ths way, prvate mutaton rates lnked to the ndvduals are subject to mutaton and therefore change over tme probablstcally. However, the process s drected due to the selecton process, whch favours good object varable nformaton as well as, ndrectly, approprate strategy parameter settngs. Expermentally, a rato of = 1=6 turned out to be a reasonable settng for the selecton pressure, balancng between mantanng sucent dversty wthn the strategy parameters on the one hand and drvng the search towards better regons of the search space on the other hand [8]. Best objectve functon value (1+2), Optmal s.-dev., Optmal (4,2), Adaptve s.-dev., Adaptve Generaton Mutaton rate Optmal Adaptve, mn Adaptve, avg Adaptve, max Number of correct bts Fgure 4: Course of the optmzaton (left) and schedule of the mutaton rate (rght) for an optmally adjusted mutaton rate and self-adaptve mutaton rates In analogy to gure 3, n the left part of gure 4 the dependence of the best objectve functon value over the number of generatons s plotted, agan for a (1+2)-GA wth a mutaton rate adjusted optmally accordng to the theoretcal schedule. Only slghtly worse behaves the (4,2)-GA wth self-adaptve mutaton rates, whch s also shown n the same graph. Ths provdes a strong hnt to the assumpton that the self-adaptaton mechansm enables the algorthm to encounter a near-optmal schedule of the mutaton rates by ts own. Ths assumpton s conrmed by the graph shown n the rght part of gure 4 where the mutaton rates over the number of correct bts are shown for the theoretcal case o (1+2)-GA as well as for the average, mnmum, and maxmum mutaton rates found n the genetc materal of self-adaptve mutaton rates for the (4,2)- GA. It t natural here to ask, why a (1+2)-GA and a (4,2)-GA are compared wth one

10 another. The theory developed n secton 3 can be extended n an approxmatve way to cover the general (,)-case as well as the (+)-case. However, the theoretcal plots for the optmal mutaton rate for a (1+2)-GA and a (4,2)-GA are very smlar, such that the (1+2)-GA s a good representatve. The further elaboraton of the theory s not presented here due to space lmtatons. The graph demonstrates clearly that the optmal mutaton rates are always avalable n the populaton durng the course of evoluton, such that by means of selecton the algorthm can make use of the most approprate mutaton rates. Furthermore, a large dversty of mutaton rates s avalable such that also at the begnnng of the search approprate mutaton rates can be utlzed. Ths leads to the concluson that the self-adaptaton mechansm works by an nteracton of sucent genetc dversty wthn the strategy parameters, an mplct lnk between advantageous settngs of strategy parameters and good objectve functon values, and hence a preference of useful strategy parameters by means o well chosen selectve pressure, strong enough to elmnate garbage and soft enough to mantan dversty of strategy parameters. 1 J. H. Holland. Adaptaton n natural and artcal systems. (The Unversty of Mchgan Press, Ann Arbor, 1975). 2 K. De Jong. An analyss of the behavour o class of genetc adaptve systems. PhD thess, (Unversty of Mchgan, 1975). 3 J. D. Schaer, R. A. Caruana, L. J. Eshelman, and R. Das. A study of control parameters aectng onlne performance of genetc algorthms for functon optmzaton. In Schaer [13], pages 51{6. 4 J. J. Grefenstette. Optmzaton of control parameters for genetc algorthms. IEEE Transactons on Systems, Man and Cybernetcs, SMC{16(1):122{128, (1986). 5 W. E. Hart and R. K. Belew. Optmzng an arbtrary functon s hard for the genetc algorthm. In Belew and Booker [12], pages 19{ I. Rechenberg. Evolutonsstratege: Optmerung technscher Systeme nach Prnzpen der bologschen Evoluton. (Frommann{Holzboog Verlag, Stuttgart, 1973). 7 H.-P. Schwefel. Numercal Optmzaton of Computer Models. (Wley, Chchester, 1981). 8 T. Back. Self-adaptaton n genetc algorthms. In Proceedngs of the Frst European Conference on Artcal Lfe, December 11-13, Pars, France, (MIT Press, 1992). 9 T. Back and F. Homester. Extended selecton mechansms n genetc algorthms. In Belew and Booker [12], pages 92{99. 1 T. C. Fogarty. Varyng the probablty of mutaton n the genetc algorthm. In Schaer [13], pages 14{ J. Hesser and R. Manner. Towards an optmal mutaton probablty n genetc algorthms. In H.-P. Schwefel and R. Manner, eds., Parallel Problem Solvng from Nature, volume 496 of Lecture Notes n Computer Scence, pages 23{32. (Sprnger, 1991). 12 R. K. Belew and L. B. Booker, eds. Proceedngs of the Fourth Internatonal Conference on Genetc Algorthms and ther Applcatons, Unversty of Calforna, San Dego, USA, (Morgan Kaufmann Publshers, 1991). 13 J. D. Schaer, ed. Proceedngs of the Thrd Internatonal Conference on Genetc Algorthms and Ther Applcatons, San Mateo, Calforna, June (Morgan Kaufmann Publshers, 1989).