Outline Genetic Progrmming Evolutionry strtegies Genetic progrmming Summry Bsed on the mteril provided y Professor Michel Negnevitsky Evolutionry Strtegies An pproch simulting nturl evolution ws proposed in Germny in the erly 1960s. Unlike genetic lgorithms, this pproch clled n evolutionry strtegy ws designed to solve engineering optimiztion prolems. In 1963 two students of the Technicl University of Berlin, Ingo Rechenerg nd Hns-Pul Schwefel, were working on the serch for the optiml shpes of odies in flow. They decided to try rndom chnges in the prmeters defining the shpe following the exmple of nturl muttion. As result, the evolution strtegy ws orn. Evolutionry strtegies were developed s n lterntive to the engineer s intuition. Unlike GAs, evolution strtegies use only muttion opertor.
Bsic evolution strtegies In its simplest form, termed s (11)-evolution strtegy, one prent genertes one offspring per genertion y pplying normlly distriuted muttion. The (11)-evolutionry strtegy cn e implemented s follows: Step 1: Choose the numer of prmeters N to represent the prolem, nd then determine fesile rnge for ech prmeter: x1 min, x1mx, x2 min, x2mx,..., x Nmin, xnmx Define stndrd devition for ech prmeter nd the function to e optimized. Step 2: Rndomly select n initil vlue for ech prmeter from the respective fesile rnge. The set of these prmeters will constitute the initil popultion of prent prmeters: x 1, x 2,..., x N Step 3: Clculte the solution ssocited with the prent prmeters: X = f (x 1, x 2,..., x N ) Step 4: Crete new (offspring) prmeter y dding normlly distriuted rndom vrile with men zero nd pre-selected devition to ech prent prmeter: xi xi 0, i = 1, 2,..., N Normlly distriuted muttions with men zero reflect the nturl process of evolution (smller chnges occur more frequently thn lrger ones). Step 5: Clculte the solution ssocited with the offspring prmeters: X f x1, x2,..., x N Step 6: Compre the offspring solution the prents. If the offspring solution is etter thn the prents, replce the prent popultion with the offspring popultion. Otherwise, keep the prent prmeters. Step 7: Go to Step 4, nd repet the process until stisfctory solution is reched, or specified numer of genertions is considered. d
An evolutionry strtegy reflects the nture of chromosome. A single gene my simultneously ffect severl chrcteristics of the living orgnism. On the other hnd, single chrcteristic of n individul my e determined y the simultneous interctions of severl genes. The nturl selection ec cts on collection o of genes, es, not on single gene in isoltion. Genetic progrmming One of the centrl prolems in computer science is how to mke computers solve prolems without eing explicitly progrmmed to do so. Genetic progrmming offers solution through the evolution of computer progrms y methods of nturl selection. In fct, genetic progrmming is n extension of the conventionl genetic lgorithm, ut the gol of genetic progrmming is not just to evolve itstring representtion of some prolem ut the computer code tht solves the prolem. Genetic progrmming n evolutionry computtion lgorithm stimulted in the 1990s y John Koz. Genetic progrmming serches the spce of possile computer progrms for progrm tht is highly fit for solving the prolem t hnd. Any computer progrm is sequence of opertions (functions) pplied to vlues (rguments), ut different progrmming lnguges my include different types of sttements nd opertions, nd hve different syntctic restrictions. Since genetic progrmming mnipultes progrms y pplying genetic opertors, progrmming lnguge should permit computer progrm to e mnipulted s dt nd the newly creted ddt to e executed s progrm. For these resons, LISP ws chosen s the min lnguge for genetic progrmming.
LISP structure LISP hs highly symol-oriented structure. Its sic dt structures re toms nd lists. An tom is the smllest indivisile element of the LISP syntx. The numer 21, the symol X nd the string This is string re exmples of LISP toms. A list is n oject composed of toms nd/or other lists. LISP lists re written s n ordered d collection of items inside pir of prentheses. For exmple, the list ( ( AB)C) LISP structure clls for the ppliction i of the sutrction function () to two rguments, nmely the list (A B) nd the tom C. First, LISP pplies the multipliction function () to the toms A nd B. Once the list (A B) is evluted, LISP pplies the sutrction function () to the two rguments, nd thus evlutes the entire list ( ( A B) C). Grphicl representtion of LISP S-expressions Both toms nd lists re clled symolic expressions or S-expressions. In LISP, ll dt nd ll progrms re S-expressions. This gives LISP the ility to operte on progrms s if they were dt. In other words, LISP progrms cn modify themselves or even write other LISP progrms. This remrkle property of LISP mkes it very ttrctive for genetic progrmming. Any LISP S-expression cn e depicted s rooted point-lelled tree with ordered rnches. LISP S-expression ( (A B) C) C A B
How do we pply genetic progrmming to prolem? Before pplying genetic progrmming to prolem, we must ccomplish hfive preprtory steps: 1. Determine the set of terminls. 2. Select the set of primitive functions. 3. Define the fitness function. 4. Decide on the prmeters for controlling the run. 5. Choose the method for designting result of the run. The Pythgoren Theorem helps us to illustrte these preprtory steps nd demonstrte the potentil of genetic progrmming. The theorem sys tht t the hypotenuse, c, of right tringle with short sides nd is given y c 2 The im of genetic progrmming is to discover progrm tht mtches this function. 2 To mesure the performnce of the s-yetundiscovered computer progrm, we will use numer of different fitness cses. The fitness cses for the Pythgoren Theorem re represented y the smples of right tringles in tle. These fitness cses re chosen t rndom over rnge of vlues of vriles nd. Side Side Hypotenuse c Side Side Hypotenuse c 3 5 5.830952 12 10 15.620499 8 14 16.124515 21 6 21.840330 18 2 18.110770 7 4 8.062258 32 11 33.837849 16 24 28.844410 4 3 5.000000 2 9 9.219545 Step 1: Determine the set of terminls. The terminls correspond to the inputs of the computer progrm to e discovered. Our progrm tkes two inputs, nd. Step 2: Select the set of primitive functions. The functions cn e presented y stndrd rithmetic opertions, stndrd progrmming opertions, stndrd mthemticl functions, logicl lfunctions or domin-specific ifi functions. Our progrm will use four stndrd rithmetic opertions +,, nd, nd one mthemticl function.
Step 3: Define the fitness function. A fitness function evlutes how well prticulr computer progrm cn solve the prolem. For our prolem, the fitness of the computer progrm cn e mesured y the error etween the ctul result produced y the progrm nd the correct result given y the fitness cse. Typiclly, the error is not mesured over just one fitness cse, ut insted clculted s sum of the solute errors over numer of fitness cses. The closer this sum is to zero, the etter the computer progrm. Step 4: Decide on the prmeters for controlling the run. For controlling run, genetic progrmming uses the sme primry prmeters s those used for GAs. They include the popultion size nd the mximum numer of genertions to e run. Step 5: Choose the method for designting result of the run. It is common prctice in genetic progrmming to designte the est-so-fr generted progrm s the result of run. Once these five steps re complete, run cn e mde. The run of genetic progrmming strts with rndom genertion of n initil popultion of computer progrms. Ech progrm is composed of functions +,,, nd, nd terminls nd. Crossover in genetic progrmming: Two prentl S-expressions In the initil popultion, ll computer progrms usully hve poor fitness, ut some individuls re more fit thn others. Just s fitter chromosome is more likely to e selected for reproduction, so fitter computer progrm is more likely to survive y copying itself into the next genertion. ( ( ( ( ( ) ( ))) ) ( )) ( ( ( ( ( ) )) ) ( ( )))
Crossover in genetic progrmming: Two offspring S-expressions Muttion in genetic progrmming A muttion opertor cn rndomly chnge ny function or ny terminl in the LISP S-expression. Under muttion, ti function cn only e replced dy function nd terminl cn only e replced y terminl. ( ( ( ( ( ) ( ))) ) ( ( ( ) ))) ( (( ) ) ( ( ))) Muttion in genetic progrmming: Originl S-expressions Muttion in genetic progrmming: Mutted S-expressions ( ( ( ( ( ) ( ))) ) ( )) ( ( ( ( ( ) )) ) ( ( ))) ( ( ( ( ( ) ( ))) ) ( )) ( ( ( ( ( ) )) ) ( ( )))
In summry, genetic progrmming cretes computer progrms y executing the following steps: Step 1: Assign the mximum numer of genertions to e run nd proilities for cloning, crossover nd muttion. Note tht the sum of the proility of cloning, the proility of crossover nd the proility of muttion must e equl to one. Step 2: Generte n initil popultion p of computer progrms of size N y comining rndomly selected functions nd terminls. Step 3: Execute ech computer progrm in the popultion nd clculte its fitness with n pproprite fitness function. Designte the estso-fr individul s the result of the run. Step 4: With the ssigned proilities, select genetic opertor to perform cloning, crossover or muttion. Step 5: If the cloning opertor is chosen, select one computer progrm from the current popultion of progrms nd copy it into new popultion. If the crossover opertor is chosen, select pir of computer progrms from the current popultion, crete pir of offspring progrms nd plce them into the new popultion. If the muttion opertor is chosen, select one computer progrm from the current popultion, perform muttion nd plce the mutnt into the new popultion. Step 6: Repet Step 4 until the size of the new popultion of computer progrms ecomes equl to the size of the initil popultion, N. Step 7: Replce the current (prent) popultion with the new (offspring) popultion. Step 8: Go to Step 3 nd repet the process until the termintion criterion is stisfied.
Fitness history of the est S-expression F i t n e s s, % 100 80 60 40 20 0 0 1 2 3 4 G e n e r t i o n s B e s t o f g e n e r t i o n Wht re the min dvntges of genetic progrmming compred to genetic lgorithms? Genetic progrmming pplies the sme evolutionry pproch. However, genetic progrmming is no longer reeding it strings tht represent coded solutions ut complete computer progrms tht solve prticulr prolem. The fundmentl difficulty of GAs lies in the prolem representtion, tht is, in the fixed-length coding. A poor representtion limits the power of GA, nd even worse, my led to flse solution. A fixed-length coding is rther rtificil. As it cnnot provide dynmic vriility in length, such coding often cuses considerle redundncy nd reduces the efficiency of genetic serch. In contrst, genetic progrmming uses high-level uilding locks of vrile length. Their size nd complexity cn chnge during reeding. Genetic progrmming works well llin lrge numer of different cses nd hs mny potentil pplictions.