University of Birmingham. 3 March 2010

Transcription

1 1 University of Birminghm 3 Mrch 2010

2 2 Introduction to evolutionry computtion Evolutionry lgorithms solution representtion fitness function initil popultion genertion genetic nd selection opertors Types of evolutionry lgorithms string nd tree representtions hyrid representtions Applictions in Prticle Physics Conclusions

3 3 Nturl selection - orgnisms with fvourle trits re more likely to survive nd reproduce thn those with unfvourle trits (Drwin & Wllce) Popultion genetics - genetic drift, muttion, gene flow => explin dpttion, specition (Mendel) Moleculr evolution - identifies DNA s the genetic mteril (Avery); explins encoding of genes in DNA (Wtson & Crick) Gol of nturl evolution - to generte popultion of individuls of incresing fitness (ility to survive nd reproduce)

4 Artificil evolution - simultion of the nturl evolution on computer 4 New field - Evolutionry Computtion (sufield of Artificil Intelligence) Gol of evolutionry computtion - to generte set of solutions to prolem of incresing qulity Alterntive serch techniques e.g. Evolutionry Algorithms

5 5 Individul cndidte solution to prolem decoding encoding Chromosome representtion of the cndidte solution Gene constituent entity of the chromosome Popultion set of individuls/chromosomes Fitness function representtion of how good cndidte solution is Genetic opertors opertors pplied on chromosomes in order to crete genetic vrition (other chromosomes)

6 6 Prolem definition Solution representtion (encoding the cndidte solution) Fitness definition Run Decoding the est fitted chromosome = solution New genertion Genetic opertors cross-over comining genetic mteril from prents muttion - rndomly chnges the vlues of genes elitism/cloning copies the est individuls in the next genertion Strt Run Initil popultion cretion (rndomly) Fitness evlution (of ech chromosome) yes Terminte? Stop no Selection of individuls (proportionl with fitness) Reproduction (genetic opertors) Replcement of the current popultion with the new one

7 7 Chromosome representtion of the cndidte solution Ech chromosome represents point in the serch spce Approprite chromosome representtion very importnt for the success of EA influence the efficiency nd complexity of the serch lgorithm Representtion schemes Binry strings ech it is oolen vlue, n integer or discretized rel numer Rel-vlued vriles Trees Comintion of strings nd trees

8 The most importnt component of EA! 8 Fitness function - representtion of how good (close to the optiml solution) cndidte solution is - mps chromosome representtion into sclr vlue F : C I R I chromosome dimension Fitness function needs to model ccurtely the optimistion prolem Used: in the selection process to define the proility of the genetic opertors Includes: ll criteri to e optimised reflects the constrints of the prolem penlising the individuls tht violtes the constrints

9 9 Genertion of the initil popultion: rndom genertion of gene vlues from the llowed set of vlues (stndrd method) Advntge - ensure the initil popultion is uniform representtion of the serch spce ised genertion towrds potentilly good solutions if prior knowledge out the serch spce exists. Disdvntge possile premture convergence to locl optimum Size of the initil popultion: smll popultion represents smll prt of the serch spce time complexity per genertion is low needs more genertions lrge popultion covers lrge re of the serch spce time complexity per genertion is higher needs less genertions to converge

10 10 Purpose to produce offspring from selected individuls to replce prents with fitter offspring Typicl opertors cross-over cretes new individuls comining genetic mteril from prents muttion - rndomly chnges the vlues of genes (introduces new genetic mteril) - hs low proility in order not to distorts the genetic structure of the chromosome nd to generte loss of good genetic mteril elitism/cloning copies the est individuls in the next genertion The exct structure of the opertors dependent on the type of EA

11 Purpose - to select individuls for pplying reproduction opertors Rndom selection individuls re selected rndomly, without ny reference to fitness Proportionl selection the proility to select n individul is proportionl with the fitness vlue F( Cn ) P( Cn ) = P(C N n ) selection proility of the chromosome C n F( C ) n = 1 n F(C n ) fitness vlue of the chromosome C n Normlised distriution y dividing to the mximum fitness - ccentute smll differences in fitness vlues (roulette wheel method) Rnk-sed selection uses the rnk order of the fitness vlue to determine the selection proility (not the fitness vlue itself) e.g. non-deterministic liner smpling individul sorted in decresing order of the fitness vlue re rndomly selected Elitism k est individuls re selected for the next genertion, without ny modifiction k clled genertion gp 11

12 12 Trnsition from one point to nother in the serch spce Strting the serch process Serch surfce informtion tht guides to the optiml solution EA Proilistic rules Prllel serch Set of points No derivtive informtion (only fitness vlue) CO Deterministic rules Sequentil serch One point Derivtive informtion (first or second order)

13 13 Hundreds of versions! String sed Genetic Algorithms (GA) (J. H. Hollnd, 1975) Evolutionry Strtegies (ES) (I. Rechenerg, H-P. Schwefel, 1975) Tree sed Genetic Progrmming (GP) (J. R. Koz, 1992) Hyrid representtions Developmentl Genetic Progrmming (DGP) (W. Benzhf, 1994) Gene Expression Progrmming (GEP) (C. Ferreir, 2001) Min differences Encoding method (solution representtion) Reproduction method

14 Solution representtion Chromosome - fixed-length inry string (common technique) Gene - ech it of the string genes chromosome Reproduction Cross-over (recomintion) exchnges prts of two chromosomes Point choosen rndomly (usul rte 0.7) Muttion chnges the gene vlue (usul rte ) Point choosen rndomly

15 15 Minly for lrge-scle optimistion nd fitting prolems Experimentl PP event selection optimistion (A. Drozdetskiy et. l. Tlk t ACAT2007) trigger optimistion (L1 nd L2 CMS SUSY trigger NIM A502 (2003) 693) neurl-netwok optimistion for Higgs serch (F. Hkl et.l., tlk t STAT2002) Theoreticl/phenomenologicl PP fitting isor models to dt for p(γ,k + )Λ (NP A 740 (2004)147) discrimintion of SUSY models (JHEP 0407:069,2004; hep-ph/ ) lttice clcultions (NP B 73 (1999) 847; (2000)837)

16 Discrimintion of SUSY models (B.C. Allnch et.l, JHEP 0407:069,2004) GA used to estimte rough ccurcy required for sprticle mss mesurements nd predictions to distinguish SUSY models I k input spce of free prmeters of model k M spce of physicl mesurements (sprticle msses) Ech point in I k is (potentilly) mpped into M with set of renormlistion group equtions (RGE) => model footprint Distnce mesure Δ = r M r M A A + r M r M B B A,B points in two footprints Minimum (over points in input spce) estimte of ccurcy of mss mesurements needed to distinguish the models

17 GA used to minimise Chromosome rel numers: vlues of the free prmeters of the two models to e compred MIR mirge scenrio EUR erly unifiction = 0.5%

18 GP serch for the computer progrm to solve the prolem, not for the solution to the prolem. Computer progrm - ny computing lnguge (in principle) - LISP (List Processor) (in prctice) LISP - highly symol-oriented Mthemticl expression *-c (-(*)c) Solution representtion S-expression Grphicl representtion of S-expression - * c 18 functions (+,*) nd terminls (,,c) (vriles or constnts) Chromosome: S-expression - vrile length => more flexiility - sintx constrints => invlid expressions Reproduction Cross-over (recomintion) nd Muttion (usuly)

19 2 + ( ) sqrt Prents 2 sqrt - 19 (sqrt(+(*)(-))) + (-(sqrt(-(*)))) - * - * 2 + (sqrt(+(*))) * + sqrt Offspring 2 ( ) (-sqrt(-(*)))(-)) sqrt *

20 function replced y nother function terminl replced y nother terminl 2 + ( ) (sqrt(+(*)(-))) 2 * ( ) (sqrt(-(*)(-))) sqrt + sqrt - - Prents Offspring 2 (-(sqrt(-(*)))) 2 (-sqrt(-(*)))) * sqrt - sqrt * - *

21 Experimentl PP - event selection Higgs serch in ATLAS K. Crnmer et.l., Comp. Phys. Com 167, 165 (2005). D, D s nd Λ c decys in FOCUS (J.M. Link et. l., NIM A 551, 504 (2005); PL B624, 166 (2005)) 21 e.g. Serch for D π K π (FOCUS) Chromosome: cndidte cuts/selection rules - tree of: functions: mthemticl functions nd opertors, oolen opertors vriles: vertexing vriles, kinemticl vriles, PID vriles Fitness function (will e minimised) S + S 2 B 10000( n) n - numer of tree nodes penlty sed on the size of the tree (ig trees must mke significnt contriution to kg reduction or signl increse)

22 Best fitted chromosomes from genertion 0 Initil selection 22 Inter point in trget Decy vertex out of trget Best cndidte, fter 40 genertions = finl selection criteri Finl selection

23 Evolution grph 23 verge size of the individuls Averge fitness of the popultion Fitness of the est individul

24 24 Chromosome - sequence of symols (functions nd terminls) Hed (h) Til (t) t=h(n-1)+1 n higest rity Q*-+cd Expression tree (ET) Q * mpping ET ends efore the end of the gene! *+-Q+//++ - c Mthemticl expression + Trnsltion (s in GP) ( ) ( c + d ) d * + - Q

25 Reproduction Genetic opertors pplied on chromosomes not on ET => lwys produce sintcticlly correct structures! Cross-over exchnges prts of two chromosomes Muttion chnges the vlue of node Trnsposition moves prt of chromosome to nother loction in the sme chromosome 25 e.g. Muttion: Q replced with * *+-Q+//++ * + - Q *+-*+//++ * + - * Lilin Teodorescu

26 26 GEP for event selection L. Teodorescu, IEEE Trns. Nucl. Phys., vol. 53, no.4, p (2006) L. Teodorescu, D. Sherwood, Comp Phys. Comm. 178, p 409 (2008) lso tlks t. CHEP06, ACAT2007 (PoS(ACAT)051 nd ACAT2008 (PoS(ACAT)066) CERN Yellow Report CERN cuts/selection criteri finding for signl/ckground clssifiction fitness function - numer of events correctly clssified s signl or ckground (mximise clssifiction ccurcy) limittion imposed y the softwre ville t the time input functions - logicl functions => cut type rules - common mthemticl functions input dt - Monte-Crlo simultion from BBr experiment for Ks production in e + e - + (~10 GeV), π π K S

27 27 No. of genes = 1, Hed length =10 Model complexity 1 Fsig 5.26, Rxy < 0.19, doc <1, Clssifiction Accurcy Trining Accurcy Testing Accurcy Pchi > Hed Size Clssifiction Accurcy = 95%

28 28 GEP nlysis optimises clssifiction ccurcy Hed Selection criteri 1 Fsig Fsig 8.80, doc <1 3 Fsig > 3.67, Rxy Pchi 4 Fsig > 3.67, Rxy Pchi 5 Fsig 3.63, Rz 2.65, Rxy < Pchi 7 Fsig 3.64, Rxy < Pchi, Pchi > 0 10 Fsig 5.26, Rxy < 0.19, doc <1, Pchi > 0 20 Fsig > 4.1, Rxy 0.2, SFL > 0.2, Pchi > 0, doc > 0, Rxy Mss Cut-sed (stndrd) nlysis optimises signl significnce Fsig 4.0 Rxy 0.2cm SFL 0cm Pchi > Reduction S: 15% B: 98% doc 0.4cm Rz 2.8cm Reduction S: 16% B: 98.3%

29 events, 8 vriles, GEP - 38 functions 1 Bckground Rejection BDT ANN GEP Signl Efficiency

30 30 Fitness GEP ngep GEP-FT ngep-ft Numer of genertion ngep new methods for creting constnts GEP-FT - evolution controlled y n online threshold on fitness FT = verge fitness per genertion * scling fctor Scling fctor optimised (typicl vlues etween 0.5 to 1.5 )

31 31 3-yer project funded y EPSRC Detiled studies nd further developments of GEP - chrcterise nd improve the solution evolvility - hyrid lgorithms (GEP + sttisticl methods) - clssifiction nd clustering lgorithms LHC dt test-ed for outcomes of the project => HEP nlysis Smll tem: myself, one RA, two Ph.D. students

32 32 Prticle physics more nd more open to new lgorithms NN ES GA GP GEP SVM Prticle physics in more need of powerful lgorithms Lilin Teodorescu

33 33 Wolpert D.H., Mcredy W.G. (1997), No Free Lunch Theorem for Optimiztion, IEEE Trnsctions on Evolutionry Computtion 1, 67. In PP - used only generl purpose lgorithms so fr - need more specilised versions?

34 34 Evolutionry lgorithms in PP used ut not extensively (t present) proved to work correctly good performnce optiml solutions, not trped in locl minim need more specilised versions for reching much etter performnce disdvntge high computtionl time - prospects for chnge new, fster lgorithms, more computing power

35 35