Evolutionary Robotics

Transcription

1 Evolutionary Robotics

2 Previously on evolutionary robotics

3 Evolving Neural Networks How do we evolve a neural network?

4 Evolving Neural Networks How do we evolve a neural network? One option: evolve the weights as a vector of numbers

5 Evolving Neural Networks How do we evolve a neural network? One option: evolve the weights as a vector of numbers Crossover is easy

6 Evolving Neural Networks How do we evolve a neural network? One option: evolve the weights as a vector of numbers Crossover is easy But is it sensible?

7 Evolving Neural Networks Competing conventions problem How do you cross these?

8 Evolving Neural Networks How do we evolve a neural network? One option: evolve the weights as a vector of numbers Do we have right topology for the task? To few nodes: cannot solve the task To many nodes: search space becomes unnecessarily large overfitting

9 Evolving Neural Networks How do we evolve a neural network? Another option: evolve both weights and topology But where do we start? What about crossover?

10 NEAT Neuro-Evolution of Augmenting Topologies (NEAT) Stanley & Miikkulainen Complexification 2. Intelligent crossover (via historical markings ) 3. Diversity ( speciation )

11 NEAT: Complexification Simplicity usually good Occam s razor generalizes better easier to search for computationally faster In neural nets: simple = fewer nodes and/or connections

12 NEAT: Complexification So, start simple; add complexity if needed Gen 0 = no hidden nodes Tend to (slowly) mutate in nodes across generations

13 NEAT: Intelligent Crossover Provides one solution to competing conventions problem Uses historical markings

14 NEAT: Encoding Direct Encoding

15 NEAT: Mutation add connection between any unconnected nodes - observing constraints, e.g. feedfoward add node in existing connection minimize initial effect - weight to new node is 1 - weight from new nodes is old weight

16 NEAT: Historical Innovation Numbers added for each new connection global counter for population

17 NEAT: Intelligent Crossover gene possibilities matching - inherited randomly disjoint (non-matching, middle) - inherited from more-fit parent excess (non-matching, end) - inherited from more-fit parent

18 NEAT: Diversity Newly added structure usually does not survive smaller structures are optimized faster new structure usually initially deleterious: it needs time to be tuned before having to compete Solution: protect it More generally, having diverse neural structures is a good idea

19 NEAT: Diversity NEAT uses speciation, but how do we determine which individuals belong to different species. We need a distance metric!

20 Genetic distance problem Genetic distance problem with same topology, distance is easy distance? but with different topologies? distance?

21 and now, we continue.

22 NEAT: Genetic Distance uses historical markings more disjoint & excess genes = less shared evolutionary history also include weight differences Genetic Distance Disjoint Genes Avg. Weight Difference in Matched Genes Normalizer: numgenes in larger genome C1, C2, & C3 weight the relative importance of each component

23 NEAT: Speciation Uses distance threshold If new org is <threshold away from any current species, add it else: create new species

24 NEAT: Speciation Each species is represented by a random genome inside the species in a previous generation

25 NEAT: Speciation Each species is represented by a random genome inside the species in a previous generation

26 NEAT: Speciation An individual is added to the first species within threshold 1 2 3

27 NEAT: Fitness Sharing & Selection adjusted fitness = fitness/numorgsinspecies species get numoffspring allocated by sum(adjustedfitnessofmembers) within a species, rank selection determines parents parents then mutated and crossed - crossover within a species only

28 NEAT: Summary 1. Complexification 2. Intelligent crossover (via historical markings ) 3. Diversity ( speciation ) Each component shown to help (Stanley & Miikkulainen 2005)

29 NEAT Would this make a good/fun homework exercise? Would probably not involve speciation. Historical markings? Crossover?

30 Encodings (for real this time)

31 Evolutionary Algorithms (EAs) Encode Problem genome Generate Population mutation and/or recombination W L Select Parents Score Population

32 Encodings how information is stored in a genome + process that produces phenotype gattaca ccatgat tggacct

33 Direct vs. Generative Encodings Direct Encoding: each genotypic element specifies an independent phenotypic element Genotype Phenotype leg 1: 2 leg 2: 2 leg 3: 2 leg 4: 2

34 Direct vs. Generative Encodings Direct Encoding: each genotypic element specifies an independent phenotypic element Genotype leg 1: 2 leg 2: 2 leg 3: 2 leg 4: 2 Phenotype Genotype' Phenotype' leg 1: 2 leg 2: 2 leg 3: 1 leg 4:.5

35 Direct vs. Generative Encodings Direct Encoding: each genotypic element specifies an independent phenotypic element Genotype Phenotype Genotype' Phenotype' leg 1: 2 leg 2: 2 leg 3: 2 leg 4: 2 leg 1: 2 leg 2: 2 leg 3: 1 leg 4:.5 X

36 Direct vs. Generative Encodings Direct Encoding: each genotypic element specifies an independent phenotypic element Genotype Phenotype Genotype' Phenotype' leg 1: 2 leg 2: 2 leg 3: 2 leg 4: 2 leg 1: 2 leg 2: 2 leg 3: 1 leg 4:.5 X Generative Encoding: genotypic elements can influence many phenotypic elements Genotype Phenotype 4x leg: 2

37 Direct vs. Generative Encodings Direct Encoding: each genotypic element specifies an independent phenotypic element Genotype Phenotype Genotype' Phenotype' leg 1: 2 leg 2: 2 leg 3: 2 leg 4: 2 leg 1: 2 leg 2: 2 leg 3: 1 leg 4:.5 X Generative Encoding: genotypic elements can influence many phenotypic elements Genotype Phenotype Genotype' Phenotype' 4x leg: 2 4x leg: 1

38 Nature s Encoding

39 Generative Encodings Desirable properties Coordinated mutational effects Scalability Low dimensional search, highly complex phenotype Structural Organization Regularity...with and without variation Modularity Hierarchy

40 Regularity reuse of information irregularity irregular compressibility less compressible less compressible Lipson (2007)

41 Regularity multiple regularities

42 Examples of Regularity in Generative Encodings Direct Encoding Generative Encoding Hornby (2005)

43 Examples of Regularity in Generative Encodings Direct Encoding (probably) Generative Encoding (probably)

44 Previous Work Generative outperforms direct on regular problems No tests across a continuum of problem regularity

45 Game Plan Case-study: generative encoding vs. direct encoding as problem regularity varies HyperNEAT Has a good direct encoding control Based on an important concept from developmental biology 2011

46 How nature builds complexity Generative encoding......where cell fate is a function of geometric position

47 How nature builds complexity Development involves producing complex coordinate frames Sean Carroll: Endless Forms Most Beautiful (2005)

48 How nature builds complexity

49 Compositional Pattern Producing Networks (CPPNs) Stanley 2007 encodes phenotypic elements as a function of their geometric location

50 Compositional Pattern Producing Networks (CPPNs) Stanley 2007 x y genome f(x,y) = fate y x value at x,y... for all x,y coordinates Adapted from: Stanley (2007)

51 sine(y) Compositional Pattern Producing Networks (CPPNs) Stanley 2007 x y gaussian(x) y x asymmetry f(x) left-right f(y) anterior-posterior symmetry gaussian(x) proximal-distal repetition within segment symmetric anterior-posterior value at x,y... for all x,y coordinates Adapted from: Stanley (2007)

52 Compositional Pattern Producing Networks (CPPNs) Stanley 2007 picbreeder.org

53 Previous Generative Encodings Sims 1994 Hornby & Pollack 2002 Dawkins 1986