COMP3411: Artificial Intelligence 10a. Evolutionary Computation

Transcription

1 COMP s1 Evolutionary Computation 1 COMP3411: Artificial Intelligence 10a. Evolutionary Computation Outline Darwinian Evolution Evolutionary Computation paradigms Simulated Hockey Evolutionary Robotics COMP s1 Evolutionary Computation 2 Overview COMP s1 Evolutionary Computation 3 Charles Darwin machine learning you are here evolutionary techniques neural networks simulated environments competitive games Darwin s theory of Natural Selection was largely inspired by whathe observed on a visit to the Galapagos Islands different species of finches from different islands unusual adaptations such as the marine iguana breeding habits of turtles Darwin was influenced by: Charles Lyell s Principles of Geology Thomas Malthus s Essay on Population his grandfather Erasmus Darwin his other grandfather, Josiah Wedgwood

2 COMP s1 Evolutionary Computation 4 Human Genome COMP s1 Evolutionary Computation 5 Evolutionary Computation human genome consists of 3 billion DNA base pairs each base pair can be one of four nucleotides A(Adenine) G(Guanine) C(Cytosine) T(Thymine) approximately 30,000 genes, each coding for a specific protein 97% of genome does not code for proteins once thought to be useless junk DNA now thought to serve some other function(s) use principles of natural selection to evolve a computational mechanism which performs well at a specified task. start with randomly initialized population repeated cycles of: evaluation selection reproduction + mutation any computational paradigm can be used, with appropriately defined reproduction and mutation operators COMP s1 Evolutionary Computation 6 Evolutionary Computation Paradigms COMP s1 Evolutionary Computation 7 Bit String Operators (Genetic Algorithm) Bit Strings (Holland Genetic Algorithm ) S-expression trees (Koza Genetic Programming ) set of continuous parameters (Swefel Evolutionary Strategy ) Lindenmeyer system (e.g. Sims Evolving Virtual Creatures ) one point crossover: two point crossover: point mutation:

3 COMP s1 Evolutionary Computation 8 S-expression Trees (Genetic Programming) COMP s1 Evolutionary Computation 9 Continuous Parameters (ES) reproduction = just copying mutation = add random noise to each weight (or parameter), from a Gaussian distribution with specified standard deviation sometimes, the standard deviation evolves as well COMP s1 Evolutionary Computation 10 Fitness Functions COMP s1 Evolutionary Computation 11 Case Study Simulated Hockey deceptive landscapes local optima premature convergence variation over time (robustness) baldwin effect (evolve to learn from environment)

4 COMP s1 Evolutionary Computation 12 COMP s1 Evolutionary Computation 13 Shock Physics Shock Actuators rectangular rink with rounded corners near-frictionless playing surface spring method of collision handling left skate right skate frictionless (never acquires any spin) (x L,y L ) (x R,y R ) askateateachendofthevehiclewithwhichitcanpushontherink in two independent directions COMP s1 Evolutionary Computation 14 COMP s1 Evolutionary Computation 15 Shock Sensors Shock Inputs each of the 6 sensors responds to three different stimuli ball / own goal opponent goal 3additionalinputsspecifythecurrentvelocityofthevehicle 6Braitenberg-stylesensorsequallyspacedaroundthevehicle total of = 21 inputs each sensor has an angular range of 90 with an overlap of 30 between neighbouring sensors

5 COMP s1 Evolutionary Computation 16 Shock Agent velocity sensor 0 sensor 1 enemy goal enemy goal sensor 2 enemy goal output sensor 3 enemy goal vector sensor 4 sensor 5 enemy goal enemy goal longitudinal (left skate) longitudinal (right skate) lateral z COMP s1 Evolutionary Computation 17 Shock Agent Perceptron with 21 inputs and 4 outputs total of 4 (21 + 1)=88 weights our genome (for Evolutionary Computation) consists of a vector of these 88 parameters mutation = add Gaussian random noise to each parameter, with standard deviation 0.05 COMP s1 Evolutionary Computation 18 Shock Task COMP s1 Evolutionary Computation 19 Evolutionary Algorithm each game begins with a random game initial condition random position for random position and orientation for player each game ends with +1 if enemy goal -1 if own goal 0iftimelimitexpires mutant champ + Gaussian noise champ and mutant play up to 5 games with same game initial conditions if mutant does better than champ, champ (1 α) champ + α mutant better means the mutant must score higher than the champ in the first game, and at least as high as the champ in each subsequent game

6 COMP s1 Evolutionary Computation 20 Evolved Behavior COMP s1 Evolutionary Computation 21 Wins and Losses COMP s1 Evolutionary Computation 22 Evolutionary Robotics COMP s1 Evolutionary Computation 23 Aibo Walk Learning (Hornby) Aibo walk learning Humanoid walk learning Evolving body as well as controller Simulation to Reality Learning done on actual robot.

7 COMP s1 Evolutionary Computation 24 Guroo Humanoid Walk Learning COMP s1 Evolutionary Computation 25 Evolving Virtual Creatures (Sims) Body evolves as a Lindenmeyer system Learning done in simulator(s), then tested on actual robot. Controller evolves as a neural network COMP s1 Evolutionary Computation 26 Golem (Lipson) COMP s1 Evolutionary Computation 27 Simulation to Reality (Hornby) Evolved in simulation, tested in reality. Evolved in simulation, tested in reality.

8 Gaps in the Fossil Record? Gaps in the Fossil Record? Partial Geographic Isolation Punctuated Equilibria

9 COMP s1 Evolutionary Computation 28 Gaps in the Fossil Record? COMP s1 Evolutionary Computation 29 Co-Evolution Eldridge & Gould partial geographic isolation punctuated equilibria ideas for Evolutionary Computation? island models co-evoluton co-operative insects/flowers brain/body competitive leopard vs. gazelle actor vs. critic mixed co-operative/competitive game theory, iterated prisoner s dilemma multi-agent systems COMP s1 Evolutionary Computation 30 Current / Future Directions Hierarchical Evolutionary Re-Combination (HERCL) scaling up to larger problems transfer learning from task to task novelty search modularity and evolution credit-assignment problem artificial ecology