12. Lecture Stochastic Optimization

Transcription

1 Soft Control (AT 3, RMA) 12. Lecture Stochastic Optimization Differential Evolution

2 12. Structure of the lecture 1. Soft control: the definition and limitations, basics of expert" systems 2. Knowledge representation and knowledge processing (Symbolic AI) application: expert systems 3. Fuzzy Systems: Dealing with Fuzzy knowledge application: Fuzzy Control 4. Connective systems: neural networks application: Identification and neural controller 5. Genetic Algorithms: Stochastic Optimization Genetic Algorithms Simulated Annealing Differential Evolution Application: Optimization 6. Summary and Literarture reference 315

3 Differential Evolution: Introduction Differential Evolution (DE), as well as genetic algorithms, belong to the population-based optimization methods DE has no natural model DE was founded and presented in 1996 by PricewaterhouseCoopers and Storn R. Storn, R. and K. Price, K. Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces, Journal of Global Optimization, 11, (1997) pp Procedures can be applied directly on minimum and maximum applied problems (see GA only Maximum-Problems) Scope Optimization in multi search areas with floating e.g. Controller design 316

4 Differential Evolution: Basic idea DE is used to search for a optimum in a multi-dimensional continuous search space A solution (x, optimum potential) is represented by a vector with the dimension (D) of the search description x1 x2 x x D The elements of the vector are floating point numbers: x i The search comes with several solutions (vectors, individuals) simultaneously searches (population-based) The quantity of solutions called population (p), with N individuals p D x 1 2, x,, x N x i, The kindness of a solution is a function described f ( x) : D : The goodness of a solution is a function described 317

5 Differential Evolution: Basic algorithm 1/2 Initialisingg Mutation Recombination Selection Initialising create Initial Population (such as random solutions) Mutation produce a new random solution by modifying an existing solution of the old generation Recombination Combine two solutions to a new solution Selection Solution for identifying new generation 318

6 Differential Evolution: Basic algorithm 2/2 4 Vectors of old Generation 3 Vectors (randomly chosen, x r1,x r2,x r3 ) Each vector of the old generation is exactly once this vector 1 Vektor (x) Mutation Recombination 1 Donator-Vector (v) 1 Test vector (u) Selection New Vector (x + ) New Generation 319

7 Differential Evolution: Mutation Each vector X of the old generation provides additional three vectors from the old generation(x r1,x r2,x r3 ), that holds: x x r1 x r2 x r3 Give the donor vector (v) as a linear combination of x r1,x r2,x r3 v x F ( x x ), F 0,2 r1 * r2 r3 Colorful interpretation Create a new solution based on x r1 from the difference of x r2 and x r3 Enhances heterogeneity of the solutions v F*(x r2 -x r3) x r1 x r3 x r2 x r2 -x r3 v x, and together are the parents pair for recombination 320

8 Differential Evolution: Recombination Create a test vector (u) by mixing the elements of x and v The mixture of the element of x and v is randomly controlled x,v,u sind Vectors of Dimension D x x x CR is the Cross-Over Rate: y is a random number: v v, v ri is a real random number: u i u u, u x and u are competitors in the selection 1 2 D vi,falls ri xi,sonst 1 2 D 1 2 D CR 0,1 j 1, 0,1 r i CR D oder i j j sorgt dafür, dass sich x und u in mindestens einem Element unterscheiden CR ist ein Parameter des Optimierungsverfahrens 321

9 Differential Evolution: Selection Choose one of the two vectors x, u for the new generation Selections are made solely on the basis of goodness (fitness) of an individual (Vector) Only the better of the two individuals is included in the new generation over No dependence of random variables in the selection x u, falls f(u) x, sonst Minimization f(x) x u, falls f(u) x, sonst Maximization f(x) f: to optimize Goodness function (fitness function) By the same goodness through mutation and recombination results individual in the new generation Enhances heterogeneity across generations Selection in DE has implicit elitism Only better or equally good individuals form the new generation 322

10 Differential Evolution: Application example Ackleys Function f ( x 2-dimensonale continuous function with several local minima and a global minimum for (0.0) 1, x 2 ) 20 e 20* e 0,2* 0.5*( x 2 1 x 2 2 ) e 0.5*(cos(2* * x 1 ) cos(2* * x 2 )) Optimization problem: Minimize f (x1, x2) 323

11 Differential Evolution: Application example (Initializing) Parameter for Optimization 20 Individuals CR: 50% F: 0,8 Initial population Minimum: 4,

12 Differential Evolution: Application (1 new generation) Minimum: 4,

13 Differential Evolution: Application (2nd new generation) Minimum: 4,

14 Differential Evolution: Application (3rd new generation) Minimum: 3,

15 Differential Evolution: Application (4. new generation) Minimum: 1,

16 Differential Evolution: Application (5. new generation) Minimum: 1,

17 Differential Evolution: Application (15. new generation) Minimum: 0,

18 Differential Evolution: Application (50. new generation) Minimum: 0,

19 Differential Evolution: Application (50. new generation) Minimum: 0,

20 Summary and learning from the 12th Lecture Genetic Algorithms and Genetic Programming Optimization through mutation and selection on the model of evolution in biological systems Parallel browsing for the search areas Well suited for new computer structures with multi-core processors When floats cost high for encoding the solution Simulated Annealing Optimization methods inspired by the emergence of lattice structures in crystals Only one solution is to use scanning No speed advantage through multi-core processors Feature: temporary deterioration is understood as an improvement Differential Evolution Artificial population-based optimization methods Well suited for new computer structures with multi-core processors Procedures for the optimization of floating point numbers 333

21 Literature (additional / continuing) 1/2 Chapter 1 or entire lecture: General information on methods of AI Götz, Güntzer (Hrsg.): Handbuch der künstlichen Intelligenz. Oldenbourg Verlag, "Umfassendes Nachschlagewerk für Interessierte. King R.E.: Computational Intelligence in Control Engineering. Marcel Dekker, 1999 "Sehr schöne Übersicht zu Soft-Control. Chapter 2: Expert Systems Polke, M.: Prozeßleittechnik. Oldenbourg Verlag, "Einige Ideen für die Anwendung in der Leittechnik in Kapitel 13. Ahrens, W.; Scheurlen, H.-J.; Spohr, G.-U.: Informationsorientierte Leittechnik. Oldenbourg Verlag, "Einführung in XPS für leittechnische Aufgaben (und etwas Fuzzy) in Kapitel 9. Lunze, J.: Künstliche Intelligenz für Ingenieure I und II. Oldenbourg Verlag, 1994/1995. "Sehr Ausführliche Behandlung von XPS. 334

22 Literature (additional / continuing) 2/2 Chapter 3: Fuzzy Kiendl, H.: Fuzzy Control methodenorientiert. Oldenbourg Verlag, "Ausführliche Darstellung mit kurzer Einführung in die Regelungstechnik und sehr sehr ausführlichem Beispiel. Chapter 4: Neuro Zakharian, S.; Ladewiw-Riebler, P.; Thoer, S.: Neuronale Netze für Ingenieure. Vieweg Verlag, "Kompakte und gut verständliche Darstellung mir Anwendungen in der Regelungstechnik." Chapter 5: Genetic Algorithms Goley, D.A.: An Introduction to Genetic Algorithms for Scientists and Engineers. World Scientific Publishing, "Sehr ausfürliche Darstellung." Fleming, P.J.; Purshouse, R.C.: Genetic algorithms in control systems engineering. IFAC PROFESSIONAL BRIEF. "Sehr gute Übersicht. 335

23 Acknowledgements Thank you for your interest during the semester 336