Evolutionary Computation

Transcription

1 Evolutionary Computation Lecture Algorithm Configura4on and Theore4cal Analysis

2 Outline Algorithm Configuration Theoretical Analysis 2

3 Algorithm Configuration Question: If an EA toolbox is available (which is true), which operators/parameters shall I use? Aim: To find an appropriate EA instance (configuration of EA) for an optimization task. Representa4on Representa4on 2 Representa4on 3 EA instance Toolbox Operator Operator 2 Operator 3 Operator & Parameter Scheme & Parameter Representa4on Scheme Scheme 2 Scheme 3 3

4 Algorithm Configuration Configuring an EA involves setting the values of categorical variables (type of operators/schemes) numerical variables (e.g., crossover rate, population-size) We refer both the categorical and numerical variables as parameter of EAs More formally, Configuring an EA is another search problem, for which the best parameter vectors are searched to maximize the performance. 4

5 Algorithm Configuration Performance measures may differ in different context: The best solution quality The time required to reach a threshold of solution quality Due to the stochastic nature of EAs, a performance measure can be viewed as a random variable. Multiple runs are usually required to get a reliable estimate of performance. 5

6 Algorithm Configuration The rough idea: Generate-and-Test Example -: A naïve approach Generate p candidate parameter vectors Get p EA instances, each configured with a parameter vector Run each EA instance for r times In each run, n solution vectors is generated and tested Compare the parameter vectors using performance measures to get the best configuration. 6

7 Algorithm Configuration The naïve approach requires calculating the objective function f(x) for prn times. sounds too costly. Smarter approach is needed to reduce search effort. Reduce p Reduce r Reduce n 7

8 Algorithm Configuration Racing procedures: An approach for reducing r. Basic idea: Stop running (testing) an EA instance if there is indication that it is not promising. By not promising, we mean an EA instance performs significantly worse than the best EA instance identified so far. EA instances are compared after each run rather than after running all of them for r times. 8

9 Algorithm Configuration Example -2:. Generate p candidate parameter vectors 2. Set E as the set of p EA instances 3. REPEAT: Run each EA instance once and update its (average) performance Identify the EA instance with the best performance Remove from E the EA instances whose performance is significantly worse than the best EA instance 4. Until size(e)= or the number of runs reaches r Many parameter vectors may be removed without running r times. 9

10 Algorithm Configuration There are a few statistical tests that can be used to indicate significant difference in performance. Analysis of Variance (ANOVA). Kruskal-Wallis (Rank Based Analysis of Variance). Hoeffding s bound. Unpaired Student T-Test. 0

11 Algorithm Configuration Racing procedure may not be sufficient in case: p is very large The set of candidate parameter vectors is infinite Basic idea: To apply an iterative search procedure to the parameter space. Meta-EA: using EAs to search for the best configuration Iterative local search Hopefully, the best configuration can be found with less trial parameter vectors.

12 Algorithm Configuration Example -3:. Generate p candidate parameter vectors (p <p) 2. Get p EA instances 3. REPEAT Run each EA instance for r times and assess its performance Generate another p parameter vectors 4. Until halting criterion is satisfied 2

13 Algorithm Configuration Iterative search can be combined with a racing procedure to reduce both p and r. Example -4:. Generate p candidate parameter vectors (p <p) 2. Get p EA instances 3. REPEAT Applying Racing procedure to compare performance of EA instances Generate another p parameter vectors 4. Until halting criterion is satisfied 3

14 Algorithm Configuration Further enhancement: Modeling the landscape of performance measures For identifying promising parameter vectors For filtering out unpromising parameter vectors The modeling task is a supervised learning problem Input data: parameter vectors Target: performance 4

15 Algorithm Configuration Example -5 (Sequential Parameter Optimization):. Generate p candidate parameter vectors (p <p) 2. Get p EA instances 3. Run each EA instance for r times and assess its performance 4. Build a model M to approximate the performance landscape 5. REPEAT Generate another p parameter vectors Identify the most promising parameter vectors with M Only the promising parameter vectors are run for r times Update M with the newly assessed parameter vectors 6. Until halting criterion is satisfied 5

16 Algorithm Configuration Remark on reducing n The key question involved: Will the relative order of two EAs maintain unchanged during the evolution? SaNSDE SaDE NSDE 0 0 Yes: Blue curve vs. Green Curve No: Blue Curve vs. Red Curve f 9 The lesson learned: the answer is problem and algorithm dependent, and thus few systematic approach have been investigated for this issue. 6

17 Algorithm Configuration So far, we mainly touched the case of configuring EA on a single optimization task (problem instance) What if we want to find a more general configuration that can be applied to multiple problem instances? Previously introduced ideas can be adapted to this scenario. Instead of looking at the results of r runs, comparisons are made mainly based on the performance on multiple problem instances. F-RACE ParamILS 7

18 Algorithm Configuration Additional Remarks Operators/parameters can also be adjusted on-the-fly. This type of EAs are usually called adaptive/self-adaptive EAs. With a unified representation of EA s behaviors, there will huge room for involving machine learning techniques. Algorithm configuration is also closely related to the emerging terminology hyper-heuristic. 8

19 Outline Algorithm Configuration Theoretical Analysis 本部分内容取自南京大学俞扬博士 IJCAI 203 报告的 PPT, 特此表示感谢 9

20 Conventional algorithm analysis Problem Algorithm Measurement Sorting Quick Sort average time complexity O(n log n) Shortest Path Dijkstra s algorithm average time complexity O( V 2 ) Linear Programming Simplex worst case time complexity: exponential smoothed complexity: polynomial 20

21 Time complexity What about an algorithm sorts (5,4,2,8,9) in 3 steps? measured in a class of problem instances e.g. all possible arrays of 5 numbers average complexity worst case complexity measure the growing rate as the problem size increases e.g. 2n 2 asymptotical notation O(n 2 ) 2

22 But for EAs... nature phenomena Problem Algorithm Measurement problem unknown not designed with knowledge of problems theoretical understanding is even more important 22

23 Markov chain modeling A general procedure: initialization evaluation & selection population offspring expand along time: reproduction population 0 population population 2 population 3... Markov chain: state 0 state state 2 state

24 Markov chain: state 0 state state 2 state solution space S P ( 3 2,, 0 )=P ( 3 2 ) optimal solutions S Markov property population space X = S m involves at least one optimal solution optimal populations X 24

25 Convergence Does an EA converge to the global optimal solutions? Considered as closed: lim P ( t 2 X )= t!+ gain of optimality in one step loss of optimality in one step Theorem: (discrete version derived from [He & Yu, 0]) Let be a Markov chain. Define t = X x/2x P ( t+ 2 X t = x)p ( t = x) X x2x P ( t+ /2 X t = x)p ( t = x). Then converges to X if and only if satisfies: P ( 0 2 X )+ +X t=0 t = 25

26 Convergence Does an EA converge to the global optimal solutions? Considered as closed: lim P ( t 2 X )= t!+ An EA that. uses global operators 2. preserves the best solution always converges to the optimal solutions gain of optimality > 0 loss of optimality = 0 But life is limited! How fast does it converge? 26

27 Examples in simple cases Problem OneMax Algorithm Measurement Linear Pseudo-Boolean Functions LeadingOnes LongPath (+)-EA Expected Running Time (ERT) 27

28 Running time analysis Running time of an EA: the number of solutions evaluated until reaching an optimal solution of the given problem for the first time the most time consuming step may meet many times Running time analysis: running time with respect to the problem size (e.g. n) the expected running time/ert e.g. O(n 2 ) expected running time ERT with high probability e.g. computational complexity O(n ln n) expected running time with probability at least 2 n 28

29 A simple EA: (+)-EA An extremely simplified EA missing some features of real EAs (+)-EA no population : s a randomly drawn solution from X 2: for t=,2,... do 3: s 0 mutate(s) 4: if f(s 0 ) f(s) then 5: s s 0 6: end if 7: terminate if meets a stopping criterion 8: end for one-bit mutation randomly choose one bit and change its value bitwise mutation change every bit with some probability (e.g. ) n no crossover for maximization, allow neutral changes find an optimal solution 29

30 OneMax Problem: fitness: arg max x2{0,} n f(x) = nx i= nx i= x i x i count the number of bits EAs do not have the knowledge of the problems only able to call f (x) no difference with any other functions f : {0, } n! R not only optimizing the problem, but also guessing the problem 30

31 probability Theoretical Analysis OneMax: f(x) = nx i= x i (+)-EA with bitwise mutation (flip each bit with probability ): the probability of flipping i particular bits: ( n )i ( monotonically decreasing n )n i n but always positive i 3

32 OneMax: f(x) = nx i= x i the the same number of -bits share the same f value 0 -bits S 0 -bits 2 -bits... n -bits S S 2 S m = S many transitions (+)-EA with bitwise mutation (flip each bit with probability ): n 32

33 OneMax: f(x) = nx i= x i the the same number of -bits share the same f value 0 -bits S 0 -bits 2 -bits... n -bits S S 2 S m = S an upper bound: a path visits all subspaces (+)-EA with bitwise mutation (flip each bit with probability ): n 33

34 OneMax: f(x) = nx i= x i the the same number of -bits share the same f value 0 -bits S 0 -bits 2 -bits... n -bits S S 2 S m = S p = n( n )(n n )n (+)-EA with bitwise mutation (flip each bit with probability ): n 34

35 OneMax: f(x) = nx i= x i the the same number of -bits share the same f value 0 -bits S 0 -bits 2 -bits... n -bits S S 2 S m = S p = n( n )(n p n )n n ( n )(n n )n (+)-EA with bitwise mutation (flip each bit with probability ): n 35

36 OneMax: f(x) = nx i= x i the the same number of -bits share the same f value 0 -bits S 0 -bits 2 -bits... n -bits S S 2 S m = S p = n( n )(n n )n n p ( n )(n n )n p n i ( n )(n n )n (+)-EA with bitwise mutation (flip each bit with probability ): n 36

37 OneMax: f(x) = nx i= x i (+)-EA with bitwise mutation (flip each bit with probability ): n n i probability of transition p ( n )(n n )n expected #steps the transition happens apple n i n ( + n )n n i n e summed up nx i=0 en i = enh n en ln n ERT upper bound O(n ln n) 37

38 General analysis tools running time analysis is commonly problem specific going to derive the ERT of an EA in a problem where to look what to calculate need a guide to tell what to look and what to follow to accomplish the analysis - Fitness Level Method - Drift Analysis - 38

39 Theoretical Analysis intro. to theory Markov chain problem dependency RTA analysis tools Fitness Level Method Noteonthat elitist never the best solu comparison with classics (i.e. on real-world situationslose summary on parameters sets, intuitively, form stairs, for which an u [Wegener, 02] a population is treated as the takenbest for getting offin every and a lower solution the stair, population Note that elitist (i.e. nb partition the(i.e. solution space in Lemma 2. S S m = Noteselect that elitist never loseformally the bestdescribed solution) EAs select solutions wit sets, intuitively, form est solution)note EAs solutions with better fitness. The level that elitist (i.e. never lose the best solution) EAs select be into subspaces solution... 2an Lemma (Fitness Level Method [39]) sets, intuitively, form stairs, for which upper bound can be derived taken for off ch an upper bound can be derived by summing upupper the time space sets, intuitively, form stairs, for bound can begetting derived byev su S2which an Sof For EA process on a problem f, get letol getting taken for gettingoffoff every stair, and aaelitist lower bound is the minimum formally described in lower boundtaken is thefor minimum time off stair. This getting every stair, and a an lower bound isisthe minimum timetime of with increasing fitness formallydescribed describedinin Lemma formally Lemma t = x) for all x 2 Si, and ui Then calculate: P ( t+ 2 [m j=i Lemma 2 (Fitness Leve process is at most Lemma 2 (Fitness Level Method [39]) Lemma 2 (Fitness Level Method [39]) For an elitist EA proces ). initialization probability of being in each subspace 0 (Si ) m X X -partition, m For an elitist EA process on a problem f, let S,..., S be < -partition, For an elitist EA process on a problem f, let S,..., S be a < let v le m m fa for m f, let S2.,..., Sm be <f -partition, let vi P ( t+ 2 [j=i+ bounds ofaprogress probability 0S (Sj i ) t = x), f all x 2 Si,ia for i : all x 2 S, and u x = 2x)Sfor P ( 2 [m m S = x) forvjall x 2 S. Then, j Sjt t j=i t t = x) for all x 2 isi, andiui Pt+ ( t+ i m 2j=i+ [j=i+ = x) for all x i2 Si. Th 2 [m j=i+ Sj t = x) for all x 2 Si. Then, the DCFHT of the EA process is at most process is atatmost process most and isand at least the ERT is thenisupper bounded by: lower bounded by: m X m 0 (Si ) m X X X X X 0 (Si ). i m 0 (S,, i )i ) 0 (S ui v j=i j vj i m i m j=i i m and is at least 39 and istoat least Optimization An Introduction Evolutionary

40 Example in OneMax partition 0 -bit -bit 2 -bits n -bits S 3... = S S S 2 S m initialization distribution: 0 (S i )= n i 2 n progress probability for : x 2 S i a lower bound: flipping one 0-bit but no -bits: n i ( n )(n n )n ERT: X appleiapplem 0 (S i ) mx j=i m X apple 0 (S 0 ) v j j= v j 2 O(n ln n) 40

41 Drift Analysis [Hajek, 82][Sasaki & Hajek, 88][He & Yao, 0][He & Yao, 04] x Then calculate: V(x) optimal solutions S. initialization probability of solutions 0 (x) 2. bounds of progress distance for every step: distance function V measuring distance of a solution to optimal solutions. V(x * )=0 P c l apple E[V ( t ) V ( t+ ) t ] c u E[V ( t ) V ( t+ ) t ] the ERT is then upper bounded by: P x2x 0(x)V (x)/c l and lower bounded by: P x2x 0(x)V (x)/c u most simplified version 4

42 Example in LeadingOnes LeadingOnes Problem: fitness: The drift: f(x) = nx iy i= j= arg max x2{0,} n x i nx i= j= Distance function: V (x) =n E[V ( t ) V ( t+ ) t ]= iy x i f(x) I(V ( t ) >V( t+ ))E[V ( t ) V ( t+ ) t ]+ I(V ( t ) <V( t+ ))E[V ( t ) V ( t+ ) t ]+ I(V ( t )=V ( t+ ))E[V ( t ) V ( t+ ) t ] Only need to care the expected progress: keep flip count the number of leading -bits f(0) = 2 distance of optimal solutions is zero zero zero probability of making progress >= probability of increasing at least one leading -bit E[V ( t ) V ( t+ ) t ] n ( n )i n ( n )n en 42

43 Example in LeadingOnes LeadingOnes Problem: fitness: f(x) = nx iy i= j= arg max x2{0,} n x i nx Distance function: V (x) =n iy i= j= x i f(x) count the number of leading -bits f(0) = 2 distance of optimal solutions is zero E[V ( t ) V ( t+ ) t ] n ( n )i n ( n )n en ERT is then upper bounded as X x2x 0 (x)v (x) en apple V ((00...0)) en = n en 2 O(n 2 ) 43

44 Additional Remarks There are more analytical tools than what we ve talked today. Different analytical tools may suit different algorithms/ problems (at least from an intuitive perspective). Theoretical analysis of EAs have been extended to more complicated problems (e.g., NP-hard problems such as Traveling Salesman, Minimum Vertex Cover, etc). In addition to deepening our understanding, theoretical analysis may also help configuring EAs for a specific problem. 44

45 End of Lecture 45