Centric Selection: a Way to Tune the Exploration/Exploitation Trade-off

Transcription

1 : a Way to Tune the Exploration/Exploitation Trade-off David Simoncini, Sébastien Verel, Philippe Collard, Manuel Clergue Laboratory I3S University of Nice-Sophia Antipolis / CNRS France Montreal, July 10 th 2008 : A way to tune the E/E trade-off

2 Introduction Exploration / exploitation tradeoff One of the fondamental problem in EA Too much exploitation : population get stuck in local optima Too much exploration : random walk on fitness landscape exploration / exploitation tradeoff : A way to tune the E/E trade-off

3 Selective Pressure in EA Introduction the ability of best solutions to conquer the whole population : A way to tune the E/E trade-off

4 Selective Pressure in EA Introduction the ability of best solutions to conquer the whole population Selective Pressure = Population Diversity = Exploration / Exploitation tradeoff : A way to tune the E/E trade-off

5 Selective Pressure in EA Introduction the ability of best solutions to conquer the whole population Selective Pressure = Population Diversity = Exploration / Exploitation tradeoff Some methods which try to tune selective pressure : Island models Sharing methods Cellular Genetic Algorithm... : A way to tune the E/E trade-off

6 Cellular Genetic Algorithms spatial structured population One solution in each cell Introduction Neighborhood : Von Neumann,... N W C S E : A way to tune the E/E trade-off

7 Cellular Genetic Algorithms spatial structured population One solution in each cell Introduction Neighborhood : Von Neumann,... N W C S E Genetic operators are local: Selection of parents within the neighborhood (tournament selection,...) After selection, crossover, mutation: Replacement of the solution in C if better : A way to tune the E/E trade-off

8 Cellular Genetic Algorithms spatial structured population One solution in each cell Introduction Neighborhood : Von Neumann,... N W C S E Genetic operators are local: Selection of parents within the neighborhood (tournament selection,...) After selection, crossover, mutation: Replacement of the solution in C if better Overlapping neighborhoods : implicit mechanism for migration control selective pressure : A way to tune the E/E trade-off

9 Goal of this work Introduction Goal is to establish a relation between: on the population the effects of recombination and mutation operators in order to explain and find an optimal exploration/exploitation trade-off : A way to tune the E/E trade-off

10 Goal of this work Introduction Goal is to establish a relation between: on the population the effects of recombination and mutation operators in order to explain and find an optimal exploration/exploitation trade-off We propose: New selection scheme able to control the selective pressure: Theoretical model which takes into account the effects of stochastic variations: : A way to tune the E/E trade-off

11 Mesure of selective pressure Introduction The Takeover Time [Goldberg 90] is the time it takes for the single best solution to conquer the whole population when the only active operator is selection Long takeover time : low selective pressure Short takeover time : high selective pressure : A way to tune the E/E trade-off

12 Grid Shape and takeover time Introduction Pop. size Avg Takeover = 2 12 Time Square grid : takeover time is short High selective pressure Narrow grid : takeover time is long Low selective pressure : A way to tune the E/E trade-off

13 Introduction Spreading of best solution: the growth curve best indiv copies * *128 16* time steps Spreading of the best solution 3 times in spreading[giacobini 05]: quadratic, linear, quadratic (it is exponential for panmitic EA) : A way to tune the E/E trade-off

14 Performance of centric selection Principe Modify the probability to participate to the tournament 5(1 β) W 5(1 β) Probability to participate to the tournament : N cell center : p c = β β 5(1 β) north, south, C E east or west cell : p s = p n = p e = p w = 1 4 (1 β) 5(1 β) S β tunes the centric selection: it is possible to slow down and control the selection pressure in a continous isotropic manner : A way to tune the E/E trade-off

15 Performance of centric selection : isotropic Fuzzy neighborhood N 5(1 β) N 0.0 N W β= C E 5(1 β) W β C 5(1 β) E 0.0 W β=1 C 0.0 E... S... 5(1 β) S S β = β β = 1 Von Neumann fuzzy isotropic parallel Neighborhood Neighborhood Hill-Climbing : A way to tune the E/E trade-off

16 Performance of centric selection and Selective Pressure Takeover time Average takeover time as a function of β for a grid Takeover time is not defined for β = 1 (no communication between cells), drops when the value of β increases. : A way to tune the E/E trade-off

17 Performance of centric selection and Selective Pressure Growth curves and the growth rates Growth curves on grid β = 0 β = 0.3 β = β = 0.7 β = 0.8 β = 0.85 β = Generations Corresponding growth rate Best solution copies Growth rate 120 β = 0 β = 0.3 β = 0.6 β = β = 0.8 β = 0.85 β = Two stages: First: linear growth rate, Second: quadratic growth rate. Interpretation: First: isotropic diffusion, roughly propagates describing an obtuse square, Second: the sides are reached, the dynamic changes Generations : A way to tune the E/E trade-off

18 Performance of centric selection Quadratic Assignment Problem (QAP) Problem of assigning a set of N facilities to a set of N locations with given distances between the locations d ij and given flows between the facilities f ij Φ(p) = N i=1 j=1 where p(i) is the location of facility i N d p(i)p(j) f ij = Find the permuation p which minimize the total flow Φ : A way to tune the E/E trade-off

19 NK fitness landscapes Performance of centric selection f (x) = 1 N N f i (x i,x i1,...,x ik ) i=1 N : length of the bit string K N 1 number of interactions x i {0, 1} {i 1,..., i K } {1,..., i 1, i + 1,...,N} f i : {0, 1} K+1 [0, 1] choosen at random : A way to tune the E/E trade-off

20 Results on QAP Performance of centric selection Avg. results and std.dev. on QAP instances Instance Std cga Best avg. results Optimal β Nug [28] 6144 [14] 0.88 Tai40a [14343] [12000] 0.84 Sko [75] [34] 0.82 Tai50a [20721] [13372] 0.82 Tai60a [27760] [19391] 0.86 The optimal value of β is arround 0.86 : A way to tune the E/E trade-off

21 Results on NK landscapes Performance of centric selection Avg. performances and std.dev. on NK instances with N = 32 K Std cga Best avg. results Optimal β [0] [0] [0,1] [0.003] [0] [0.01] [0.003] [0.01] [0.004] [0.01] [0.003] [0.01] [0.009] 1 The optimal value of β is 1.0 : A way to tune the E/E trade-off

22 Performance of centric selection Optimal explotation/exploiration tradeoff The optimal exploration/exploration tradeoff is different according to the class of problem: Avg. Performances according to β: Cost Performance β β nug30 NK with N = 32 and K = 10 : A way to tune the E/E trade-off

23 Performance of centric selection Optimal explotation/exploiration tradeoff The optimal exploration/exploration tradeoff is different according to the class of problem: Avg. Performances according to β: Cost Performance β β nug30 NK with N = 32 and K = 10 Questions: How to explain theoreticaly the difference? How to find an optimal trade-off? : A way to tune the E/E trade-off

24 Optimal theoretical tradeoff From Equilibrium model to Typical run on a minimization problem: Best fitness Punctuated equilibria dynamic: Long period without improvement Rapid change: a new best solution is found Generations : A way to tune the E/E trade-off

25 Optimal theoretical tradeoff From Equilibrium model to Equilibrium Model: : Goal To study To study selective pressure selective pressure and the effect of variation operators Simul. Only the selection + Probability to find a new Ope. operator is active best solution by crossover and mutation Init. Only one best Only one best pop. solution in the population solution in the population Observ. Takeover time, Probability and time growth curve to find a new best solution : A way to tune the E/E trade-off

26 Optimal theoretical tradeoff Initialization: cea initialized with random solutions, the best solution is unique. Selection operator: centric selection Simulation of crossover and mutation operator: probabilities to find a new best solution according to the mating type Three different types of matings: between two copies of the best solution (mating 11), between one copy of the best solution and one sub-optimal solution (mating 01) between two sub-optimal solutions (mating 00). Probabilities P 11, P 01 and P 00 that matings of type 11, 01 and 00 produce a new best solution : A way to tune the E/E trade-off

27 Optimal theoretical tradeoff With this model, Probability of finding a new best solution at a given gen. t p(t) = 1 (1 P 00 ) n 00(t) (1 P 01 ) n 01(t) (1 P 11 ) n 11(t) where n 00 (t), n 01 (t) and n 11 (t) are the number of matings of each type for the generation t. Average time to find a new best solution E = t 1 tp(t) : A way to tune the E/E trade-off

28 Optimal theoretical tradeoff With this model, Probability of improving the best solution in T generations P = 1 (1 P 00 ) Σ 00(T) (1 P 01 ) Σ 01(T) (1 P 11 ) Σ 11(T) with Σ ij (T) = T t=1 n ij(t) Intuitively, ideal selection process maximizes the Σ ij which have the higher P ij : A way to tune the E/E trade-off

29 Optimal theoretical tradeoff : ideal trade-off P = 1 (1 P 00 ) Σ 00 (1 P 01 ) Σ 01 (1 P 11 ) Σ 11 Optimal value β of control parameter β dp dβ (β ) = 0 PE Model explains the tradeoff between: Exploitation: selection pressure given by Σ ij (control by β) Exploration: effect of variation operator given by P ij (problem dependent) If it is possible to have a model of Σ ij (β), it would be possible to calculate the optimal β as a function of P ij. : A way to tune the E/E trade-off

30 Optimal theoretical tradeoff Estimated P ij on QAP and NK landscapes P ij for the QAP problem nug e-05 1e Generations NK with N = 32 and K = 10 Probabilities Probabilities e-05 1e Generations Method: Results: Estimation of P ij with a Bayesian process during the runs. Average the values obtained by generations over 500 runs. For both: P 01 > P 00 and P 11 curve intercepts the others The intercept point is not the same according to the class of the problems : A way to tune the E/E trade-off

31 β β Selection in Cellular Evolutionary Algorithms Theoretical optimal value of β Optimal theoretical tradeoff QAP problem nug Generations NK with N = 32 and K = QAP: Transition between the generation 700 and 850 Before optimal value is β = After optimal value is β = 1.0 NK: Optimal value increases very fast After a short transition: optimal value is Generations : A way to tune the E/E trade-off

32 β β Selection in Cellular Evolutionary Algorithms Theoretical optimal value of β Optimal theoretical tradeoff QAP problem nug Generations NK with N = 32 and K = 10 According to the model, when β is constant, the optimal value β should be: QAP: intermediate and higher than 0.7 NK: very high around Which correspond to the experimental observation Generations : A way to tune the E/E trade-off

33 Conclusion and Future Works Optimal theoretical tradeoff We have proposed: New model of selection in cellular GA: centric selection Control the selective pressure with a continous parameter New theoretical model to explain exploitation/exploration trade-off: Future works : Apply the PE model to other types of EA Increases the accuracy of PE model to take into account other types of matings Auto-adaptation: predict the optimal value of β according to an online estimation of P ij : A way to tune the E/E trade-off