Bounding the Population Size of IPOP-CMA-ES on the Noiseless BBOB Testbed

Transcription

1 Bounding the Population Size of OP-CMA-ES on the Noiseless BBOB Testbed Tianjun Liao IRIDIA, CoDE, Université Libre de Bruxelles (ULB), Brussels, Belgium Thomas Stützle IRIDIA, CoDE, Université Libre de Bruxelles (ULB), Brussels, Belgium ABSTRACT A variant of CMA-ES that uses occasional restarts coupled with an increasing population size, which is called OP- CMA-ES, has shown to be a top performing algorithm on the BBOB benchmark set. In this paper, we test a mechanism that bounds the maximum size that the population may reach in OP-CMA-ES, and we experimentally explore the impact of a maximum population size on the BBOB benchmark set. In the proposed bounding mechanism, we use a maximum population size of 0 D, where D is problem dimension. Once the maximum population size is reached or surpassed, the population size is reset to its default starting value λ, which is defined by the λ = + ln(d). Our experimental results show that our scheme for the populationsize update can lead to improved performances on separable and weakly structured multi-modal functions. Categories and Subject Descriptors G.. [Numerical Analysis]: Optimization global optimization, unconstrained optimization; F.. [Analysis of Algorithms and Problem Complexity]: Numerical Algorithms and Problems General Terms Algorithms Keywords Benchmarking, Black-box optimization. INTRODUCTION OP-CMA-ES [] is a variant of CMA-ES [, 0] that uses occasional restarts, which are triggered when the search process is deemed to stagnate, combined with an increasing population size. OP-CMA-ES and several of its variants [,,, ] have shown very good results on the BBOB Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. GECCO Companion, July 0, 0, Amsterdam, The Netherlands. Copyright 0 ACM //0...$.00. benchmark. In this paper, we base our analysis on OP- CMA-ES using its default parameter settings. In particular, the initial population size in OP-CMA-ES is set to λ = + ln(d), where D is dimension of the problem being tackled. At each restart, OP-CMA-ES increases the population size by a factor of two. This setting leads to an exponential increase of the population size in OP- CMA-ES in the number of restarts. In particular, on difficult, multi-modal functions many restarts may occur and, thus, very large population sizes may result if OP-CMA- ES doesn t find a solution better than the optimal threshold or a possible target value. In this paper, we use a mechanism to bound the maximum population size that OP-CMA-ES may use; in fact, bounding the maximum population size is motivated by the fact that sometimes very large populations may, at least theoretically, decrease the performances []. However, occasionally CMA-ES may benefit from large populations [9], which is also the motivation for increasing the population size in OP-CMA-ES. Thus, in our bound on the population size, we do not want to be too restrictive. Thus, we set the upper bound of the population size to 0 D, which leaves for higher dimensional problems the possibility to reach rather large populations (e.g. 000 for D = 0). Once this upper bound is reached, we reset the population size to its initial value, given by λ = + ln(d). Additionally, it gives some additional robustness with respect to the maximum bound on the population size we use. We label the resulting OP- CMA-ES variant -0DDr. The original OP-CMA-ES is labeled.. EXPERIMENTAL PROCEDURE We used the C version of OP-CMA-ES (last modification date 0//0) from Hansen s webpage lri.fr/~hansen/cmaesintro.html. To ensure that the final best solution is inside the bounds, the bound constraints are enforced by clamping each generated solution that violates the bound constraint to the nearest solution on the bounds. The default parameter settings of OP-CMA-ES were used. A maximum of 0 D function evaluations was used for the experiments.. RESULTS The results from the experiments that follow the experimental protocol [] on the benchmark functions given in [, 8] are presented in Figures, and and in Tables and. The expected running time (ERT), used in the figures and tables, depends on a given target function value,

2 f t = f opt + f, and it is computed across all relevant trials as the number of function evaluations executed during each trial while the best function value did not reach f t, summed over all trials and divided by the number of trials that actually reached f t [, ]. Statistical significance is tested with the rank-sum test for a given target f t (0 8 as in Figure ) using, for each trial, either the number of needed function evaluations to reach f t (inverted and multiplied by ), or, if the target was not reached, the best f-value achieved, measured only up to the smallest number of overall function evaluations for any unsuccessful trial under consideration. In the experiments, we found that -0DDr reaches solutions below the optimal threshold of 0 8 in various cases where the default version of OP-CMA-ES, here labeled, could not find such solutions. This was the case for functions f (D =, ), f (D = 0), f (D = 0) and f (D =, ). Compared to, -0DDr uses fewer function evaluations to reach optimal threshold in functions f (D =,, 0), f (D = ), f (D =,,, 0, 0), f (D =,, ), f (D = 0, 0); only in functions f 9 (D = 0) and f 0 (D =, 0, 0), -0DDr uses slightly more function evaluations to reach optimal threshold than. We next examine the impact of the specific choice on the maximum population size. To do so, we explore another bound mechanism, where the maximum population size is set to a constant value of 00; a population size larger than 00 is then kept to 00. We label the resulting algorithm Figure shows that -0DDr clearly performs better than -00. To situate the performance of -0DDr better with respect to other variants of OP-CMA-ES, in Figure we show the comparisons between -0DDr and the performance data for the OP-CMA-ES variants, CMA mah [] and OPsaACM [] in the aforementioned functions f, f,f,f 9, f 0, f,f,f,f. We find that -0DDr reaches the optimal threshold in functions f (D = ), f 9(D = 0), f (D = 0), f (D = 0) and f (D = ) where both, CMA mah and OPsaACM, cannot reach optimal threshold. In functions f (D =,, 0), f (D =, ), f (D = ), f (D =, ), f (D =,, 0) -0DDr uses fewer function evaluations to reach optimal threshold than CMA mah and OPsaACM.. CPU TIMING EXPERIMENT The -0DDr was run on f 8 until at least 0 seconds have passed. These experiment were conducted with Intel Xeon E0 (. GHz) on Linux (kernel ). The results were.e 0,.E 0,.E 0, 9.E 0,.E 0 and.0e 0 seconds per function evaluation in dimensions,,, 0, 0, and 0, respectively.. CONCLUSIONS In this paper, we have studied the impact of bounding the population size in OP-CMA-ES together with re-initialization of the population size. Obviously, using a maximum population size of 0 D does not worsen results on functions that are easy for OP-CMA-ES, that is, on functions where OP-CMA-ES within the first trial or very few restarts finds the optimum in such cases the bounds do not take effect. However, for various difficult, multi-modal functions we observed improved performance of our new OP-CMA-ES variants over the default OP-CMA-ES. Hence, these results would encourage us to explore bounds on the maximum population size also for other OP-CMA-ES variants such as Bipop-CMA-ES. Finally, one may further explore different settings for the bounds on the maximum population size, which may lead to further improvements in performance.. ACKNOWLEDGMENTS The authors would like to thank the great and hard work of the BBOB team. This work was supported by the Meta-X project funded by the Scientific Research Directorate of the French Community of Belgium. Thomas Stützle acknowledges support from the Belgian F.R.S.-FNRS, of which he is a Research Associate. Tianjun Liao acknowledges a fellowship from the China Scholarship Council.. REFERENCES [] A. Auger and N. Hansen. A restart CMA evolution strategy with increasing population size. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 00), pages 9. IEEE Press, 00. [] D. Brockhoff, A. Auger, and N. Hansen. On the impact of active covariance matrix adaptation in the CMA-ES with mirrored mutations and small initial population size on the noiseless BBOB testbed. In GECCO 0 (Companion), pages 9 9, 00. [] D. Brockhoff, A. Auger, and N. Hansen. Comparing mirrored mutations and active covariance matrix adaptation in the OP-CMA-ES on the noiseless BBOB testbed. In T. Soule, editor, GECCO 0 (Companion), pages 9 0. ACM, 0. [] D. Brockhoff, A. Auger, and N. Hansen. On the impact of a small initial population size in the OP active CMA-ES with mirrored mutations on the noiseless BBOB testbed. In T. Soule, editor, GECCO 0 (Companion), pages 9 9. ACM, 0. [] T. Chen, K. Tang, G. Chen, and X. Yao. A large population size can be unhelpful in evolutionary algorithms. Theoretical Computer Science, : 0, 0. [] S. Finck, N. Hansen, R. Ros, and A. Auger. Real-parameter black-box optimization benchmarking 009: Presentation of the noiseless functions. Technical Report 009/0, Research Center PPE, 009. Updated February 00. [] N. Hansen, A. Auger, S. Finck, and R. Ros. Real-parameter black-box optimization benchmarking 0: Experimental setup. Technical report, INRIA, 0. [8] N. Hansen, S. Finck, R. Ros, and A. Auger. Real-parameter black-box optimization benchmarking 009: Noiseless functions definitions. Technical Report RR-89, INRIA, 009. Updated February 00. [9] N. Hansen and S. Kern. Evaluating the CMA evolution strategy on multimodal test functions. In Parallel Problem Solving from Nature PPSN VIII, volume of Lecture Notes in Computer Science, pages 8 9. Springer, Heidelberg, Germany, 00. [0] N. Hansen, S. Muller, and P. Koumoutsakos. Reducing the time complexity of the derandomized evolution

3 -0DDr -00 Sphere Linear slope Ellipsoid separable Attractive sector 8 Rastrigin separable Step-ellipsoid 8 Skew Rastrigin-Bueche separ Rosenbrock original Rosenbrock rotated Ellipsoid Discus Bent cigar Sharp ridge Sum of different powers Rastrigin Weierstrass Schaffer F, condition Schaffer F, condition Griewank-Rosenbrock F8F Schwefel x*sin(x) Gallagher 0 peaks Gallagher peaks Katsuuras Lunacek bi-rastrigin DDr Figure : Expected running time (ERT in number of f-evaluations) divided by dimension for target function value 0 8 as log 0 values versus dimension. Different symbols correspond to different algorithms given in the legend of f and f. Light symbols give the maximum number of function evaluations from the longest trial divided by dimension. Horizontal lines give linear scaling, slanted dotted lines give quadratic scaling. Black stars indicate statistically better result compared to all other algorithms with p < and Bonferroni correction number of dimensions (six). Legend: :-0DDr, :, :-00

4 Rastrigin separable Griewank-Rosenbrock F8F Gallagher peaks Skew Rastrigin-Bueche separ Schwefel x*sin(x) Katsuuras Weierstrass Gallagher 0 peaks Lunacek bi-rastrigin -0DDr CMA_mah OPsaACM Figure : Comparing -0DDr to CMA mah and OPsaACM. CMA mah is the active covariance matrix adaptation version of OP-CMA-ES with mirrored mutation and a small initial population size. CMA sa is self-adaptive surrogate-assisted version of OP-CMA-ES. Expected running time (ERT) divided by dimension for target function value 0 8 as log 0 values. Different symbols correspond to different algorithms given in legend of f. Light symbols give the maximum number of function evaluations from all trials divided by the dimension. Horizontal lines give linear scaling, the slanted dotted lines give quadratic scaling. strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, (): 8, 00. [] N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9():9 9, 00. [] I. Loshchilov, M. Schoenauer, and M. Sebag. Black-box optimization benchmarking of OP-SaACM-ES and Bipop-SaACM-ES on the BBOB-0 noiseless testbed. In T. Soule, editor, GECCO 0 (Companion), pages 8. ACM, 0. [] K. Price. Differential evolution vs. the functions of the second ICEO. In Proceedings of the IEEE International Congress on Evolutionary Computation, pages, 99.

5 .0 f-,-d separable fcts 0 8 log0 of (# f-evals / dimension).0 f0-,-d ill-conditioned fcts 0 8 log0 of (# f-evals / dimension).0 f0-,-d weakly structured multi-modal fcts 0 8 log0 of (# f-evals / dimension) best 009-0DDr -00 best 009-0DDr -00 best 009-0DDr f-9,-d moderate fcts 0 8 log0 of (# f-evals / dimension).0 f-9,-d multi-modal fcts 0 8 log0 of (# f-evals / dimension).0 f-,-d all functions 0 8 log0 of (# f-evals / dimension) best 009-0DDr -00 best DDr best 009-0DDr -00 Figure : Bootstrapped empirical cumulative distribution of the number of objective function evaluations divided by dimension (FEvals/D) for 0 targets in 0 [ 8..] for all functions and subgroups in -D. The best 009 line corresponds to the best ERT observed during BBOB 009 for each single target.

6 .0 f-,0-d separable fcts 0 8 log0 of (# f-evals / dimension).0 f0-,0-d ill-conditioned fcts 0 8 log0 of (# f-evals / dimension).0 f0-,0-d weakly structured multi-modal fcts 0 8 log0 of (# f-evals / dimension) best DDr best 009-0DDr -00 best 009-0DDr f-9,0-d moderate fcts 0 8 log0 of (# f-evals / dimension).0 f-9,0-d multi-modal fcts 0 8 log0 of (# f-evals / dimension).0 f-,0-d all functions 0 8 log0 of (# f-evals / dimension) best 009-0DDr -00 best 009-0DDr -00 best 009-0DDr -00 Figure : Bootstrapped empirical cumulative distribution of the number of objective function evaluations divided by dimension (FEvals/D) for 0 targets in 0 [ 8..] for all functions and subgroups in 0-D. The best 009 line corresponds to the best ERT observed during BBOB 009 for each single target.

7 f / -0DD.0().() () () 0() () /.0().() () () 0() () / -00.0().() () () 0() () / f / -0DD() () 9() () () () / () () 9() () () () / -00 () () 9() () () () / f 0 / -0DD0.99() () 0() 0() () () / 0.99() 8() 90(00) (9) 0() (0) 8/ () 8() () 8() 9() 9() / f / -0DD.() (9).e(e).0e(e).8e(e).8e(e) /.() e 0/ -00.() e 0/ f / -0DD.() (8) 0() () () () /.() (8) 0() () () () / -00.() (8) 0() () () () / f / -0DD.(0.).8(0.).(0.).(0.).(0.).(0.) /.(0.).8(0.).(0.).(0.).(0.).(0.) / -00.(0.).8(0.).(0.).(0.).(0.).(0.) / f 9 / -0DD.().(0.).(0.9).(0.).(0.).() /.().(0.).(0.9).(0.).(0.).() / -00.().(0.).(0.9).(0.).(0.).() / f8 9 0 / -0DD.(.0).().().().8().() /.(.0).().().().8().() / -00.(.0).().().().8().() / f / -0DD.() 0(0) 8.().().().() /.() 0(0) 8.().().().() / -00.() 0(0) 8.().().().() / f / -0DD.(0.9).(0.).(0.).(0.).(0.).(0.) /.(0.9).(0.).(0.).(0.).(0.).(0.) / -00.(0.9).(0.).(0.).(0.).(0.).(0.) / f 0 / -0DD.().(0.).(0.).(0.).(0.).(0.) /.().(0.).(0.).(0.).(0.).(0.) / -00.().(0.).(0.).(0.).(0.).(0.) / f / -0DD.9().9().0().().8().8() /.9().9().0().().8().8() / -00.9().9().0().().8().8() / f / -0DD.().8().().(0.).9().9() /.().8().().(0.).9().9() / -00.().8().().(0.).9().9() / f / -0DD.().().(.0).(0.9).(0.).(0.) /.().().(.0).(0.9).(0.).(0.) / -00.().().(.0).(0.9).(0.).(0.) / f / -0DD.(0.).(0.9) (0.) (0.) (0.) (0.) /.(0.).(0.9) (0.) (0.) (0.) (0.) / -00.(0.).(0.9) (0.) (0.) (0.) (0.) / f / -0DD.0().().() (0.) 0.(0.) 0.(0.) /.0().().() (0.) 0.(0.) 0.(0.) / -00.0().().() (0.) 0.(0.) 0.(0.) / f / -0DD.() 0.9(0.) 0.(0.) 0.(0.).(0.).(0.) /.() 0.9(0.) 0.(0.) 0.(0.).(0.).(0.) / -00.() 0.9(0.) 0.(0.) 0.(0.).(0.).(0.) / f / -0DD.(0.).8(0.) 0.(0.).0(0.) 0.9(0.).0(0.) /.(0.).8(0.) 0.(0.).0(0.) 0.9(0.).0(0.) / -00.(0.).8(0.) 0.(0.).0(0.) 0.9(0.).0(0.) / f9.e.e.e / -0DD() (0) 9(8).0().0().0() / () (0) 8().().().() / -00 () (0) 8().().().() / f / -0DD.() ().().().().() /.() ().().().().() / -00.() ().().().().() / f 0 9 / -0DD.() () (8) (8) () () /.() 8.(9) 90(9) 8() () () / -00.() 8.(9) () 0() 0() 0() / f / -0DD.() () (9) 0() 9() 9() /.() () (8) () 8() 8(08) / -00.() 9() 80(80) 9() 9(0) 90() / f / -0DD.().().() 0.9() 0.9() 0.9() /.().().0().().().() / -00.().().0().().().() / f.e.e 9.e.e.e / -0DD.0() 0() e 0/.0() e 0/ -00.0() e 0/ Table : Expected running time (ERT in number of function evaluations) divided by the respective best ERT measured during BBOB-009 (given in the respective first row) for different f values in dimension. The central 80% range divided by two is given in braces. The median number of conducted function evaluations is additionally given in italics, if ERT(0 ) =. #succ is the number of trials that reached the final target f opt Best results are printed in bold.. -0DD in the table denotes -0DDr.

8 f / -0DD.(0.) (0.) 0() () () () /.(0.) (0.) 0() () () () / -00.(0.) (0.) 0() () () () / f / -0DD() () 8() () () () / () () 8() () () () / -00 () () 8() () () () / f 0 / -0DD() e 0/ () e 0/ -00 ().9e(e) e 0/ f e 9/ -0DD.9e(e) e 0/ e 0/ -00 e 0/ f / -0DD() (9) (0) () 09() 0(9) / () (9) (0) () 09() 0(9) / -00 () (9) (0) () 09() 0(9) / f / -0DD.(0.).(0.).(0.).(0.).(0.).(0.) /.(0.).(0.).(0.).(0.).(0.).(0.) / -00.(0.).(0.).(0.).(0.).(0.).(0.) / f / -0DD.().().9().(.0).(.0).(.0) /.().().9().(.0).(.0).(.0) / -00.().().9().(.0).(.0).(.0) / f / -0DD.().(0.).(0.).8(0.).8(0.).9(0.) /.().(0.).(0.).8(0.).8(0.).9(0.) / -00.().(0.).(0.).8(0.).8(0.).9(0.) / f9 0 9 / -0DD.0().9().().().().() /.0().9().().().().() / -00.0().9().().().().() / f / -0DD.8(0.).9(0.).(0.).().().() /.8(0.).9(0.).(0.).().().() / -00.8(0.).9(0.).(0.).().().() / f / -0DD0(0.).(0.).0(0.).().().() / 0(0.).(0.).0(0.).().().() / -00 0(0.).(0.).0(0.).().().() / f / -0DD.().().().().8(0.).9(0.) /.().().().().8(0.).9(0.) / -00.().().().().8(0.).9(0.) / f / -0DD.(0.).().().().(0.9).(0.9) /.(0.).().().().(0.9).(0.9) / -00.(0.).().().().(0.9).(0.9) / f / -0DD.8(0.).(0.).(0.).(0.).(0.).(0.) /.8(0.).(0.).(0.).(0.).(0.).(0.) / -00.8(0.).(0.).(0.).(0.).(0.).(0.) / f 08.e.e.e.e.e / -0DD.(0.) 0.99(0.) 0.0(0.) 0.(0.) 0.(0.) 0.(0.) /.(0.) 0.99(0.) 0.0(0.) 0.(0.) 0.(0.) 0.(0.) / -00.(0.) 0.99(0.) 0.(0.) 0.(0.) 0.(0.) 0.(0.) / f 8 0.9e.0e.e / -0DD.(0.) 0.(0.) 0.(0.) 0.8(0.) (0.) 0.9(0.) /.(0.) 0.(0.) 0.(0.) 0.8(0.) (0.) 0.9(0.) / -00.(0.) 0.(0.) 0.(0.) 0.8(0.) (0.) 0.(0.) / f / -0DD.9() 0.8(0.) 0.(0.) 0.9(0.).(0.).(0.) /.9() 0.8(0.) 0.(0.) 0.9(0.).(0.).(0.) / -00.9() 0.8(0.) 0.(0.) 0.9(0.).(0.).(0.) / f e.e / -0DD(0.) 0.(0.) 0.(0.).() 0.9(0.).0(0.) / (0.) 0.(0.) 0.(0.).() 0.9(0.).0(0.) / -00 (0.) 0.(0.) 0.(0.).() 0.9(0.).0(0.) / f9.e.e.e.e / -0DD8().e(e).0() (0.).0(0.9).0(0.9) / 8().e(e).0() 0.(0.) 0.(0.) 0.(0.) / -00 8().e(e).0() e 0/ f0 8 0.e.e.e.e / -0DD.().() 0.9(0.).().().() /.().() (0.) 0.(0.) 0.(0.) 0.(0.) / -00.().8() e 0/ f 0 89 / -0DD.() (89) 0() () 90(0) () /.() 9(0) 89() () 9() 9(890) / -00.() 8(9) 80() () 8(0) 8() / f e / -0DD 9(8) 8(8) e 0/ 9(8) (9) e 0/ -00 () 0(89) e 0/ f..9e 8.e 8.e / -0DD.() (9).() () () () /.() (9).() 00(0) 0() 8() / -00.() (9).() (0) () () / f.e.e.e.e.e.e / -0DD e 0/ e 0/ -00 e 0/ Table : Expected running time (ERT in number of function evaluations) divided by the respective best ERT measured during BBOB-009 (given in the respective first row) for different f values in dimension 0. The central 80% range divided by two is given in braces. The median number of conducted function evaluations is additionally given in italics, if ERT(0 ) =. #succ is the number of trials that reached the final target f opt Best results are printed in bold.. -0DD in the table denotes -0DDr.