Investigating the Impact of Adaptation Sampling in Natural Evolution Strategies on Black-box Optimization Testbeds

Transcription

1 Investigating the Impact o Adaptation Sampling in Natural Evolution Strategies on Black-box Optimization Testbeds Tom Schaul Courant Institute o Mathematical Sciences, New York University Broadway, New York, USA schaul@cims.nyu.edu ABSTRACT Natural Evolution Strategies (NES) are a recent member o the class o real-valued optimization algorithms that are based on adapting search distributions. Exponential NES (xnes) are the most common instantiation o NES, and particularly appropriate or the BBOB benchmarks, given that many are non-separable, and their relatively small problem dimensions. The technique o adaptation sampling, which adapts learning rates online urther improves the algorithm s perormance. This report provides an extensive empirical comparison to study the impact o adaptation sampling in xnes, both on the noise-ree and noisy BBOB testbeds. Categories and Subject Descriptors G.. [Numerical Analysis]: Optimization global optimization, unconstrained optimization; F.. [Analysis o Algorithms and Problem Complexity]: Numerical Algorithms and Problems General Terms Algorithms Keywords Evolution Strategies, Natural Gradient, Benchmarking. INTRODUCTION Evolution strategies (ES), in contrast to traditional evolutionary algorithms, aim at repeating the type o mutation that led to those good individuals. We can characterize those mutations by an explicitly parameterized search distribution rom which new candidate samples are drawn, akin to estimation o distribution algorithms (EDA). Covariance matrix adaptation ES (CMA-ES []) innovated the ield by introducing a parameterization that includes the ull covariance matrix, allowing them to solve highly non-separable problems. Permission to make digital or hard copies o all or part o this work or personal or classroom use is granted without ee provided that copies are not made or distributed or proit or commercial advantage and that copies bear this notice and the ull citation on the irst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speciic permission and/or a ee. GECCO, July,, Philadelphia, USA. Copyright ACM ----//...$.. A more recent variant, natural evolution strategies (NES [,,, ]) aims at a higher level o generality, providing a procedure to update the search distribution s parameters or any type o distribution, by ascending the gradient towards higher expected itness. Further, it has been shown [, ] that ollowing the natural gradient to adapt the search distribution is highly beneicial, because it appropriately normalizes the update step with respect to its uncertainty and makes the algorithm scale-invariant. Exponential NES (xnes), the most common instantiation o NES, used a search distribution parameterized by a mean vector and a ull covariance matrix, and is thus most similar to CMA-ES (in act, the precise relation is described in [] and []). Given the relatively small problem dimensions o the BBOB benchmarks, and the act that many are non-separable, it is also among the most appropriate NES variants or the task. Adaptation sampling (AS) is a technique or the online adaptation o its learning rate, which is designed to speed up convergence. This may be beneicial to algorithms like xnes, because the optimization traverses qualitatively dierent phases, during which dierent learning rates may be optimal. In this report, we retain the original ormulation o xnes (including all parameter settings, except or an added stopping criterion), and compare this base-line algorithm with an AS-augmented variant. We describe the empirical perormance on all benchmark unctions (both noise-ree and noisy) o the BBOB workshop.. NATURAL EVOLUTION STRATEGIES Natural evolution strategies (NES) maintain a search distribution π and adapt the distribution parameters θ by ollowing the natural gradient [] o expected itness J, that is, maximizing Z J(θ) = E θ [(z)] = (z) π(z θ) dz Just like their close relative CMA-ES [], NES algorithms are invariant under monotone transormations o the itness unction and linear transormations o the search space. Each iteration the algorithm produces n samples z i π(z θ), i {,..., n}, i.i.d. rom its search distribution, which is parameterized by θ. The gradient w.r.t. the parameters θ can be rewritten (see []) as Z θ J(θ) = θ (z) π(z θ) dz = E θ [(z) θ log π(z θ)]

2 rom which we obtain a Monte Carlo estimate θ J(θ) nx (z i) θ log π(z i θ) n i= o the search gradient. The key step then consists in replacing this gradient h by the natural gradient deined as F θ J(θ) where F = E θ log π (z θ) θ log π (z θ) i is the Fisher inormation matrix. The search distribution is iteratively updated using natural gradient ascent θ θ + ηf θ J(θ) with learning rate parameter η.. Exponential NES While the NES ormulation is applicable to arbitrary parameterizable search distributions [, ], the most common variant employs multinormal search distributions. For that case, two helpul techniques were introduced in [], namely an exponential parameterization o the covariance matrix, which guarantees positive-deiniteness, and a novel method or changing the coordinate system into a natural one, which makes the algorithm computationally eicient. The resulting algorithm, NES with a multivariate Gaussian search distribution and using both these techniques is called xnes, and the pseudocode is given in Algorithm. Algorithm : Exponential NES (xnes) input:, µ init, η σ, η B, u k initialize µ µ init σ B I repeat or k =... n do draw sample s k N (, I) z k µ + σb s k evaluate the itness (z k ) end sort {(s k, z k )} with respect to (z k ) and assign utilities u k to each sample compute gradients δ J P n k= u k s k M J P n k= u k (s k s k I) σj tr( M J)/d B J M J σj I update parameters µ µ + σb δ J σ σ exp(η σ/ σj) B B exp(η B / B J) until stopping criterion is met. Adaptation Sampling First introduced in [] (chapter, section.), adaptation sampling is a new meta-learning technique [] that can adapt hyper-parameters online, in an economical way that is grounded on a measure statistical improvement. Here, we apply it to the learning rate o the global stepsize η σ. The idea is to consider whether a larger learningrate η σ = ησ would have been more likely to generate the good samples in the current batch. For this we determine the (hypothetical) search distribution that would have resulted rom such a larger update π( θ ). Then we compute importance weights w k = π(z k θ ) π(z k θ) or each o the n samples z k in our current population, generated rom the actual search distribution π( θ). We then conduct a weighted Mann-Whitney test [] (appendix A) to determine i the set {rank(z k )} is inerior to its reweighted counterpart {w k rank(z k )} (corresponding to the larger learning rate), with statistical signiicance ρ. I so, we increase the learning rate by a actor o + c, up to at most η σ = (where c =.). Otherwise it decays to its initial value: η σ ( c ) η σ + c η σ,init The procedure is summarized in algorithm (or details and derivations, see []). The combination o xnes with adaptation sampling is dubbed xnes-as. One interpretation o why adaptation sampling is helpul is that hal-way into the search, (ater a local attractor has been ound, e.g., towards the end o the valley on the Rosenbrock benchmarks or ), the convergence speed can be boosted by an increased learning rate. For such situations, an online adaptation o hyper-parameters is inherently wellsuited. Algorithm : Adaptation sampling input : η σ,t,η σ,init, θ t, θ t, {(z k, (z k ))}, c, ρ output: η σ,t+ compute hypothetical θ, given θ t and using /η σ,t or k =... n do w k = π(z k θ ) π(z k θ) end S {rank(z k )} S {w k rank(z k )} i weighted-mann-whitney(s, S ) < ρ then return ( c ) η σ + c η σ,init else return min(( + c ) η σ, ) end. EXPERIMENTAL SETTINGS We use identical deault hyper-parameter values or all benchmarks (both noisy and noise-ree unctions), which are taken rom [, ]. Table summarizes all the hyperparameters used. In addition, we make use o the provided target itness opt to trigger independent algorithm restarts, using a simple ad-hoc procedure: I the log-progress during the past d It turns out that this use o opt is technically not permitted by the BBOB guidelines, so strictly speaking a dierent restart strategy should be employed, or example the one described in [].

3 Table : Deault parameter values or xnes (including the utility unction and adaptation sampling) as a unction o problem dimension d. parameter deault value n η σ = η B u k ρ + log(d) ( + log(d)) d d max `, log( n + ) log(k) P n j= max `, log( n + ) log(j) n (d + ) c evaluations is too small, i.e., i opt t log opt t d < (r + ) m / [log opt t + ] where m is the remaining budget o evaluations divided by d, t is the best itness encountered until evaluation t and r is the number o restarts so ar. The total budget is d / evaluations. Implementations o this and other NES algorithm variants are available in Python through the PyBrain machine learning library [], as well as in other languages at www. idsia.ch/~tom/nes.html.. RESULTS Results rom experiments according to [] on the benchmark unctions given in [,,, ] are presented in Figures, and, and in Tables, and. The expected running time (ERT), used in the igures and table, depends on a given target unction value, t = opt +, and is computed over all relevant trials as the number o unction evaluations executed during each trial while the best unction value did not reach t, summed over all trials and divided by the number o trials that actually reached t [, ]. Statistical signiicance is tested with the rank-sum test or a given target t ( using, or each trial, either the number o needed unction evaluations to reach t (inverted and multiplied by ), or, i the target was not reached, the best -value achieved, measured only up to the smallest number o overall unction evaluations or any unsuccessul trial under consideration. Some o the result plots (like perormance scaling with dimension), as well as the CPU-timing results were omitted here but are available in stand-alone benchmarking reports [, ].. DISCUSSION Naturally, the core algorithm (xnes) being identical in both variants presented here, we do not expect drastically dierent results. In act, the direct comparison Figures and show remarkably similar results on the majority o unctions, which we see rom the points clearly scattered along the diagonal. This indicates that on most benchmarks, adaptation sampling is not used (much) and xnes-as operates with the same conservative learning rates as xnes. The unctions where we observe dierences are telling, however. Most notably on the sphere unction (, and its noisy siblings), adaptation sampling drastically improves perormance, by up to a actor. Adaptation sampling still improves perormance signiicantly on many other unctions (see Table ), but the speed-ups are less striking. There is a single unction where adaptation sampling signiicantly hurts perormance, namely on in dimension (the sharp ridge): apparently the learning rates are increased to an overly aggressive level. In conclusion, we can recommend using adaptation sampling or xnes in general, and particularly so i the itness landscapes are smooth. Acknowlegements The author wants to thank the organizers o the BBOB workshop or providing such a well-designed benchmark setup, and especially such high-quality post-processing utilities. This work was unded in part through AFR postdoc grant number, o the National Research Fund Luxembourg.. REFERENCES [] S. I. Amari. Natural Gradient Works Eiciently in Learning. Neural Computation, :,. [] S. Finck, N. Hansen, R. Ros, and A. Auger. : Presentation o the noiseless unctions. Technical Report /, Research Center PPE,. Updated February. [] S. Finck, N. Hansen, R. Ros, and A. Auger. : Presentation o the noisy unctions. Technical Report /, Research Center PPE,. [] N. Fukushima, Y. Nagata, S. Kobayashi, and I. Ono. Proposal o distance-weighted exponential natural evolution strategies. In IEEE Congress o Evolutionary Computation, pages. IEEE,. [] T. Glasmachers, T. Schaul, and J. Schmidhuber. A Natural Evolution Strategy or Multi-Objective Optimization. In Parallel Problem Solving rom Nature (PPSN),. [] T. Glasmachers, T. Schaul, Y. Sun, D. Wierstra, and J. Schmidhuber. Exponential Natural Evolution Strategies. In Genetic and Evolutionary Computation Conerence (GECCO), Portland, OR,. [] N. Hansen, A. Auger, S. Finck, and R. Ros. : Experimental setup. Technical report, INRIA,. [] N. Hansen, S. Finck, R. Ros, and A. Auger. : Noiseless unctions deinitions. Technical Report RR-, INRIA,. Updated February. [] N. Hansen, S. Finck, R. Ros, and A. Auger. : Noisy unctions deinitions. Technical Report RR-, INRIA,. Updated February.

4 Sphere Sphere Ellipsoid Ellipsoid separable separable Rastrigin Rastrigin separable separable Skew Skew Rastrigin-Bueche Rastrigin-Bueche separ Linear Linear slope slope Attractive sector Step-ellipsoid Step-ellipsoid Rosenbrock Rosenbrock original original Rosenbrock rotated Sharp Sharp ridge ridge Schaer Schaer F, F, condition cond. Gallagher Gallagher peaks peaks Ellipsoid Ellipsoid Sum Sum o o dierent di. powers powers Schaer Schaer F, F, condition cond. Gallagher Gallagher peaks peaks Discus Rastrigin Rastrigin Griewank-Rosenbrock FF Katsuuras Bent Bent cigar cigar Weierstrass Schweel x*sin(x) Schweel x*sin(x) Lunacek Lunacek bi-rastrigin bi-rastrigin Figure : Expected running time (ERT in log o number o unction evaluations) o xnes (x-axis) versus xnes-as (y-axis) or target values [, ] in each dimension on unctions. Markers on the upper or right edge indicate that the target value was never reached. Markers represent dimension: :+, :, :, :, :, :.

5 Sphere Gauss moderate noise Gauss uniorm Sphere moderate noise uni Sphere Cauchy moderate noise Cauchy Gauss Ellipsoid noise Gauss uniorm Ellipsoid noise uni Cauchy Ellipsoid Cauchy noise Sphere (moderate) Rosenbrock (moderate) Sphere Rosenbrock Step-ellipsoid Sphere (moderate) Rosenbrock (moderate) Sphere Rosenbrock Step-ellipsoid Rosenbrock moderate uni Rosenbrock moderate Gauss Sphere Gauss Sphere uni Sphere (moderate) Rosenbrock (moderate) Sphere Rosenbrock Step-ellipsoid Rosenbrock moderate Cauchy Sphere Cauchy Ellipsoid Sum o di. powers Schaer F Griewank-Rosenbrock Gallagher Sum o di powers Gauss Schaer F Gauss Ellipsoid Sum o di. powers Schaer F Griewank-Rosenbrock Gallagher Ellipsoid Sum o di. powers Schaer F Griewank-Rosenbrock Gallagher Sum o di powers uni Sum o di powers Cauchy Schaer F uni Schaer F Cauchy Rosenbrock Gauss Rosenbrock uni Rosenbrock Cauchy Griewank-Rosenbrock uni Griewank-Rosenbrock Cauchy Griewank-Rosenbrock Gauss Step-ellipsoid Gauss Step-ellipsoid uni Step-ellipsoid Cauchy Gallagher Gauss Gallagher uni Gallagher Cauchy Figure : Expected running time (ERT in log o number o unction evaluations) o xnes (x-axis) versus xnes-as (y-axis) or target values [, ] in each dimension on unctions. Markers on the upper or right edge indicate that the target value was never reached. Markers represent dimension: :+, :, :, :, :, :.

6 -D -D e+ e- e- e- e- #succ e+ e- e- e- e- #succ / / : xnes.() () () () () / : xnes.() () () () () / : +as.() () () () () / : +as.() () () () () / / / : xnes.() () () () () / : xnes (.) (.) () (.) () / : +as () () () () () / : +as () () () () () / / / : xnes.() () () () () / : xnes ().e / : +as.(.) () () () () / : +as ().e / /.e / : xnes.() () () () () / : xnes().e / : +as.() () () () () / : +as ().e / / / : xnes.() () () () () / : xnes () () () () () / : +as () () () () () / : +as.() () () () () / / / : xnes.().(.).(.).(.).(.) / : xnes.(.).(.).(.).(.).(.) / : +as.().(.).(.).(.).(.) / : +as.(.).(.).(.).(.).(.) / / / : xnes.().().().().() / : xnes.(.).(.).(.).(.).(.) / : +as.().().().().() / : +as.(.).(.).(.).(.).(.) / / / : xnes.().().().().() / : xnes.(.).().().(.) (.) / : +as.().() () () () / : +as.(.).().() () () / / / : xnes.().().().().() / : xnes.().().() () () / : +as.() () () () () / : +as.() () () () () / / / : xnes.(.).(.).(.).(.).(.) / : xnes.(.).(.).(.).(.).(.) / : +as.(.).(.).(.).(.).(.) / : +as.(.).(.).(.).(.).(.) / / / : xnes.().(.).(.).(.).(.) / : xnes.(.).(.).(.).(.).(.) / : +as.().(.).(.).(.).(.) / : +as.(.).(.).(.).(.).(.) / / / : xnes() () () () () / : xnes (.).().(.).(.).(.) / : +as () () () () () / : +as.() () () () () / / / : xnes.(.).(.).(.).(.).(.) / : +as.(.).(.).(.).(.) : xnes (.).(.).(.).(.).(.) /.(.) / : +as.() () () () () / / / : xnes.().().(.).(.).(.) / : +as.().().().(.).(.) : xnes.(.) (.) (.) (.).(.) / / : +as.(.) () () ().(.) / /.e.e.e.e / : xnes.() () () () () / : xnes ().e / : +as.() () () () () / : +as ().e / /.e.e.e / : xnes.().().().().() / : xnes () () () () () / : +as.().().().().() / : +as ().() () () () /. / / : xnes.().(.).(.).().() / : xnes.().(.).(.).() () / : +as.().(.).(.).().() / : +as.(.).(.).(.).(.) () / /.e.e / : xnes.().(.).(.).().() / : +as.(.).(.).(.) : xnes.(.).(.).(.).() () /.(.).() / : +as.(.).(.).(.).(.).() /.e.e.e / : xnes() ().().().() /.e.e.e.e / : xnes ().e / : +as () () () () () / : +as ().e / / : xnes.() ().().().() /.e.e.e.e / : xnes.().e / : +as.() () () () () / : +as.().e / / : xnes() () () () () / / : xnes () () () () () / : +as () () () () () / : +as () () () () () / / : xnes() () () () () /.e / : xnes () () () () () / : +as () () () () () / : +as () () () () () /. / : xnes.() ().e /..e.e.e / : xnes.().e / : +as.().e / : +as.().e /.e.e.e.e / : xnes.().e /.e.e.e.e.e / : xnes.e / : +as.().e / : +as.e / Table : ERT in number o unction evaluations divided by the best ERT measured during BBOB- given in the respective irst row with the central % range divided by two in brackets or dierent values. #succ is the number o trials that reached the inal target opt +. :xnes is xnes and :+as is xnes-as. Bold entries are statistically signiicantly better compared to the other algorithm, with p =. or p = k where k {,,,... } is the number ollowing the symbol, with Bonerroni correction o. A indicates the same tested against the best BBOB-.

7 noiseless unctions proportion o trials D -D proportion +: / -: / -: / -: / proportion o trials proportion +: / -: / -: / -: /. log o FEvals / DIM log o FEvals ratio. log o FEvals / DIM log o FEvals ratio noisy unctions proportion o trials proportion +: / -: / -: / -: / proportion o trials proportion +: / -: / -: / -: /. log o FEvals / DIM log o FEvals ratio. log o FEvals / DIM log o FEvals ratio Figure : Empirical cumulative distributions (ECDF) o run lengths and speed-up ratios in -D (let) and - D (right). Let sub-columns: ECDF o the number o unction evaluations divided by dimension D (FEvals/D) to reach a target value opt + with = k, where k {,,, } is given by the irst value in the legend, or xnes ( ) and xnes-as ( ). Light beige lines show the ECDF o FEvals or target value = o all algorithms benchmarked during BBOB-. Right sub-columns: ECDF o FEval ratios o xnes divided by xnes-as, all trial pairs or each unction. Pairs where both trials ailed are disregarded, pairs where one trial ailed are visible in the limits being > or <. The legends indicate the number o unctions that were solved in at least one trial (xnes irst). -D -D e+ e- e- e- e- #succ e+ e- e- e- e- #succ / / : xnes.().(.) () () (.) / : xnes.() (.) (.) (.) (.) / : +as.().().() () () / : +as.().().().().() / / / : xnes.().().() () () / : xnes.(.) (.) (.) (.) (.) / : +as.().().().() () / : +as.(.).().().().() / / / : xnes.().() () () () / : xnes.() (.).(.) (.) (.) / : +as.().() () () () / : +as.().().().().() / /.e.e.e.e / : xnes.(.).().().().() / : xnes.().e / : +as.(.).().().().() / : +as ().e / /.e.e.e.e.e / : xnes.(.).().().().() / : xnes().e / : +as.(.) () () () () / : +as ().e / / / : xnes.(.).(.).(.).(.).() / : xnes.(.).(.).(.).(.).(.) / : +as.(.).().().().() / : +as.(.).(.).(.).(.).(.) / / / : xnes.() () ().() () / : xnes.(.) ().e / : +as.().(.).().().() / : +as.(.) ().e / /.e.e.e.e / : xnes() ().e / : xnes.e / : +as () ().e / : +as.e / / / : xnes.().(.).(.).(.).(.)/ : xnes.(.).(.) (.) (.) (.) / : +as.().(.).(.).(.).(.)/ : +as.(.).(.).(.) (.) () /.e.e.e.e / : xnes.(.).().().().() / : xnes / : +as.(.).().().().() / : +as /.e.e.e.e / : xnes.().(.).e / : xnes / : +as.().().e / : +as / / / : xnes.().().() () () / : xnes.(.) ().e / : +as.().() () () () / : +as.(.) () () () () / /.e.e.e.e / : xnes().().().().() / : xnes.().e / : +as.().().().().() / : +as.().e / /.e.e.e.e.e / : xnes() ().e / : xnes.e / : +as.() ().e / : +as.e / /.e.e.e / : xnes.().().().().() / : xnes.(.).(.).(.).(.).(.) / : +as.(.).().().().() / : +as.(.).(.).(.).(.).(.) / Table : Relative ERT in number o -evaluations, see Table or details.

8 -D -D e+ e- e- e- e- #succ e+ e- e- e- e- #succ /.e.e.e.e.e / : xnes.().().().().() / : xnes().e / : +as.().().().().() / : +as ().e /.e.e.e.e /.e.e.e.e.e / : xnes.().e / : xnes.e / : +as.().e / : +as.e / / / : xnes.(.).(.).(.).(.).(.)/ : xnes.(.).(.).(.).(.).(.) / : +as.(.).(.).(.).(.).(.)/ : +as.(.).(.).(.).(.).(.) / /.e.e.e / : xnes.().(.).().() () / : xnes.(.) ().e / : +as.().().().().() / : +as.(.) ().e /.e.e /.e.e.e.e / : xnes.() ().e / : xnes ().e / : +as () ().e / : +as.().e /. / / : xnes.().(.).(.).().() / : xnes.(.).(.).(.).(.).(.) / : +as.().(.).(.).().() / : +as.(.).(.).(.).(.).() /.e /.e.e.e.e / : xnes.().() ().e / : xnes.(.).e / : +as.().() ().e / : +as.(.).e /.e.e.e /.e.e.e.e : xnes().e / : xnes.().e / : +as.().e / : +as ().e / /.e.e.e / : xnes.().().() () () / : xnes.(.).(.).().e / : +as.().(.).() () () / : +as.(.).(.).().e /.e.e.e /.e.e.e / : xnes. ().e / : xnes.().e / : +as.(.) ().e / : +as.().e / : xnes. () / : xnes.(.) / : +as.(.) () / : +as.() /.e.e.e /.e.e.e / : xnes. () ().e / : xnes.().e / : +as.(.) ().e / : +as.(.).e / /.e.e.e.e.e / : xnes.().().().().() / : xnes ().().().().() / : +as ().().().().() / : +as ().(.).().(.).() /.e.e.e /.e.e.e.e.e / : xnes() () ().e / : xnes.e / : +as ().() ().e / : +as.e / /.e.e.e.e / : xnes() ().().().() / : xnes ().().().().() / : +as (.) ().().().() / : +as ().().().().() / Table : Relative ERT in number o -evaluations, see Table or details. [] N. Hansen and A. Ostermeier. Completely derandomized sel-adaptation in evolution strategies. IEEE Transactions on Evolutionary Computation, :,. [] K. Price. Dierential evolution vs. the unctions o the second ICEO. In Proceedings o the IEEE International Congress on Evolutionary Computation, pages,. [] T. Schaul. Studies in Continuous Black-box Optimization. Ph.D. thesis, Technische Universität München,. [] T. Schaul. Benchmarking Exponential Natural Evolution Strategies on the Noiseless and Noisy Black-box Optimization Testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conerence, Philadelphia, PA,. [] T. Schaul. Benchmarking Natural Evolution Strategies with Adaptation Sampling on the Noiseless and Noisy Black-box Optimization Testbeds. In Black-box Optimization Benchmarking Workshop, Genetic and Evolutionary Computation Conerence, Philadelphia, PA,. [] T. Schaul. Natural Evolution Strategies Converge on Sphere Functions. In Genetic and Evolutionary Computation Conerence (GECCO), Philadelphia, PA,. [] T. Schaul, J. Bayer, D. Wierstra, Y. Sun, M. Felder, F. Sehnke, T. Rückstieß, and J. Schmidhuber. PyBrain. Journal o Machine Learning Research, :,. [] T. Schaul and J. Schmidhuber. Metalearning. Scholarpedia, ():,. [] Y. Sun, D. Wierstra, T. Schaul, and J. Schmidhuber. Stochastic search using the natural gradient. In International Conerence on Machine Learning (ICML),. [] D. Wierstra, T. Schaul, T. Glasmachers, Y. Sun, and J. Schmidhuber. Natural Evolution Strategies. Technical report,. [] D. Wierstra, T. Schaul, J. Peters, and J. Schmidhuber. Natural Evolution Strategies. In Proceedings o the IEEE Congress on Evolutionary Computation (CEC), Hong Kong, China,.