Evolution Strategies

Evolution strategies are evolutionary algorithms that date back to the 1960s and that are most commonly applied to black-box optimization problems in continuous search spaces. Inspired by biological evolution, their original formulation is based on the application of mutation, recombination and selection in populations of candidate solutions. From the algorithmic viewpoint, evolution strategies are optimization methods that sample new candidate solutions stochastically, most commonly from a multivariate normal probability distribution. Their two most prominent design principles are unbiasedness and adaptive control of parameters of the sample distribution. In this overview the important concepts of success based step-size control, self-adaptation and derandomization are covered, as well as more recent developments like covariance matrix adaptation and natural evolution strategies. The latter give new insights into the fundamental mathematical rationale behind evolution strategies. A broad discussion of theoretical results includes progress rate results on various function classes and convergence proofs for evolution strategies.

[1]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[2]  K. Steiglitz,et al.  Adaptive step size random search , 1968 .

[3]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[4]  H. P. Schwefel,et al.  Numerische Optimierung von Computermodellen mittels der Evo-lutionsstrategie , 1977 .

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  A. A. Zhigli︠a︡vskiĭ,et al.  Theory of Global Random Search , 1991 .

[7]  Andreas Ostermeier,et al.  An Evolution Strategy with Momentum Adaptation of the Random Number Distribution , 1992, PPSN.

[8]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[9]  Michael Herdy,et al.  The number of offspring as strategy parameter in hierarchically organized evolution strategies , 1993, SIGB.

[10]  Nikolaus Hansen,et al.  A Derandomized Approach to Self-Adaptation of Evolution Strategies , 1994, Evolutionary Computation.

[11]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: On the Benefits of Sex the (/, ) Theory , 1995, Evolutionary Computation.

[12]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[13]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: Self-Adaptation , 1995, Evolutionary Computation.

[14]  Nikolaus Hansen,et al.  On the Adaptation of Arbitrary Normal Mutation Distributions in Evolution Strategies: The Generating Set Adaptation , 1995, ICGA.

[15]  Cornelia Kappler,et al.  Are Evolutionary Algorithms Improved by Large Mutations? , 1996, PPSN.

[16]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[17]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[18]  H.-G. Beyer,et al.  Mutate large, but inherit small ! On the analysis of rescaled mutations in (1, λ)-ES with noisy fitness data , 1998 .

[19]  G. Unter Rudolph.  Local Convergence Rates of Simple Evolutionary Algorithms with Cauchy Mutations , 1998 .

[20]  Ralf Salomon,et al.  Evolutionary algorithms and gradient search: similarities and differences , 1998, IEEE Trans. Evol. Comput..

[21]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[22]  N. Hansen,et al.  An Evolution Strategy with Coordinate System Invariant Adaptation of Arbitrary Normal Mutation Distr , 1999 .

[23]  Hans-Georg Beyer,et al.  Analysis of the (/, )-ES on the Parabolic Ridge , 2000, Evolutionary Computation.

[24]  Nikolaus Hansen,et al.  Invariance, Self-Adaptation and Correlated Mutations and Evolution Strategies , 2000, PPSN.

[25]  Hans-Georg Beyer,et al.  Local Performance of the (μ/μ, μ)-ES in a Noisy Environment , 2000, FOGA.

[26]  Günter Rudolph,et al.  Self-adaptive mutations may lead to premature convergence , 2001, IEEE Trans. Evol. Comput..

[27]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[28]  Andreas Zell,et al.  Main vector adaptation: a CMA variant with linear time and space complexity , 2001 .

[29]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[30]  Hans-Georg Beyer,et al.  Local performance of the (1 + 1)-ES in a noisy environment , 2002, IEEE Trans. Evol. Comput..

[31]  Thomas Philip Runarsson,et al.  Reducing Random Fluctuations in Mutative Self-adaptation , 2002, PPSN.

[32]  Juan Julián Merelo Guervós,et al.  Parallel Problem Solving from Nature — PPSN VII , 2002, Lecture Notes in Computer Science.

[33]  Dirk V. Arnold,et al.  Noisy Optimization With Evolution Strategies , 2002, Genetic Algorithms and Evolutionary Computation.

[34]  Hans-Georg Beyer,et al.  On the Benefits of Populations for Noisy Optimization , 2003, Evolutionary Computation.

[35]  Olivier François,et al.  Global convergence for evolution strategies in spherical problems: some simple proofs and difficulties , 2003, Theor. Comput. Sci..

[36]  Petros Koumoutsakos,et al.  Learning probability distributions in continuous evolutionary algorithms – a comparative review , 2004, Natural Computing.

[37]  Hans-Georg Beyer,et al.  Performance analysis of evolutionary optimization with cumulative step length adaptation , 2004, IEEE Transactions on Automatic Control.

[38]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[39]  Petros Koumoutsakos,et al.  Learning Probability Distributions in Continuous Evolutionary Algorithms - a Comparative Review , 2004, Nat. Comput..

[40]  David E. Goldberg,et al.  The parameter-less genetic algorithm in practice , 2004, Inf. Sci..

[41]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[42]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[43]  A. Auger.  Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[44]  Dirk V. Arnold,et al.  Improving Evolution Strategies through Active Covariance Matrix Adaptation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[45]  Dirk V. Arnold,et al.  Hierarchically organised evolution strategies on the parabolic ridge , 2006, GECCO '06.

[46]  Hans-Georg Beyer,et al.  Evolution strategies with cumulative step length adaptation on the noisy parabolic ridge , 2008, Natural Computing.

[47]  Jens Jägersküpper,et al.  How the (1+1) ES using isotropic mutations minimizes positive definite quadratic forms , 2006, Theor. Comput. Sci..

[48]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[49]  Hans-Georg Beyer,et al.  Optimum Tracking with Evolution Strategies , 2006, Evolutionary Computation.

[50]  Nikolaus Hansen,et al.  An Analysis of Mutative -Self-Adaptation on Linear Fitness Functions , 2006, Evolutionary Computation.

[51]  Hans-Georg Beyer,et al.  A general noise model and its effects on evolution strategy performance , 2006, IEEE Transactions on Evolutionary Computation.

[52]  Dirk V. Arnold,et al.  Weighted multirecombination evolution strategies , 2006, Theor. Comput. Sci..

[53]  Anne Auger,et al.  Reconsidering the progress rate theory for evolution strategies in finite dimensions , 2006, GECCO '06.

[54]  Anne Auger,et al.  Log-Linear Convergence and Optimal Bounds for the (1+1)-ES , 2007, Artificial Evolution.

[55]  Stefan Roth,et al.  Covariance Matrix Adaptation for Multi-objective Optimization , 2007, Evolutionary Computation.

[56]  Dirk V. Arnold,et al.  On the use of evolution strategies for optimising certain positive definite quadratic forms , 2007, GECCO '07.

[57]  Jens Jägersküpper,et al.  Algorithmic analysis of a basic evolutionary algorithm for continuous optimization , 2007, Theor. Comput. Sci..

[58]  Hans-Georg Beyer,et al.  Mutative self-adaptation on the sharp and parabolic ridge , 2007, FOGA'07.

[59]  James N. Knight,et al.  Reducing the space-time complexity of the CMA-ES , 2007, GECCO '07.

[60]  Jens Jägersküpper.  Lower Bounds for Hit-and-Run Direct Search , 2007, SAGA.

[61]  Bernhard Sendhoff,et al.  Covariance Matrix Adaptation Revisited - The CMSA Evolution Strategy - , 2008, PPSN.

[62]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[63]  Dirk V. Arnold,et al.  Step Length Adaptation on Ridge Functions , 2008, Evolutionary Computation.

[64]  Jens Jägersküpper,et al.  Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods General Lower Bounds for Randomized Direct Search with Isotropic Sampling , 2008 .

[65]  Dirk V. Arnold,et al.  On the Behaviour of the (1+1)-ES for a Simple Constrained Problem , 2008, PPSN.

[66]  Christian Igel,et al.  Efficient covariance matrix update for variable metric evolution strategies , 2009, Machine Learning.

[67]  Tom Schaul,et al.  Stochastic search using the natural gradient , 2009, ICML '09.

[68]  Hans-Georg Beyer,et al.  On strategy parameter control by Meta-ES , 2009, GECCO '09.

[69]  Anne Auger,et al.  Log-Linear Convergence and Divergence of the Scale-Invariant (1+1)-ES in Noisy Environments , 2011, Algorithmica.

[70]  Isao Ono,et al.  Bidirectional Relation between CMA Evolution Strategies and Natural Evolution Strategies , 2010, PPSN.

[71]  Christian Igel,et al.  Improved step size adaptation for the MO-CMA-ES , 2010, GECCO '10.

[72]  Hans-Georg Beyer,et al.  On the Behaviour of Evolution Strategies Optimising Cigar Functions , 2010, Evolutionary Computation.

[73]  Olivier Teytaud,et al.  Lower Bounds for Comparison Based Evolution Strategies Using VC-dimension and Sign Patterns , 2011, Algorithmica.

[74]  Dirk V. Arnold,et al.  Analysis of a repair mechanism for the (1,λ)-ES applied to a simple constrained problem , 2011, GECCO '11.

[75]  Anne Auger,et al.  Mirrored sampling in evolution strategies with weighted recombination , 2011, GECCO '11.

[76]  Dirk V. Arnold,et al.  On the behaviour of the (1,λ)-es for a simple constrained problem , 2011, FOGA '11.

[77]  Anne Auger,et al.  Analyzing the impact of mirrored sampling and sequential selection in elitist evolution strategies , 2011, FOGA '11.

[78]  Stephen R. Marsland,et al.  Convergence Properties of (μ + λ) Evolutionary Algorithms , 2011, AAAI.

[79]  Anne Auger,et al.  Theory of Evolution Strategies: A New Perspective , 2011, Theory of Randomized Search Heuristics.

[80]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[81]  Fernando G. Lobo,et al.  Introduction to Estimation of Distribution Algorithms , 2012 .

[82]  Anne Auger,et al.  Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles , 2011, J. Mach. Learn. Res..