Log-Linear Convergence and Divergence of the Scale-Invariant (1+1)-ES in Noisy Environments

Noise is present in many real-world continuous optimization problems. Stochastic search algorithms such as Evolution Strategies (ESs) have been proposed as effective search methods in such contexts. In this paper, we provide a mathematical analysis of the convergence of a (1+1)-ES on unimodal spherical objective functions in the presence of noise. We prove for a multiplicative noise model that for a positive expected value of the noisy objective function, convergence or divergence happens depending on the infimum of the support of the noise. Moreover, we investigate convergence rates and show that log-linear convergence is preserved in presence of noise. This result is a strong theoretical foundation of the robustness of ESs with respect to noise.

[1]  K. Steiglitz,et al.  Adaptive step size random search , 1968 .

[2]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[3]  Hans-Georg Beyer,et al.  Toward a Theory of Evolution Strategies: Some Asymptotical Results from the (1,+ )-Theory , 1993, Evolutionary Computation.

[4]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[5]  Volker Nissen,et al.  On the robustness of population-based versus point-based optimization in the presence of noise , 1998, IEEE Trans. Evol. Comput..

[6]  Hans-Georg Beyer,et al.  Efficiency and mutation strength adaptation of the (μ/μI, λ)-ES in a noisy environment , 2000 .

[7]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[8]  Hans-Georg Beyer,et al.  Efficiency and Mutation Strength Adaptation of the in a Noisy Environment , 2000 .

[9]  Hans-Georg Beyer,et al.  Investigation of the (/spl mu/, /spl lambda/)-ES in the presence of noise , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[10]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[11]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[12]  Hans-Georg Beyer,et al.  Local performance of the (1 + 1)-ES in a noisy environment , 2002, IEEE Trans. Evol. Comput..

[13]  Dirk V. Arnold,et al.  Noisy Optimization With Evolution Strategies , 2002, Genetic Algorithms and Evolutionary Computation.

[14]  Hans-Georg Beyer,et al.  A Comparison of Evolution Strategies with Other Direct Search Methods in the Presence of Noise , 2003, Comput. Optim. Appl..

[15]  Olivier François,et al.  Global convergence for evolution strategies in spherical problems: some simple proofs and difficulties , 2003, Theor. Comput. Sci..

[16]  Jens Jägersküpper,et al.  Analysis of a Simple Evolutionary Algorithm for Minimization in Euclidean Spaces , 2003, ICALP.

[17]  Dirk V. Arnold,et al.  Optimal Weighted Recombination , 2005, FOGA.

[18]  Anne Auger,et al.  Convergence results for the (1, lambda)-SA-ES using the theory of phi-irreducible Markov chains , 2005, Theor. Comput. Sci..

[19]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[20]  Jens Jägersküpper,et al.  Rigorous Runtime Analysis of the (1+1) ES: 1/5-Rule and Ellipsoidal Fitness Landscapes , 2005, FOGA.

[21]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[22]  A. Auger.  Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[23]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[24]  Anne Auger,et al.  Reconsidering the progress rate theory for evolution strategies in finite dimensions , 2006, GECCO '06.

[25]  Anne Auger,et al.  Log-Linear Convergence and Optimal Bounds for the (1+1)-ES , 2007, Artificial Evolution.

[26]  Jens Jägersküpper,et al.  Algorithmic analysis of a basic evolutionary algorithm for continuous optimization , 2007, Theor. Comput. Sci..

[27]  Anne Auger,et al.  On Multiplicative Noise Models for Stochastic Search , 2008, PPSN.

[28]  Takéhiko Nakama.  Markov Chain Analysis of Genetic Algorithms in a Noisy Environment , 2008 .

[29]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[30]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 noisy testbed , 2009, GECCO '09.

[31]  Takéhiko Nakama,et al.  Markov chain analysis of genetic algorithms in a wide variety of noisy environments , 2009, GECCO.

[32]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[33]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[34]  Stephen R. Marsland,et al.  Convergence Properties of (μ + λ) Evolutionary Algorithms , 2011, AAAI.