Evolution Strategies with Additive Noise: A Convergence Rate Lower Bound

We consider the problem of optimizing functions corrupted with additive noise. It is known that Evolutionary Algorithms can reach a Simple Regret O(1/√n) within logarithmic factors, when n is the number of function evaluations. Here, Simple Regret at evaluation $n$ is the difference between the evaluation of the function at the current recommendation point of the algorithm and at the real optimum. We show mathematically that this bound is tight, for any family of functions that includes sphere functions, at least for a wide set of Evolution Strategies without large mutations.

[1]  Christian Igel,et al.  Hoeffding and Bernstein races for selecting policies in evolutionary direct policy search , 2009, ICML '09.

[2]  H.-G. Beyer,et al.  Mutate large, but inherit small ! On the analysis of rescaled mutations in (1, λ)-ES with noisy fitness data , 1998 .

[3]  Anne Auger,et al.  Convergence results for the (1, lambda)-SA-ES using the theory of phi-irreducible Markov chains , 2005, Theor. Comput. Sci..

[4]  Rémi Coulom,et al.  CLOP: Confident Local Optimization for Noisy Black-Box Parameter Tuning , 2011, ACG.

[5]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[6]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[7]  Olivier Teytaud,et al.  Noisy optimization complexity under locality assumption , 2013, FOGA XII '13.

[8]  Z. K. Zhang,et al.  Global convergence of unconstrained and bound constrained surrogate-assisted evolutionary search in aerodynamic shape design , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[9]  Olivier Teytaud,et al.  Lower Bounds for Evolution Strategies Using VC-Dimension , 2008, PPSN.

[10]  V. Fabian.  Stochastic Approximation of Minima with Improved Asymptotic Speed , 1967 .

[11]  Yew-Soon Ong,et al.  Hierarchical surrogate-assisted evolutionary optimization framework , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[12]  Hans-Georg Beyer,et al.  Investigation of the (/spl mu/, /spl lambda/)-ES in the presence of noise , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[13]  A. Auger.  Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[14]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[15]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[16]  Ohad Shamir,et al.  On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization , 2012, COLT.

[17]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[18]  Hung Chen.  Lower Rate of Convergence for Locating a Maximum of a Function , 1988 .

[19]  Olivier Teytaud,et al.  Local and global order 3/2 convergence of a surrogate evolutionary algorithm , 2005, GECCO '05.

[20]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[21]  Olivier Teytaud,et al.  Adaptive Noisy Optimization , 2010, EvoApplications.

[22]  James C. Spall,et al.  Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2007, IEEE Transactions on Automatic Control.

[23]  Olivier Teytaud,et al.  Algorithms (X, sigma, eta): Quasi-random Mutations for Evolution Strategies , 2005, Artificial Evolution.

[24]  James C. Spall,et al.  Adaptive stochastic approximation by the simultaneous perturbation method , 2000, IEEE Trans. Autom. Control..

[25]  Hans-Georg Beyer,et al.  Local performance of the (1 + 1)-ES in a noisy environment , 2002, IEEE Trans. Evol. Comput..

[26]  Nataliya Sokolovska,et al.  Handling expensive optimization with large noise , 2011, FOGA '11.

[27]  Olivier Teytaud,et al.  Log-log Convergence for Noisy Optimization , 2013, Artificial Evolution.

[28]  Marie-Liesse Cauwet.  Noisy Optimization: Convergence with a Fixed Number of Resamplings , 2014, EvoApplications.

[29]  J. Spall.  Adaptive stochastic approximation by the simultaneous perturbation method , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).