Log-Linear Convergence and Optimal Bounds for the (1+1)-ES

The (1 + 1)-ES is modeled by a general stochastic processwhose asymptotic behavior is investigated. Under general assumptions, itis shown that the convergence of the related algorithm is sub-log-linear,bounded below by an explicit log-linear rate. For the specific case ofspherical functions and scale-invariant algorithm, it is proved using theLaw of Large Numbers for orthogonal variables, that the linear convergenceholds almost surely and that the best convergence rate is reached.Experimental simulations illustrate the theoretical results.

[1]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[2]  Edmund K. Burke,et al.  Parallel Problem Solving from Nature - PPSN IX: 9th International Conference, Reykjavik, Iceland, September 9-13, 2006, Proceedings , 2006, PPSN.

[3]  Anne Auger,et al.  Convergence results for the (1, lambda)-SA-ES using the theory of phi-irreducible Markov chains , 2005, Theor. Comput. Sci..

[4]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[5]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[6]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[7]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[8]  Jens Jägersküpper.  Lower Bounds for Hit-and-Run Direct Search , 2007, SAGA.

[9]  A. Auger.  Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[10]  Anne Auger,et al.  Reconsidering the progress rate theory for evolution strategies in finite dimensions , 2006, GECCO '06.

[11]  Olivier François,et al.  Global convergence for evolution strategies in spherical problems: some simple proofs and difficulties , 2003, Theor. Comput. Sci..

[12]  Michel Loève,et al.  Probability Theory I , 1977 .

[13]  Petros Koumoutsakos,et al.  Learning probability distributions in continuous evolutionary algorithms – a comparative review , 2004, Natural Computing.