A tight runtime analysis for the (μ + λ) EA

Despite significant progress in the theory of evolutionary algorithms, the theoretical understanding of true population-based evolutionary algorithms remains challenging and only few rigorous results exist. Already for the most basic problem, the determination of the asymptotic runtime of the $(\mu+\lambda)$ evolutionary algorithm on the simple OneMax benchmark function, only the special cases $\mu=1$ and $\lambda=1$ have been solved. In this work, we analyze this long-standing problem and show the asymptotically tight result that the runtime $T$, the number of iterations until the optimum is found, satisfies \[E[T] = \Theta\bigg(\frac{n\log n}{\lambda}+\frac{n}{\lambda / \mu} + \frac{n\log^+\log^+ \lambda/ \mu}{\log^+ \lambda / \mu}\bigg),\] where $\log^+ x := \max\{1, \log x\}$ for all $x > 0$. The same methods allow to improve the previous-best $O(\frac{n \log n}{\lambda} + n \log \lambda)$ runtime guarantee for the $(\lambda+\lambda)$~EA with fair parent selection to a tight $\Theta(\frac{n \log n}{\lambda} + n)$ runtime result.

[1]  Leslie Ann Goldberg,et al.  Adaptive Drift Analysis , 2011, Algorithmica.

[2]  Benjamin Doerr,et al.  The Efficiency Threshold for the Offspring Population Size of the ($\mu$, $\lambda$) EA , 2019, 1904.06981.

[3]  Xin Yao,et al.  A New Approach for Analyzing Average Time Complexity of Population-Based Evolutionary Algorithms on Unimodal Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Timo Kötzing,et al.  Robustness of Populations in Stochastic Environments , 2014, Algorithmica.

[5]  Per Kristian Lehre,et al.  Escaping Local Optima Using Crossover With Emergent Diversity , 2018, IEEE Transactions on Evolutionary Computation.

[6]  Mehryar Mohri,et al.  Tight Lower Bound on the Probability of a Binomial Exceeding its Expectation , 2013, ArXiv.

[7]  Benjamin Doerr,et al.  The efficiency threshold for the offspring population size of the (µ, λ) EA , 2019, GECCO.

[8]  Benjamin Doerr,et al.  Multiplicative Drift Analysis , 2010, GECCO '10.

[9]  Marvin Künnemann,et al.  Optimizing linear functions with the (1+λ) evolutionary algorithm - Different asymptotic runtimes for different instances , 2015, Theor. Comput. Sci..

[10]  Per Kristian Lehre,et al.  On the Impact of Mutation-Selection Balance on the Runtime of Evolutionary Algorithms , 2012, IEEE Trans. Evol. Comput..

[11]  Thomas Jansen,et al.  Mutation Rate Matters Even When Optimizing Monotonic Functions , 2013, Evolutionary Computation.

[12]  Benjamin Doerr,et al.  Money for Nothing: Speeding Up Evolutionary Algorithms Through Better Initialization , 2015, GECCO.

[13]  Frank Neumann,et al.  Optimal Fixed and Adaptive Mutation Rates for the LeadingOnes Problem , 2010, PPSN.

[14]  Ingo Wegener,et al.  Theoretical Aspects of Evolutionary Algorithms , 2001, ICALP.

[15]  Benjamin Doerr,et al.  From black-box complexity to designing new genetic algorithms , 2015, Theor. Comput. Sci..

[16]  Benjamin Doerr,et al.  Analyzing randomized search heuristics via stochastic domination , 2019, Theor. Comput. Sci..

[17]  Xin Yao,et al.  A study of drift analysis for estimating computation time of evolutionary algorithms , 2004, Natural Computing.

[18]  Johannes Lengler,et al.  A General Dichotomy of Evolutionary Algorithms on Monotone Functions , 2018, IEEE Transactions on Evolutionary Computation.

[19]  Duc-Cuong Dang,et al.  Level-Based Analysis of Genetic Algorithms and Other Search Processes , 2018, IEEE Trans. Evol. Comput..

[20]  Kenneth A. De Jong,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods on the Choice of the Offspring Population Size in Evolutionary Algorithms on the Choice of the Offspring Population Size in Evolutionary Algorithms , 2004 .

[21]  Thomas Jansen,et al.  On benefits and drawbacks of aging strategies for randomized search heuristics , 2011, Theor. Comput. Sci..

[22]  Carsten Witt,et al.  Population size versus runtime of a simple evolutionary algorithm , 2008, Theor. Comput. Sci..

[23]  Yang Yu,et al.  Switch Analysis for Running Time Analysis of Evolutionary Algorithms , 2015, IEEE Transactions on Evolutionary Computation.

[24]  Duc-Cuong Dang,et al.  Runtime Analysis of Non-elitist Populations: From Classical Optimisation to Partial Information , 2016, Algorithmica.

[25]  Xin Yao,et al.  Drift analysis and average time complexity of evolutionary algorithms , 2001, Artif. Intell..

[26]  A. Sinclair,et al.  A computational view of population genetics , 1998 .

[27]  Per Kristian Lehre,et al.  Negative Drift in Populations , 2010, PPSN.

[28]  Benjamin Doerr,et al.  The ($$1+\lambda $$1+λ) Evolutionary Algorithm with Self-Adjusting Mutation Rate , 2018, Algorithmica.

[29]  Per Kristian Lehre,et al.  Unbiased Black-Box Complexity of Parallel Search , 2014, PPSN.

[30]  Dirk Sudholt,et al.  The choice of the offspring population size in the (1, λ) evolutionary algorithm , 2014, Theor. Comput. Sci..

[31]  Benjamin Doerr,et al.  Analyzing Randomized Search Heuristics: Tools from Probability Theory , 2011, Theory of Randomized Search Heuristics.

[32]  Carsten Witt,et al.  Runtime Analysis of the ( + 1) EA on Simple Pseudo-Boolean Functions , 2006, Evolutionary Computation.

[33]  C. Witt Population size vs. runtime of a simple EA , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[34]  Dirk Sudholt,et al.  The impact of parametrization in memetic evolutionary algorithms , 2009, Theor. Comput. Sci..

[35]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[36]  Benjamin Doerr,et al.  Probabilistic Tools for the Analysis of Randomized Optimization Heuristics , 2018, Theory of Evolutionary Computation.

[37]  H. Robbins A Remark on Stirling’s Formula , 1955 .

[38]  Benjamin Doerr,et al.  A tight runtime analysis for the (μ + λ) EA , 2018, GECCO.

[39]  Benjamin Doerr,et al.  Fast genetic algorithms , 2017, GECCO.

[40]  Benjamin Doerr,et al.  Crossover can provably be useful in evolutionary computation , 2012, Theor. Comput. Sci..

[41]  Yang Yu,et al.  A Lower Bound Analysis of Population-Based Evolutionary Algorithms for Pseudo-Boolean Functions , 2016, IDEAL.

[42]  Benjamin Doerr,et al.  Monotonic functions in EC: anything but monotone! , 2014, GECCO.

[43]  Jens Jägersküpper,et al.  Rigorous runtime analysis of a (μ+1)ES for the sphere function , 2005, GECCO '05.

[44]  Carsten Witt,et al.  Tight Bounds on the Optimization Time of a Randomized Search Heuristic on Linear Functions† , 2013, Combinatorics, Probability and Computing.

[45]  Carsten Witt,et al.  The Interplay of Population Size and Mutation Probability in the (1+λ) EA on OneMax , 2017, Algorithmica.

[46]  Benjamin Doerr,et al.  Better Runtime Guarantees via Stochastic Domination , 2018, EvoCOP.

[47]  Yuval Rabani,et al.  A computational view of population genetics , 1995, STOC '95.

[48]  Stefan Droste,et al.  Not all linear functions are equally difficult for the compact genetic algorithm , 2005, GECCO '05.

[49]  Benjamin Doerr,et al.  Lower bounds for the runtime of a global multi-objective evolutionary algorithm , 2013, 2013 IEEE Congress on Evolutionary Computation.

[50]  Benjamin Doerr,et al.  Optimal Static and Self-Adjusting Parameter Choices for the (1+(λ,λ))\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$( , 2017, Algorithmica.

[51]  Carsten Witt,et al.  Runtime Analysis of the ( μ +1) EA on Simple Pseudo-Boolean Functions , 2006 .

[52]  Benjamin Doerr,et al.  Runtime Analysis for Self-adaptive Mutation Rates , 2018, Algorithmica.

[53]  Benjamin Doerr,et al.  An Elementary Analysis of the Probability That a Binomial Random Variable Exceeds Its Expectation , 2017, Statistics & Probability Letters.