A Tight Runtime Analysis of the $(1+(\lambda, \lambda))$ Genetic Algorithm on OneMax

Understanding how crossover works is still one of the big challenges in evolutionary computation research, and making our understanding precise and proven by mathematical means might be an even bigger one. As one of few examples where crossover provably is useful, the $(1+(\lambda, \lambda))$ Genetic Algorithm (GA) was proposed recently in [Doerr, Doerr, Ebel: TCS 2015]. Using the fitness level method, the expected optimization time on general OneMax functions was analyzed and a $O(\max\{n\log(n)/\lambda, \lambda n\})$ bound was proven for any offspring population size $\lambda \in [1..n]$. We improve this work in several ways, leading to sharper bounds and a better understanding of how the use of crossover speeds up the runtime in this algorithm. We first improve the upper bound on the runtime to $O(\max\{n\log(n)/\lambda, n\lambda \log\log(\lambda)/\log(\lambda)\})$. This improvement is made possible from observing that in the parallel generation of $\lambda$ offspring via crossover (but not mutation), the best of these often is better than the expected value, and hence several fitness levels can be gained in one iteration. We then present the first lower bound for this problem. It matches our upper bound for all values of $\lambda$. This allows to determine the asymptotically optimal value for the population size. It is $\lambda = \Theta(\sqrt{\log(n)\log\log(n)/\log\log\log(n)})$, which gives an optimization time of $\Theta(n \sqrt{\log(n)\log\log\log(n)/\log\log(n)})$. Hence the improved runtime analysis gives a better runtime guarantee along with a better suggestion for the parameter $\lambda$. We finally give a tail bound for the upper tail of the runtime distribution, which shows that the actual runtime exceeds our runtime guarantee by a factor of $(1+\delta)$ with probability $O((n/\lambda^2)^{-\delta})$ only.

[1]  Xin Yao,et al.  Drift analysis and average time complexity of evolutionary algorithms , 2001, Artif. Intell..

[2]  Xin Yao,et al.  Runtime Analysis of Evolutionary Algorithms for Discrete Optimization , 2011, Theory of Randomized Search Heuristics.

[3]  Benjamin Doerr,et al.  From black-box complexity to designing new genetic algorithms , 2015, Theor. Comput. Sci..

[4]  Kenneth A. De Jong,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods on the Choice of the Offspring Population Size in Evolutionary Algorithms on the Choice of the Offspring Population Size in Evolutionary Algorithms , 2004 .

[5]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms , 2015, Natural Computing Series.

[6]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms: The Computer Science Perspective , 2012 .

[7]  John H. Holland,et al.  When will a Genetic Algorithm Outperform Hill Climbing , 1993, NIPS.

[8]  Marvin Künnemann,et al.  Optimizing linear functions with the (1+λ) evolutionary algorithm - Different asymptotic runtimes for different instances , 2015, Theor. Comput. Sci..

[9]  Ingo Wegener,et al.  The Ising Model on the Ring: Mutation Versus Recombination , 2004, GECCO.

[10]  Martin Raab,et al.  "Balls into Bins" - A Simple and Tight Analysis , 1998, RANDOM.

[11]  Benjamin Doerr,et al.  Crossover can provably be useful in evolutionary computation , 2008, GECCO '08.

[12]  Ingo Wegener,et al.  Theoretical Aspects of Evolutionary Algorithms , 2001, ICALP.

[13]  Thomas Jansen,et al.  The Analysis of Evolutionary Algorithms—A Proof That Crossover Really Can Help , 2002, Algorithmica.

[14]  William F. Punch,et al.  Parameter-less population pyramid , 2014, GECCO.

[15]  Benjamin Doerr,et al.  Improved analysis methods for crossover-based algorithms , 2009, GECCO.

[16]  Frank Neumann,et al.  More Effective Crossover Operators for the All-Pairs Shortest Path Problem , 2010, PPSN.

[17]  Ingo Wegener,et al.  Methods for the Analysis of Evolutionary Algorithms on Pseudo-Boolean Functions , 2003 .

[18]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[19]  Dirk Sudholt,et al.  Crossover is provably essential for the Ising model on trees , 2005, GECCO '05.

[20]  Timo Kötzing Concentration of First Hitting Times Under Additive Drift , 2015, Algorithmica.

[21]  Per Kristian Lehre,et al.  Black-Box Search by Unbiased Variation , 2010, GECCO '10.

[22]  Benjamin Doerr,et al.  Lessons from the black-box: fast crossover-based genetic algorithms , 2013, GECCO '13.

[23]  Carsten Witt,et al.  Fitness levels with tail bounds for the analysis of randomized search heuristics , 2014, Inf. Process. Lett..

[24]  Benjamin Doerr,et al.  Analyzing Randomized Search Heuristics: Tools from Probability Theory , 2011, Theory of Randomized Search Heuristics.