Standard Steady State Genetic Algorithms Can Hillclimb Faster Than Mutation-Only Evolutionary Algorithms

Explaining to what extent the real power of genetic algorithms (GAs) lies in the ability of crossover to recombine individuals into higher quality solutions is an important problem in evolutionary computation. In this paper we show how the interplay between mutation and crossover can make GAs hillclimb faster than their mutation-only counterparts. We devise a Markov chain framework that allows to rigorously prove an upper bound on the runtime of standard steady state GAs to hillclimb the OneMax function. The bound establishes that the steady-state GAs are 25% faster than all standard bit mutation-only evolutionary algorithms with static mutation rate up to lower order terms for moderate population sizes. The analysis also suggests that larger populations may be faster than populations of size 2. We present a lower bound for a greedy (2 + 1) GA that matches the upper bound for populations larger than 2, rigorously proving that two individuals cannot outperform larger population sizes under greedy selection and greedy crossover up to lower order terms. In complementary experiments the best population size is greater than 2 and the greedy GAs are faster than standard ones, further suggesting that the derived lower bound also holds for the standard steady state (2 + 1) GA.

[1]  Dirk Sudholt,et al.  A New Method for Lower Bounds on the Running Time of Evolutionary Algorithms , 2011, IEEE Transactions on Evolutionary Computation.

[2]  Dirk Sudholt,et al.  How crossover helps in pseudo-boolean optimization , 2011, GECCO '11.

[3]  Carsten Witt,et al.  Runtime Analysis of the ( + 1) EA on Simple Pseudo-Boolean Functions , 2006, Evolutionary Computation.

[4]  Pietro Simone Oliveto,et al.  Analysis of Diversity-Preserving Mechanisms for Global Exploration , 2009, Evolutionary Computation.

[5]  Benjamin Doerr,et al.  Fast genetic algorithms , 2017, GECCO.

[6]  Per Kristian Lehre,et al.  Theoretical analysis of rank-based mutation - combining exploration and exploitation , 2009, 2009 IEEE Congress on Evolutionary Computation.

[7]  Benjamin Doerr,et al.  Playing Mastermind With Many Colors , 2012, SODA.

[8]  Dirk Sudholt,et al.  How Crossover Speeds up Building Block Assembly in Genetic Algorithms , 2014, Evolutionary Computation.

[9]  Benjamin Doerr,et al.  The Right Mutation Strength for Multi-Valued Decision Variables , 2016, GECCO.

[10]  Xin Yao,et al.  Runtime Analysis of Evolutionary Algorithms for Discrete Optimization , 2011, Theory of Randomized Search Heuristics.

[11]  Per Kristian Lehre,et al.  Crossover can be constructive when computing unique input–output sequences , 2011, Soft Comput..

[12]  David B. Fogel,et al.  Evolutionary algorithms in theory and practice , 1997, Complex.

[13]  Ingo Wegener,et al.  Real royal road functions--where crossover provably is essential , 2001, Discret. Appl. Math..

[14]  Benjamin Doerr,et al.  Crossover can provably be useful in evolutionary computation , 2008, GECCO '08.

[15]  Thomas Jansen,et al.  The Analysis of Evolutionary Algorithms—A Proof That Crossover Really Can Help , 2002, Algorithmica.

[16]  Pietro Simone Oliveto,et al.  On the runtime analysis of the opt-IA artificial immune system , 2017, GECCO.

[17]  Benjamin Doerr,et al.  Optimal Parameter Choices Through Self-Adjustment: Applying the 1/5-th Rule in Discrete Settings , 2015, GECCO.

[18]  Benjamin Doerr,et al.  Playing Mastermind with Constant-Size Memory , 2012, Theory of Computing Systems.

[19]  Benjamin Doerr,et al.  From black-box complexity to designing new genetic algorithms , 2015, Theor. Comput. Sci..

[20]  Yang Yu,et al.  An analysis on recombination in multi-objective evolutionary optimization , 2011, GECCO '11.

[21]  Pietro Simone Oliveto,et al.  On the runtime analysis of generalised selection hyper-heuristics for pseudo-boolean optimisation , 2017, GECCO.

[22]  Per Kristian Lehre,et al.  Escaping Local Optima Using Crossover With Emergent Diversity , 2018, IEEE Transactions on Evolutionary Computation.

[23]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms: The Computer Science Perspective , 2012 .

[24]  Dirk Sudholt,et al.  Crossover is provably essential for the Ising model on trees , 2005, GECCO '05.

[25]  Pietro Simone Oliveto,et al.  On the effectiveness of crossover for migration in parallel evolutionary algorithms , 2011, GECCO '11.

[26]  Carsten Witt,et al.  Tight Bounds on the Optimization Time of a Randomized Search Heuristic on Linear Functions† , 2013, Combinatorics, Probability and Computing.

[27]  Kenneth de Jong,et al.  Generation gap methods , 2018, Evolutionary Computation 1.

[28]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[29]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[30]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[31]  Duc-Cuong Dang,et al.  Escaping Local Optima with Diversity Mechanisms and Crossover , 2016, GECCO.

[32]  Benjamin Doerr,et al.  k-Bit Mutation with Self-Adjusting k Outperforms Standard Bit Mutation , 2016, PPSN.

[33]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[34]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[35]  Duc-Cuong Dang,et al.  Escaping Local Optima using Crossover with Emergent or Reinforced Diversity , 2016, ArXiv.

[36]  Ingo Wegener,et al.  Real royal road functions for constant population size , 2003, Theor. Comput. Sci..

[37]  Carsten Witt,et al.  Runtime Analysis of the ( μ +1) EA on Simple Pseudo-Boolean Functions , 2006 .

[38]  Benjamin Doerr,et al.  A Tight Runtime Analysis of the (1+(λ, λ)) Genetic Algorithm on OneMax , 2015, GECCO.

[39]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[40]  John H. Holland,et al.  When will a Genetic Algorithm Outperform Hill Climbing , 1993, NIPS.

[41]  Per Kristian Lehre,et al.  Black-Box Search by Unbiased Variation , 2010, GECCO '10.

[42]  Duc-Cuong Dang,et al.  Emergence of Diversity and Its Benefits for Crossover in Genetic Algorithms , 2016, PPSN.

[43]  Melanie Mitchell,et al.  Relative Building-Block Fitness and the Building Block Hypothesis , 1992, FOGA.