General Upper Bounds on the Runtime of Parallel Evolutionary Algorithms*

We present a general method for analyzing the runtime of parallel evolutionary algorithms with spatially structured populations. Based on the fitness-level method, it yields upper bounds on the expected parallel runtime. This allows for a rigorous estimate of the speedup gained by parallelization. Tailored results are given for common migration topologies: ring graphs, torus graphs, hypercubes, and the complete graph. Example applications for pseudo-Boolean optimization show that our method is easy to apply and that it gives powerful results. In our examples the performance guarantees improve with the density of the topology. Surprisingly, even sparse topologies such as ring graphs lead to a significant speedup for many functions while not increasing the total number of function evaluations by more than a constant factor. We also identify which number of processors lead to the best guaranteed speedups, thus giving hints on how to parameterize parallel evolutionary algorithms.

[1]  Marco Tomassini,et al.  Takeover time curves in random and small-world structured populations , 2005, GECCO '05.

[2]  Carsten Witt,et al.  Bioinspired Computation in Combinatorial Optimization , 2010, Bioinspired Computation in Combinatorial Optimization.

[3]  Carsten Witt,et al.  Runtime Analysis of the ( μ +1) EA on Simple Pseudo-Boolean Functions , 2006 .

[4]  Frank Neumann,et al.  Approximating Covering Problems by Randomized Search Heuristics Using Multi-Objective Models , 2010, Evolutionary Computation.

[5]  Dirk Sudholt,et al.  Adaptive population models for offspring populations and parallel evolutionary algorithms , 2011, FOGA '11.

[6]  J. Jensen.  Sur les fonctions convexes et les inégalités entre les valeurs moyennes , 1906 .

[7]  G. Rudolph.  On Takeover Times in Spatially Structured Populations : Array and Ring , 2001 .

[8]  Dirk Sudholt,et al.  Design and Analysis of Schemes for Adapting Migration Intervals in Parallel Evolutionary Algorithms , 2015, Evolutionary Computation.

[9]  Dirk Sudholt,et al.  Analysis of different MMAS ACO algorithms on unimodal functions and plateaus , 2009, Swarm Intelligence.

[10]  Enrique Alba,et al.  Parallel evolutionary algorithms can achieve super-linear performance , 2002, Inf. Process. Lett..

[11]  Carsten Witt,et al.  On the Utility of Island Models in Dynamic Optimization , 2015, GECCO.

[12]  Thomas Jansen,et al.  On the Analysis of Evolutionary Algorithms - A Proof That Crossover Really Can Help , 1999 .

[13]  Dirk Sudholt,et al.  A New Method for Lower Bounds on the Running Time of Evolutionary Algorithms , 2011, IEEE Transactions on Evolutionary Computation.

[14]  Benjamin Doerr,et al.  Tight Analysis of the (1+1)-EA for the Single Source Shortest Path Problem , 2011, Evolutionary Computation.

[15]  Dirk Sudholt,et al.  How crossover helps in pseudo-boolean optimization , 2011, GECCO '11.

[16]  Carsten Witt,et al.  Runtime Analysis of the ( + 1) EA on Simple Pseudo-Boolean Functions , 2006, Evolutionary Computation.

[17]  Dirk Sudholt,et al.  General Scheme for Analyzing Running Times of Parallel Evolutionary Algorithms , 2010, PPSN.

[18]  Enrique Alba,et al.  Parallel Evolutionary Computations , 2006, Studies in Computational Intelligence.

[19]  Frank Neumann,et al.  Bioinspired computation in combinatorial optimization: algorithms and their computational complexity , 2012, GECCO '12.

[20]  Enrique Alba,et al.  Growth Curves and Takeover Time in Distributed Evolutionary Algorithms , 2004, GECCO.

[21]  Dong Zhou,et al.  The use of tail inequalities on the probable computational time of randomized search heuristics , 2012, Theor. Comput. Sci..

[22]  Benjamin Doerr,et al.  A tight analysis of the (1 + 1)-EA for the single source shortest path problem , 2007, IEEE Congress on Evolutionary Computation.

[23]  Daniel Johannsen,et al.  Random combinatorial structures and randomized search heuristics , 2010 .

[24]  Enrique Alba,et al.  Parallel Metaheuristics: A New Class of Algorithms , 2005 .

[25]  Dirk Sudholt,et al.  Running time analysis of Ant Colony Optimization for shortest path problems , 2012, J. Discrete Algorithms.

[26]  Christian Horoba.  Exploring the Runtime of an Evolutionary Algorithm for the Multi-Objective Shortest Path Problem , 2010, Evolutionary Computation.

[27]  Enrique Alba,et al.  Selection intensity in cellular evolutionary algorithms for regular lattices , 2005, IEEE Transactions on Evolutionary Computation.

[28]  Pietro Simone Oliveto,et al.  On the effectiveness of crossover for migration in parallel evolutionary algorithms , 2011, GECCO '11.

[29]  Dirk Sudholt,et al.  Homogeneous and Heterogeneous Island Models for the Set Cover Problem , 2012, PPSN.

[30]  Dirk Sudholt,et al.  Analysis of speedups in parallel evolutionary algorithms and (1+λ) EAs for combinatorial optimization , 2014, Theor. Comput. Sci..

[31]  B. H. Rudall.  Lecture notes in computer science. Vol. 15—L-systems , 1978 .

[32]  J. Rowe,et al.  Propagation time in stochastic communication networks , 2008, 2008 2nd IEEE International Conference on Digital Ecosystems and Technologies.

[33]  Zbigniew Skolicki,et al.  The influence of migration sizes and intervals on island models , 2005, GECCO '05.

[34]  Enrique Alba,et al.  Selection Intensity in Asynchronous Cellular Evolutionary Algorithms , 2003, GECCO.

[35]  Per Kristian Lehre,et al.  Fitness-levels for non-elitist populations , 2011, GECCO '11.

[36]  Xin Yao,et al.  A study of drift analysis for estimating computation time of evolutionary algorithms , 2004, Natural Computing.

[37]  Anne Auger,et al.  Theory of Randomized Search Heuristics: Foundations and Recent Developments , 2011, Theory of Randomized Search Heuristics.

[38]  Günter Rudolph,et al.  Takeover time in parallel populations with migration , 2006 .

[39]  Dirk Sudholt,et al.  Parallel Evolutionary Algorithms , 2015, Handbook of Computational Intelligence.

[40]  Dirk Sudholt,et al.  The benefit of migration in parallel evolutionary algorithms , 2010, GECCO '10.

[41]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms , 2015, Natural Computing Series.

[42]  Xin Yao,et al.  On the approximation ability of evolutionary optimization with application to minimum set cover , 2010, Artif. Intell..

[43]  W. Gutjahr,et al.  Runtime Analysis of Ant Colony Optimization with Best-So-Far Reinforcement , 2008 .

[44]  Ingo Wegener,et al.  Methods for the Analysis of Evolutionary Algorithms on Pseudo-Boolean Functions , 2003 .

[45]  Xin Yao,et al.  Towards an analytic framework for analysing the computation time of evolutionary algorithms , 2003, Artif. Intell..

[46]  Kenneth A. De Jong,et al.  An Analysis of Local Selection Algorithms in a Spatially Structured Evolutionary Algorithm , 1997, ICGA.

[47]  Carsten Witt,et al.  UNIVERSITY OF DORTMUND REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods Worst-Case and Average-Case Approximations by Simple Randomized Search Heuristics , 2004 .

[48]  Enrique Alba,et al.  Parallel Genetic Algorithms , 2011, Studies in Computational Intelligence.

[49]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms: The Computer Science Perspective , 2012 .

[50]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[51]  Erick Cantú-Paz,et al.  A Survey of Parallel Genetic Algorithms , 2000 .

[52]  Per Kristian Lehre,et al.  On the Impact of Mutation-Selection Balance on the Runtime of Evolutionary Algorithms , 2012, IEEE Trans. Evol. Comput..

[53]  Dirk Sudholt,et al.  Runtime analysis of a binary particle swarm optimizer , 2010, Theor. Comput. Sci..

[54]  Marco Tomassini,et al.  Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series) , 2005 .

[55]  Xin Yao,et al.  Time complexity of evolutionary algorithms for combinatorial optimization: A decade of results , 2007, Int. J. Autom. Comput..

[56]  Marco Tomassini,et al.  Modeling Selection Intensity for Linear Cellular Evolutionary Algorithms , 2003, Artificial Evolution.

[57]  Dirk Sudholt,et al.  The choice of the offspring population size in the (1,λ) EA , 2012, GECCO '12.