Time Complexity Analysis of Evolutionary Algorithms on Random Satisfiable k-CNF Formulas

We contribute to the theoretical understanding of randomized search heuristics by investigating their optimization behavior on satisfiable random k-satisfiability instances both in the planted solution model and the uniform model conditional on satisfiability. Denoting the number of variables by n, our main technical result is that the simple ($$1+1$$1+1) evolutionary algorithm with high probability finds a satisfying assignment in time $$O(n \log n)$$O(nlogn) when the clause-variable density is at least logarithmic. For low density instances, evolutionary algorithms seem to be less effective, and all we can show is a subexponential upper bound on the runtime for densities below $$\frac{1}{k(k-1)}$$1k(k-1). We complement these mathematical results with numerical experiments on a broader density spectrum. They indicate that, indeed, the ($$1+1$$1+1) EA is less efficient on lower densities. Our experiments also suggest that the implicit constants hidden in our main runtime guarantee are low. Our main result extends and considerably improves the result obtained by Sutton and Neumann (Lect Notes Comput Sci 8672:942–951, 2014) in terms of runtime, minimum density, and clause length. These improvements are made possible by establishing a close fitness-distance correlation in certain parts of the search space. This approach might be of independent interest and could be useful for other average-case analyses of randomized search heuristics. While the notion of a fitness-distance correlation has been around for a long time, to the best of our knowledge, this is the first time that fitness-distance correlation is explicitly used to rigorously prove a performance statement for an evolutionary algorithm.

[1]  Toby Walsh,et al.  Local Search and the Number of Solutions , 1996, CP.

[2]  S Kirkpatrick,et al.  Critical Behavior in the Satisfiability of Random Boolean Expressions , 1994, Science.

[3]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms , 2015, Natural Computing Series.

[4]  Qiang Zhang,et al.  On the simulation of expressional animation based on facial MoCap , 2011, Science China Information Sciences.

[5]  Victor J. Rayward-Smith,et al.  Fitness Distance Correlation and Ridge Functions , 1998, PPSN.

[6]  Frank Neumann,et al.  Runtime Analysis of Evolutionary Algorithms on Randomly Constructed High-Density Satisfiable 3-CNF Formulas , 2014, PPSN.

[7]  Carsten Witt,et al.  Fitness levels with tail bounds for the analysis of randomized search heuristics , 2014, Inf. Process. Lett..

[8]  Evgeny S. Skvortsov A Theoretical Analysis of Search in GSAT , 2009, SAT.

[9]  C.H. Papadimitriou,et al.  On selecting a satisfying truth assignment , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[10]  Amin Coja-Oghlan,et al.  The asymptotic k-SAT threshold , 2014, STOC.

[11]  Ming-Te Chao,et al.  Probabilistic analysis of a generalization of the unit-clause literal selection heuristics for the k satisfiability problem , 1990, Inf. Sci..

[12]  Benjamin Doerr,et al.  Analyzing Randomized Search Heuristics: Tools from Probability Theory , 2011, Theory of Randomized Search Heuristics.

[13]  Leslie Ann Goldberg,et al.  Adaptive Drift Analysis , 2010, PPSN.

[14]  Abraham D. Flaxman,et al.  A spectral technique for random satisfiable 3CNF formulas , 2003, SODA '03.

[15]  Christos H. Papadimitriou,et al.  On the Greedy Algorithm for Satisfiability , 1992, Information Processing Letters.

[16]  Alan M. Frieze,et al.  Analysis of Two Simple Heuristics on a Random Instance of k-SAT , 1996, J. Algorithms.

[17]  Carsten Witt,et al.  Bioinspired Computation in Combinatorial Optimization , 2010, Bioinspired Computation in Combinatorial Optimization.

[18]  Lee Altenberg,et al.  Fitness Distance Correlation Analysis: An Instructive Counterexample , 1997, ICGA.

[19]  Thomas Jansen,et al.  Performance analysis of randomised search heuristics operating with a fixed budget , 2014, Theor. Comput. Sci..

[20]  James M. Crawford,et al.  Experimental Results on the Crossover Point in Random 3-SAT , 1996, Artif. Intell..

[21]  Jeanette P. Schmidt,et al.  Component structure in the evolution of random hypergraphs , 1985, Comb..

[22]  Thomas Jansen,et al.  Analyzing Evolutionary Algorithms: The Computer Science Perspective , 2012 .

[23]  Michael Molloy,et al.  Cores in random hypergraphs and Boolean formulas , 2005, Random Struct. Algorithms.

[24]  Allan Sly,et al.  Proof of the Satisfiability Conjecture for Large k , 2014, STOC.

[25]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[26]  Carsten Witt,et al.  UNIVERSITY OF DORTMUND REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods Worst-Case and Average-Case Approximations by Simple Randomized Search Heuristics , 2004 .

[27]  Benjamin Doerr,et al.  Multiplicative drift analysis , 2010, GECCO.

[28]  Gabriel Istrate,et al.  CRITICAL BEHAVIOR IN THE SATISFIABILITY OF RANDOM K-HORN FORMULAE , 2007 .

[29]  Federico Ricci-Tersenghi,et al.  On the solution-space geometry of random constraint satisfaction problems , 2006, STOC '06.

[30]  M. Mitzenmacher Tight Thresholds for The Pure Literal Rule , 1997 .

[31]  Yuren Zhou,et al.  Exponential bounds for the random walk algorithm on random planted 3-SAT , 2012, Science China Information Sciences.

[32]  Berthold Vöcking,et al.  Worst Case and Probabilistic Analysis of the 2-Opt Algorithm for the TSP , 2007, SODA '07.

[33]  Eli Ben-Sasson,et al.  Linear Upper Bounds for Random Walk on Small Density Random 3-CNFs , 2007, SIAM J. Comput..

[34]  Thomas Jansen,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods on Classifications of Fitness Functions on Classifications of Fitness Functions , 2022 .

[35]  Michael Krivelevich,et al.  Solving random satisfiable 3CNF formulas in expected polynomial time , 2006, SODA '06.

[36]  Hector J. Levesque,et al.  Hard and Easy Distributions of SAT Problems , 1992, AAAI.

[37]  Dimitris Achlioptas,et al.  Random Satisfiability , 2009, Handbook of Satisfiability.

[38]  Dirk Sudholt,et al.  A fixed budget analysis of randomized search heuristics for the traveling salesperson problem , 2014, GECCO.

[39]  Dan Gutfreund,et al.  Finding a randomly planted assigment in a random 3CNF , 2002 .

[40]  Anne Auger,et al.  Theory of Randomized Search Heuristics: Foundations and Recent Developments , 2011, Theory of Randomized Search Heuristics.

[41]  Andrei A. Bulatov,et al.  Phase Transition for Local Search on Planted SAT , 2008, MFCS.

[42]  Thomas Jansen,et al.  On the analysis of the (1+1) evolutionary algorithm , 2002, Theor. Comput. Sci..

[43]  Frank Neumann,et al.  Bioinspired computation in combinatorial optimization: algorithms and their computational complexity , 2012, GECCO '12.

[44]  Alan M. Frieze,et al.  Analyzing Walksat on Random Formulas , 2011, ANALCO.

[45]  Frank Neumann,et al.  Improved Runtime Bounds for the (1+1) EA on Random 3-CNF Formulas Based on Fitness-Distance Correlation , 2015, GECCO.

[46]  Frank Neumann,et al.  Theoretical analysis of two ACO approaches for the traveling salesman problem , 2011, Swarm Intelligence.

[47]  Tobias Storch,et al.  Finding large cliques in sparse semi-random graphs by simple randomized search heuristics , 2007, Theor. Comput. Sci..

[48]  Dirk Sudholt,et al.  When do evolutionary algorithms optimize separable functions in parallel? , 2013, FOGA XII '13.