On the solution-space geometry of random constraint satisfaction problems

For a number of random constraint satisfaction problems, such as random k-SAT and random graph/hypergraph coloring, there are very good estimates of the largest constraint density for which solutions exist. Yet, all known polynomial-time algorithms for these problems fail to find solutions even at much lower densities. To understand the origin of this gap we study how the structure of the space of solutions evolves in such problems as constraints are added. In particular, we prove that much before solutions disappear, they organize into an exponential number of clusters, each of which is relatively small and far apart from all other clusters. Moreover, inside each cluster most variables are frozen, i.e., take only one value. The existence of such frozen variables gives a satisfying intuitive explanation for the failure of the polynomial-time algorithms analyzed so far. At the same time, our results establish rigorously one of the two main hypotheses underlying Survey Propagation, a heuristic introduced by physicists in recent years that appears to perform extraordinarily well on random constraint satisfaction problems.

[1]  R. Paley,et al.  A note on analytic functions in the unit circle , 1932, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  Miklós Simonovits,et al.  Supersaturated graphs and hypergraphs , 1983, Comb..

[3]  Ming-Te Chao,et al.  Probabilistic Analysis of Two Heuristics for the 3-Satisfiability Problem , 1986, SIAM J. Comput..

[4]  Bruce A. Reed,et al.  Mick gets some (the odds are on his side) (satisfiability) , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[5]  Tail Bounds for Occupancy and the Satisfiability Threshold Conjecture , 1995, Random Struct. Algorithms.

[6]  N. Wormald.  Differential Equations for Random Processes and Random Graphs , 1995 .

[7]  Alan M. Frieze,et al.  Analysis of Two Simple Heuristics on a Random Instance of k-SAT , 1996, J. Algorithms.

[8]  Yacine Boufkhad,et al.  A General Upper Bound for the Satisfiability Threshold of Random r-SAT Formulae , 1997, J. Algorithms.

[9]  L. Kirousis,et al.  Approximating the unsatisfiability threshold of random formulas , 1998, Random Struct. Algorithms.

[10]  E. Friedgut,et al.  Sharp thresholds of graph properties, and the -sat problem , 1999 .

[11]  Robert J. McEliece,et al.  The generalized distributive law , 2000, IEEE Trans. Inf. Theory.

[12]  Olivier Dubois,et al.  Typical random 3-SAT formulae and the satisfiability threshold , 2000, SODA '00.

[13]  Cristopher Moore,et al.  The asymptotic order of the random k-SAT threshold , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[14]  Lefteris M. Kirousis,et al.  The probabilistic analysis of a greedy satisfiability algorithm , 2002, Random Struct. Algorithms.

[15]  M. Mézard,et al.  Random K-satisfiability problem: from an analytic solution to an efficient algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[17]  Michael Alekhnovich,et al.  Linear upper bounds for random walk on small density random 3-CNFs , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[18]  Riccardo Zecchina,et al.  Survey propagation as local equilibrium equations , 2003, ArXiv.

[19]  Lefteris M. Kirousis,et al.  Selecting Complementary Pairs of Literals , 2003, Electron. Notes Discret. Math..

[20]  Giorgio Parisi.  Some remarks on the survey decimation algorithm for K-satisfiability , 2003, ArXiv.

[21]  Yuval Peres,et al.  The threshold for random k-SAT is 2k (ln 2 - O(k)) , 2003, STOC '03.

[22]  Assaf Naor,et al.  The two possible values of the chromatic number of a random graph , 2004, STOC '04.

[23]  Dimitris Achlioptas,et al.  THE THRESHOLD FOR RANDOM k-SAT IS 2k log 2 O(k) , 2004, FOCS 2004.

[24]  M. Mézard,et al.  Pairs of SAT Assignments and Clustering in Random Boolean Formulae , 2005 .

[25]  Thierry Mora,et al.  Pairs of SAT Assignment in Random Boolean Formulae , 2005, ArXiv.

[26]  Thierry Mora,et al.  Clustering of solutions in the random satisfiability problem , 2005, Physical review letters.

[27]  Assaf Naor,et al.  Rigorous location of phase transitions in hard optimization problems , 2005, Nature.

[28]  Martin J. Wainwright,et al.  A new look at survey propagation and its generalizations , 2004, SODA '05.

[29]  E. Friedgut.  Hunting for sharp thresholds , 2005 .

[30]  Efthimios G. Lalas,et al.  The probabilistic analysis of a greedy satisfiability algorithm , 2006 .

[31]  D. Achlioptas.  The Threshold for Random kSAT is 2 k ( ln 2 + o ( 1 ) ) , 2007 .

[32]  Eli Ben-Sasson,et al.  Linear Upper Bounds for Random Walk on Small Density Random 3-CNFs , 2007, SIAM J. Comput..

[33]  Andrea Montanari,et al.  Gibbs states and the set of solutions of random constraint satisfaction problems , 2006, Proceedings of the National Academy of Sciences.

[34]  Pekka Orponen,et al.  Circumspect descent prevails in solving random constraint satisfaction problems , 2007, Proceedings of the National Academy of Sciences.

[35]  Amin Coja-Oghlan,et al.  Algorithmic Barriers from Phase Transitions , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[36]  Andrea Montanari,et al.  Clusters of solutions and replica symmetry breaking in random k-satisfiability , 2008, ArXiv.

[37]  Lenka Zdeborová,et al.  Exhaustive enumeration unveils clustering and freezing in random 3-SAT , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[38]  Riccardo Zecchina,et al.  Entropy landscape and non-Gibbs solutions in constraint satisfaction problems , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  Thierry Mora,et al.  Pairs of SAT-assignments in random Boolean formulæ , 2005, Theor. Comput. Sci..