On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms

We introduce a version of the cavity method for diluted mean-field spin models that allows the computation of thermodynamic quantities similar to the Franz–Parisi quenched potential in sparse random graph models. This method is developed in the particular case of partially decimated random constraint satisfaction problems. This allows us to develop a theoretical understanding of a class of algorithms for solving constraint satisfaction problems, in which elementary degrees of freedom are sequentially assigned according to the results of a message passing procedure (belief propagation). We confront this theoretical analysis with the results of extensive numerical simulations.

[1]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[2]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[3]  Franco P. Preparata.  Theoretical computer science , 2001 .

[4]  M. Mézard,et al.  Survey-propagation decimation through distributed local computations , 2005, cond-mat/0512002.

[5]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[6]  Alan L. Yuille,et al.  CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.

[7]  Dimitris Achlioptas,et al.  THE THRESHOLD FOR RANDOM k-SAT IS 2k log 2 O(k) , 2004, FOCS 2004.

[8]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[9]  Alan M. Frieze,et al.  Analysis of Two Simple Heuristics on a Random Instance of k-SAT , 1996, J. Algorithms.

[10]  Mi Heggie,et al.  Journal of Physics: Conference Series: Preface , 2011 .

[11]  P. Gács,et al.  Algorithms , 1992 .

[12]  Olivier Dubois,et al.  Upper bounds on the satisfiability threshold , 2001, Theor. Comput. Sci..

[13]  M. Mézard,et al.  Reconstruction on Trees and Spin Glass Transition , 2005, cond-mat/0512295.

[14]  B. M. Fulk.  MATH , 1992 .

[15]  G. Parisi,et al.  Recipes for metastable states in spin glasses , 1995 .

[16]  Lenka Zdeborová,et al.  Constraint satisfaction problems with isolated solutions are hard , 2008, ArXiv.

[17]  M. Pretti.  A message-passing algorithm with damping , 2005 .

[18]  Rémi Monasson,et al.  Relationship between clustering and algorithmic phase transitions in the random k-XORSAT model and its NP-complete extensions , 2007, ArXiv.

[19]  Martin J. Wainwright,et al.  A new look at survey propagation and its generalizations , 2004, SODA '05.

[20]  Dan Suciu,et al.  Journal of the ACM , 2006 .

[21]  John V. Franco.  Results related to threshold phenomena research in satisfiability: lower bounds , 2001, Theor. Comput. Sci..

[22]  Rémi Monasson,et al.  Can rare SAT formulas be easily recognized? On the efficiency of message passing algorithms for K-SAT at large clause-to-variable ratios , 2006, ArXiv.

[23]  Florent Krzakala,et al.  Phase Transitions in the Coloring of Random Graphs , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  M. Mézard,et al.  Random K-satisfiability problem: from an analytic solution to an efficient algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  R. Monasson,et al.  Rigorous decimation-based construction of ground pure states for spin-glass models on random lattices. , 2002, Physical review letters.

[26]  R. Monasson,et al.  Relaxation and metastability in a local search procedure for the random satisfiability problem. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  Rémi Monasson,et al.  Criticality and universality in the unit-propagation search rule , 2005, ArXiv.

[28]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[29]  Dimitris Achlioptas,et al.  Lower bounds for random 3-SAT via differential equations , 2001, Theor. Comput. Sci..

[30]  M. Mézard,et al.  Information, Physics, and Computation , 2009 .

[31]  Hilary Putnam,et al.  A Computing Procedure for Quantification Theory , 1960, JACM.

[32]  M. Mézard,et al.  The Bethe lattice spin glass revisited , 2000, cond-mat/0009418.

[33]  Alexander K. Hartmann,et al.  Solving satisfiability problems by fluctuations: The dynamics of stochastic local search algorithms , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  Pekka Orponen,et al.  Focused local search for random 3-satisfiability , 2005, ArXiv.

[35]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[36]  E. Friedgut,et al.  Sharp thresholds of graph properties, and the -sat problem , 1999 .

[37]  R. Zecchina,et al.  Complexity transitions in global algorithms for sparse linear systems over finite fields , 2002, cond-mat/0203613.

[38]  T. Kurtz.  Solutions of ordinary differential equations as limits of pure jump markov processes , 1970, Journal of Applied Probability.

[39]  J. van Leeuwen,et al.  Theoretical Computer Science , 2003, Lecture Notes in Computer Science.

[40]  Thomas G. Dietterich,et al.  In Advances in Neural Information Processing Systems 12 , 1991, NIPS 1991.

[41]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[42]  M. M..  Algæ , 2022 .

[43]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[44]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[45]  Costas S. Iliopoulos,et al.  Proceedings of the International Conference on Analysis of Algorithms , 2007 .

[46]  M. Talagrand,et al.  Spin Glasses: A Challenge for Mathematicians , 2003 .

[47]  Rémi Monasson,et al.  THE EUROPEAN PHYSICAL JOURNAL B c○ EDP Sciences , 1999 .

[48]  A. Montanari,et al.  How to compute loop corrections to the Bethe approximation , 2005, cond-mat/0506769.

[49]  Andrea Montanari,et al.  Clusters of solutions and replica symmetry breaking in random k-satisfiability , 2008, ArXiv.

[50]  M. Mézard,et al.  Two Solutions to Diluted p-Spin Models and XORSAT Problems , 2003 .

[51]  Andrea Montanari,et al.  Gibbs states and the set of solutions of random constraint satisfaction problems , 2006, Proceedings of the National Academy of Sciences.