Harnessing the Bethe free energy†

ABSTRACT A wide class of problems in combinatorics, computer science and physics can be described along the following lines. There are a large number of variables ranging over a finite domain that interact through constraints that each bind a few variables and either encourage or discourage certain value combinations. Examples include the k‐SAT problem or the Ising model. Such models naturally induce a Gibbs measure on the set of assignments, which is characterised by its partition function. The present paper deals with the partition function of problems where the interactions between variables and constraints are induced by a sparse random (hyper)graph. According to physics predictions, a generic recipe called the “replica symmetric cavity method” yields the correct value of the partition function if the underlying model enjoys certain properties [Krzkala et al., PNAS (2007) 10318–10323]. Guided by this conjecture, we prove general sufficient conditions for the success of the cavity method. The proofs are based on a “regularity lemma” for probability measures on sets of the form Ωn for a finite Ω and a large n that may be of independent interest. © 2016 Wiley Periodicals, Inc. Random Struct. Alg., 49, 694–741, 2016

[1]  Charles Bordenave,et al.  Large deviations of empirical neighborhood distribution in sparse random graphs , 2013, 1308.5725.

[2]  Alan M. Frieze,et al.  Random k-Sat: A Tight Threshold For Moderately Growing k , 2005, Comb..

[3]  P. Erdös.  Some remarks on the theory of graphs , 1947 .

[4]  Will Perkins,et al.  Limits of discrete distributions and Gibbs measures on random graphs , 2015, Eur. J. Comb..

[5]  Allan Sly,et al.  The Computational Hardness of Counting in Two-Spin Models on d-Regular Graphs , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[6]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[7]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[8]  Amin Coja-Oghlan,et al.  A positive temperature phase transition in random hypergraph 2-coloring , 2014, ArXiv.

[9]  Yuval Peres,et al.  On the maximum satisfiability of random formulas , 2007, JACM.

[10]  F. Guerra.  Broken Replica Symmetry Bounds in the Mean Field Spin Glass Model , 2002, cond-mat/0205123.

[11]  Assaf Naor,et al.  The two possible values of the chromatic number of a random graph , 2004, STOC '04.

[12]  Rick Durrett,et al.  Probability: Theory and Examples, 4th Edition , 2010 .

[13]  Cristopher Moore,et al.  Random k-SAT: Two Moments Suffice to Cross a Sharp Threshold , 2003, SIAM J. Comput..

[14]  Avi Wigderson,et al.  2-source dispersers for sub-polynomial entropy and Ramsey graphs beating the Frankl-Wilson construction , 2006, STOC '06.

[15]  Nicholas C. Wormald,et al.  Almost All Regular Graphs Are Hamiltonian , 1994, Random Struct. Algorithms.

[16]  Andrea Montanari,et al.  Gibbs states and the set of solutions of random constraint satisfaction problems , 2006, Proceedings of the National Academy of Sciences.

[17]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[18]  Terence Tao.  Szemerédi's regularity lemma revisited , 2006, Contributions Discret. Math..

[19]  H. Bethe.  Statistical Theory of Superlattices , 1935 .

[20]  Elchanan Mossel,et al.  On the hardness of sampling independent sets beyond the tree threshold , 2007, math/0701471.

[21]  Amin Coja-Oghlan,et al.  The condensation phase transition in the regular k-SAT model , 2016, APPROX-RANDOM.

[22]  A. Leaf.  GRAPH THEORY AND PROBABILITY , 1957 .

[23]  D. Panchenko.  The Sherrington-Kirkpatrick Model , 2013 .

[24]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[25]  F. Hollander,et al.  The parabolic Anderson model in a dynamic random environment: space-time ergodicity for the quenched Lyapunov exponent , 2013, Probability Theory and Related Fields.

[26]  P. Turán.  On a Theorem of Hardy and Ramanujan , 1934 .

[27]  D. Gamarnik,et al.  Counting without sampling: Asymptotics of the log-partition function for certain statistical physics models , 2008 .

[28]  David Gamarnik,et al.  Combinatorial approach to the interpolation method and scaling limits in sparse random graphs , 2010, STOC '10.

[29]  Allan Sly,et al.  Communications in Mathematical Physics The Replica Symmetric Solution for Potts Models on d-Regular Graphs , 2022 .

[30]  Assaf Naor,et al.  Rigorous location of phase transitions in hard optimization problems , 2005, Nature.

[31]  D. Achlioptas.  The Threshold for Random kSAT is 2 k ( ln 2 + o ( 1 ) ) , 2007 .

[32]  Allan Sly,et al.  Proof of the Satisfiability Conjecture for Large k , 2014, STOC.

[33]  D. Aldous.  Representations for partially exchangeable arrays of random variables , 1981 .

[34]  Yuval Peres,et al.  The threshold for random k-SAT is 2k (ln 2 - O(k)) , 2003, STOC '03.

[35]  S. Kak.  Information, physics, and computation , 1996 .

[36]  Allan Sly,et al.  Satisfiability Threshold for Random Regular nae-sat , 2013, Communications in Mathematical Physics.

[37]  M. Talagrand,et al.  Bounds for diluted mean-fields spin glass models , 2004, math/0405357.

[38]  L. Rüschendorf,et al.  A general limit theorem for recursive algorithms and combinatorial structures , 2004 .

[39]  Yuval Peres,et al.  On the maximum satisfiability of random formulas , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[40]  Lenka Zdeborová,et al.  The condensation transition in random hypergraph 2-coloring , 2011, SODA.

[41]  Martin Bálek,et al.  Lovász , Large Networks and Graph Limits , 2013 .

[42]  Elchanan Mossel,et al.  Reconstruction and estimation in the planted partition model , 2012, Probability Theory and Related Fields.

[43]  Amin Coja-Oghlan,et al.  Algorithmic Barriers from Phase Transitions , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[44]  Amin Coja-Oghlan,et al.  The Condensation Phase Transition in Random Graph Coloring , 2014, APPROX-RANDOM.

[45]  Andrea Montanari,et al.  Counting good truth assignments of random k-SAT formulae , 2006, SODA '07.

[46]  Jaroslav Nešetřil,et al.  A Combinatorial Classic — Sparse Graphs with High Chromatic Number , 2013 .

[47]  Andrea Montanari,et al.  Factor models on locally tree-like graphs , 2011, ArXiv.

[48]  D. Panchenko.  SPIN GLASS MODELS FROM THE POINT OF VIEW OF SPIN DISTRIBUTIONS , 2010, 1005.2720.

[49]  Will Perkins,et al.  Belief Propagation on Replica Symmetric Random Factor Graph Models , 2016, APPROX-RANDOM.

[50]  Svante Janson,et al.  Random Regular Graphs: Asymptotic Distributions and Contiguity , 1995, Combinatorics, Probability and Computing.

[51]  Eric Vigoda,et al.  Inapproximability for antiferromagnetic spin systems in the tree non-uniqueness region , 2013, STOC.

[52]  Michele Leone,et al.  Replica Bounds for Optimization Problems and Diluted Spin Systems , 2002 .

[53]  A. Dembo,et al.  Ising models on locally tree-like graphs , 2008, 0804.4726.

[54]  J. Michael Steele,et al.  The Objective Method: Probabilistic Combinatorial Optimization and Local Weak Convergence , 2004 .

[55]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[56]  R. Durrett.  Probability: Theory and Examples , 1993 .

[57]  V. Bapst,et al.  The Condensation Phase Transition in Random Graph Coloring , 2016 .

[58]  David Gamarnik,et al.  Counting without sampling: Asymptotics of the log‐partition function for certain statistical physics models , 2008, Random Struct. Algorithms.

[59]  E. Szemerédi.  Regular Partitions of Graphs , 1975 .

[60]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[61]  László Lovász,et al.  Large Networks and Graph Limits , 2012, Colloquium Publications.

[62]  P. Contucci,et al.  Antiferromagnetic Potts Model on the Erdős-Rényi Random Graph , 2011, 1106.4714.

[63]  Allan Sly,et al.  Maximum independent sets on random regular graphs , 2013, 1310.4787.

[64]  Amin Coja-Oghlan,et al.  The asymptotic k-SAT threshold , 2014, STOC.

[65]  Allan Sly,et al.  Satisfiability Threshold for Random Regular nae-sat , 2016 .

[66]  Amin Coja-Oghlan,et al.  Harnessing the Bethe Free Energy , 2015, APPROX-RANDOM.

[67]  F. Papangelou.  GIBBS MEASURES AND PHASE TRANSITIONS (de Gruyter Studies in Mathematics 9) , 1990 .

[68]  Hans-Otto Georgii,et al.  Gibbs Measures and Phase Transitions , 1988 .