Approximate counting, the Lovasz local lemma, and inference in graphical models

In this paper we introduce a new approach for approximately counting in bounded degree systems with higher-order constraints. Our main result is an algorithm to approximately count the number of solutions to a CNF formula Ф when the width is logarithmic in the maximum degree. This closes an exponential gap between the known upper and lower bounds. Moreover our algorithm extends straightforwardly to approximate sampling, which shows that under Lovász Local Lemma-like conditions it is not only possible to find a satisfying assignment, it is also possible to generate one approximately uniformly at random from the set of all satisfying assignments. Our approach is a significant departure from earlier techniques in approximate counting, and is based on a framework to bootstrap an oracle for computing marginal probabilities on individual variables. Finally, we give an application of our results to show that it is algorithmically possible to sample from the posterior distribution in an interesting class of graphical models.

[1]  Dror Weitz,et al.  Counting independent sets up to the tree threshold , 2006, STOC '06.

[2]  Michael Luby,et al.  An Optimal Approximation Algorithm for Bayesian Inference , 1997, Artif. Intell..

[3]  Mark Jerrum,et al.  Uniform sampling through the Lovasz local lemma , 2016, STOC.

[4]  Dimitris Achlioptas,et al.  Random Walks That Find Perfect Objects and the Lovasz Local Lemma , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[5]  Martin E. Dyer,et al.  Stopping Times, Metrics and Approximate Counting , 2006, ICALP.

[6]  Pinyan Lu,et al.  FPTAS for Counting Monotone CNF , 2015, SODA.

[7]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[8]  Allan Sly,et al.  Computational Transition at the Uniqueness Threshold , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[9]  Jan Vondrák,et al.  Computing the independence polynomial in Shearer's region for the LLL , 2016, ArXiv.

[10]  Leslie G. Valiant,et al.  Random Generation of Combinatorial Structures from a Uniform Distribution , 1986, Theor. Comput. Sci..

[11]  P. Erdos-L Lovász.  Problems and Results on 3-chromatic Hypergraphs and Some Related Questions , 2022 .

[12]  Prasad Tetali,et al.  The correlation decay (CD) tree and strong spatial mixing in multi-spin systems , 2007 .

[13]  Ivona Bezáková,et al.  Approximation via Correlation Decay when Strong Spatial Mixing Fails , 2015, ICALP.

[14]  Gábor Tardos,et al.  A constructive proof of the general lovász local lemma , 2009, JACM.

[15]  Donald E. Knuth,et al.  The art of computer programming: V.1.: Fundamental algorithms , 1997 .

[16]  József Beck,et al.  An Algorithmic Approach to the Lovász Local Lemma. I , 1991, Random Struct. Algorithms.

[17]  Robin A. Moser.  A constructive proof of the Lovász local lemma , 2008, STOC '09.

[18]  Eric Vigoda,et al.  Inapproximability of the Partition Function for the Antiferromagnetic Ising and Hard-Core Models , 2012, Combinatorics, Probability and Computing.

[19]  Allan Sly,et al.  Counting in two-spin models on d-regular graphs , 2014 .

[20]  Elchanan Mossel,et al.  Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms , 2007, SIAM J. Comput..

[21]  Mark Jerrum,et al.  Approximate Counting, Uniform Generation and Rapidly Mixing Markov Chains , 1987, WG.

[22]  Piyush Srivastava,et al.  Approximation Algorithms for Two-State Anti-Ferromagnetic Spin Systems on Bounded Degree Graphs , 2011, Journal of Statistical Physics.

[23]  Jan Vondrák,et al.  An Algorithmic Proof of the Lovasz Local Lemma via Resampling Oracles , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[24]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[25]  Aravind Srinivasan,et al.  A constructive algorithm for the Lovász Local Lemma on permutations , 2014, SODA.

[26]  Vladimir Kolmogorov,et al.  Commutativity in the Algorithmic Lovász Local Lemma , 2015, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[27]  Noga Alon,et al.  A parallel algorithmic version of the local lemma , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[28]  Donald E. Knuth,et al.  The Art of Computer Programming, Volume I: Fundamental Algorithms, 2nd Edition , 1997 .

[29]  Jan Vondrák,et al.  Computing the Independence Polynomial: from the Tree Threshold down to the Roots , 2016, SODA.

[30]  Aravind Srinivasan,et al.  New Constructive Aspects of the Lovasz Local Lemma , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[31]  Dmitriy Katz,et al.  Correlation decay and deterministic FPTAS for counting colorings of a graph , 2012, J. Discrete Algorithms.

[32]  JerrumMark,et al.  Uniform Sampling Through the Lovász Local Lemma , 2019 .