Approximate Inference on Planar Graphs using Loop Calculus and Belief Propagation

We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006b) allows to express the exact partition function Z of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable. We develop an algorithm for the approach presented in Chertkov et al. (2008) which represents an efficient truncation scheme on planar graphs and a new representation of the series in terms of Pfaffians of matrices. We analyze in detail both the loop series and the Pfaffian series for models with binary variables and pairwise interactions, and show that the first term of the Pfaffian series can provide very accurate approximations. The algorithm outperforms previous truncation schemes of the loop series and is competitive with other state-of-the-art methods for approximate inference.

[1]  Hilbert J. Kappen,et al.  On the properties of the Bethe approximation and loopy belief propagation on binary networks , 2004 .

[2]  Martin J. Wainwright,et al.  A new class of upper bounds on the log partition function , 2002, IEEE Transactions on Information Theory.

[3]  Brendan J. Frey,et al.  A Revolution: Belief Propagation in Graphs with Cycles , 1997, NIPS.

[4]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[5]  Tommi S. Jaakkola,et al.  Approximate inference using planar graph decomposition , 2006, NIPS.

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  Francesco Maffioli,et al.  on the Computation of Pfaffians , 1994, Discret. Appl. Math..

[8]  Nicol N. Schraudolph,et al.  Efficient Exact Inference in Planar Ising Models , 2008, NIPS.

[9]  Wojciech Rytter,et al.  Fast parallel algorithms for graph matching problems , 1998 .

[10]  M. Fisher.  Statistical Mechanics of Dimers on a Plane Lattice , 1961 .

[11]  Joris M. Mooij,et al.  libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models , 2010, J. Mach. Learn. Res..

[12]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[13]  Ian McGraw,et al.  Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing , 2006, UAI.

[14]  Michael Chertkov,et al.  Loop series for discrete statistical models on graphs , 2006, ArXiv.

[15]  Erik B. Sudderth,et al.  Loop Series and Bethe Variational Bounds in Attractive Graphical Models , 2007, NIPS.

[16]  M. Fisher,et al.  Dimer problem in statistical mechanics-an exact result , 1961 .

[17]  Michael Chertkov,et al.  Belief propagation and loop series on planar graphs , 2008, ArXiv.

[18]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[19]  Vicenç Gómez,et al.  Truncating the Loop Series Expansion for Belief Propagation , 2006, J. Mach. Learn. Res..

[20]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[21]  K. Fukumizu,et al.  Loop series expansion with propagation diagrams , 2008, 0808.1155.

[22]  J. Vondrák,et al.  New algorithm for the Ising problem: partition function for finite lattice graphs. , 2000, Physical review letters.

[23]  Y L Loh,et al.  Efficient algorithm for random-bond ising models in 2D. , 2006, Physical review letters.

[24]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Process. Mag..

[25]  P. W. Kasteleyn.  Dimer Statistics and Phase Transitions , 1963 .

[26]  Michael Chertkov,et al.  Loop Calculus Helps to Improve Belief Propagation and Linear Programming Decodings of Low-Density-Parity-Check Codes , 2006, ArXiv.

[27]  Yuan Qi,et al.  Tree-structured Approximations by Expectation Propagation , 2003, NIPS.

[28]  P. W. Kasteleyn.  The statistics of dimers on a lattice: I. The number of dimer arrangements on a quadratic lattice , 1961 .

[29]  Gilles Schaeffer,et al.  Random sampling of large planar maps and convex polyhedra , 1999, STOC '99.

[30]  Hilbert J. Kappen,et al.  Approximate Inference and Constrained Optimization , 2002, UAI.

[31]  M. Jerrum.  Counting, Sampling and Integrating: Algorithms and Complexity , 2003 .

[32]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[33]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.