Locally Conditioned Belief Propagation

Conditioned Belief Propagation (CBP) is an algorithm for approximate inference in probabilistic graphical models. It works by conditioning on a subset of variables and solving the remainder using loopy Belief Propagation. Unfortunately, CBP's runtime scales exponentially in the number of conditioned variables. Locally Conditioned Belief Propagation (LCBP) approximates the results of CBP by treating conditions locally, and in this way avoids the exponential blow-up. We formulate LCBP as a variational optimization problem and derive a set of update equations that can be used to solve it. We show empirically that LCBP delivers results that are close to those obtained from CBP, while the computational cost scales favorably with problem size.

[1]  Joao Marques-Silva,et al.  Empirical Study of the Anatomy of Modern Sat Solvers , 2011, SAT.

[2]  Toby Walsh,et al.  Handbook of satisfiability , 2009 .

[3]  Josiane Zerubia,et al.  A Hierarchical Markov Random Field Model and Multitemperature Annealing for Parallel Image Classification , 1996, CVGIP Graph. Model. Image Process..

[4]  Cesare Tinelli,et al.  Handbook of Satisfiability , 2021, Handbook of Satisfiability.

[5]  Rina Dechter,et al.  Enhancement Schemes for Constraint Processing: Backjumping, Learning, and Cutset Decomposition , 1990, Artif. Intell..

[6]  Vibhav Gogate,et al.  SampleSearch: Importance sampling in presence of determinism , 2011, Artif. Intell..

[7]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[8]  Guillaume Bouchard,et al.  Split variational inference , 2009, ICML '09.

[9]  Toniann Pitassi,et al.  Combining Component Caching and Clause Learning for Effective Model Counting , 2004, SAT.

[10]  Thomas P. Minka,et al.  Divergence measures and message passing , 2005 .

[11]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[12]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[13]  Adnan Darwiche,et al.  DPLL with a Trace: From SAT to Knowledge Compilation , 2005, IJCAI.

[14]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[15]  Susanne Biundo-Stephan,et al.  Conditioned Belief Propagation Revisited , 2014, ECAI.

[16]  Toniann Pitassi,et al.  Value Elimination: Bayesian Interence via Backtracking Search , 2002, UAI.

[17]  Roberto J. Bayardo,et al.  Counting Models Using Connected Components , 2000, AAAI/IAAI.

[18]  Joao Marques-Silva,et al.  GRASP-A new search algorithm for satisfiability , 1996, Proceedings of International Conference on Computer Aided Design.

[19]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[20]  Max Welling,et al.  On the Choice of Regions for Generalized Belief Propagation , 2004, UAI.

[21]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[22]  Ian McGraw,et al.  Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing , 2006, UAI.

[23]  Thomas P. Minka,et al.  Gates , 2008, NIPS.

[24]  A. Montanari,et al.  How to compute loop corrections to the Bethe approximation , 2005, cond-mat/0506769.

[25]  Adnan Darwiche,et al.  Recursive conditioning , 2001, Artif. Intell..

[26]  Rina Dechter,et al.  Cutset Sampling for Bayesian Networks , 2011, J. Artif. Intell. Res..

[27]  Thomas Geier,et al.  Conditioned Belief Propagation revisited (extended version) , 2014 .

[28]  David Declercq,et al.  A novel region graph construction based on trapping sets for the Generalized Belief Propagation , 2012, 2012 IEEE International Conference on Communication Systems (ICCS).

[29]  T. Jaakkola,et al.  Improving the Mean Field Approximation Via the Use of Mixture Distributions , 1999, Learning in Graphical Models.

[30]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[31]  Hilbert J. Kappen,et al.  Sufficient Conditions for Convergence of the Sum–Product Algorithm , 2005, IEEE Transactions on Information Theory.

[32]  Zoubin Ghahramani,et al.  Choosing a Variable to Clamp: Approximate Inference Using Conditioned Belief Propagation , 2009 .

[33]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..