On probabilistic inference by weighted model counting

A recent and effective approach to probabilistic inference calls for reducing the problem to one of weighted model counting (WMC) on a propositional knowledge base. Specifically, the approach calls for encoding the probabilistic model, typically a Bayesian network, as a propositional knowledge base in conjunctive normal form (CNF) with weights associated to each model according to the network parameters. Given this CNF, computing the probability of some evidence becomes a matter of summing the weights of all CNF models consistent with the evidence. A number of variations on this approach have appeared in the literature recently, that vary across three orthogonal dimensions. The first dimension concerns the specific encoding used to convert a Bayesian network into a CNF. The second dimensions relates to whether weighted model counting is performed using a search algorithm on the CNF, or by compiling the CNF into a structure that renders WMC a polytime operation in the size of the compiled structure. The third dimension deals with the specific properties of network parameters (local structure) which are captured in the CNF encoding. In this paper, we discuss recent work in this area across the above three dimensions, and demonstrate empirically its practical importance in significantly expanding the reach of exact probabilistic inference. We restrict our discussion to exact inference and model counting, even though other proposals have been extended for approximate inference and approximate model counting.

[1]  Rina Dechter,et al.  Bayesian Inference in the Presence of Determinism , 2003, AISTATS.

[2]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[3]  Adnan Darwiche,et al.  Compiling Bayesian Networks with Local Structure , 2005, IJCAI.

[4]  David Allen,et al.  Exploiting Evidence in Probabilistic Inference , 2005, UAI.

[5]  Adnan Darwiche,et al.  Recursive conditioning , 2001, Artif. Intell..

[6]  Frank Jensen,et al.  Approximations in Bayesian Belief Universe for Knowledge Based Systems , 2013, UAI 1990.

[7]  Ross D. Shachter Evidence Absorption and Propagation through Evidence Reversals , 2013, UAI.

[8]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[9]  T K Gaylord,et al.  Logical minimization of multilevel coded functions. , 1986, Applied optics.

[10]  Toniann Pitassi,et al.  Value Elimination: Bayesian Interence via Backtracking Search , 2002, UAI.

[11]  Nevin Lianwen Zhang,et al.  Exploiting Causal Independence in Bayesian Network Inference , 1996, J. Artif. Intell. Res..

[12]  Roberto J. Bayardo,et al.  Counting Models Using Connected Components , 2000, AAAI/IAAI.

[13]  Bart Selman,et al.  Towards Efficient Sampling: Exploiting Random Walk Strategies , 2004, AAAI.

[14]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[15]  Adnan Darwiche,et al.  On the Tractable Counting of Theory Models and its Application to Truth Maintenance and Belief Revision , 2001, J. Appl. Non Class. Logics.

[16]  Adnan Darwiche,et al.  A differential semantics for jointree algorithms , 2002, Artif. Intell..

[17]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[18]  Toniann Pitassi,et al.  Algorithms and complexity results for #SAT and Bayesian inference , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[19]  Adnan Darwiche,et al.  A compiler for deterministic, decomposable negation normal form , 2002, AAAI/IAAI.

[20]  Adnan Darwiche,et al.  Sensitivity Analysis in Bayesian Networks: From Single to Multiple Parameters , 2004, UAI.

[21]  Manfred Jaeger,et al.  Compiling relational Bayesian networks for exact inference , 2006, Int. J. Approx. Reason..

[22]  Adnan Darwiche,et al.  New Advances in Compiling CNF into Decomposable Negation Normal Form , 2004, ECAI.

[23]  Pierre Marquis,et al.  A Knowledge Compilation Map , 2002, J. Artif. Intell. Res..

[24]  Adnan Darwiche,et al.  A Logical Approach to Factoring Belief Networks , 2002, KR.

[25]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[26]  Rina Dechter,et al.  Bucket elimination: A unifying framework for probabilistic inference , 1996, UAI.

[27]  Adnan Darwiche,et al.  Compiling Bayesian Networks Using Variable Elimination , 2007, IJCAI.

[28]  Bart Selman,et al.  Exploiting random walk strategies in automated reasoning , 2005 .

[29]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[30]  Rina Dechter,et al.  AND/OR search spaces for graphical models , 2007, Artif. Intell..

[31]  Nevin Lianwen Zhang,et al.  Exploiting Contextual Independence In Probabilistic Inference , 2011, J. Artif. Intell. Res..

[32]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[33]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[34]  Dan Geiger,et al.  Exact genetic linkage computations for general pedigrees , 2002, ISMB.

[35]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[36]  David Heckerman,et al.  A Tractable Inference Algorithm for Diagnosing Multiple Diseases , 2013, UAI.

[37]  Adnan Darwiche,et al.  Solving MAP Exactly using Systematic Search , 2002, UAI.

[38]  Dan Geiger,et al.  Maximum Likelihood Haplotyping for General Pedigrees , 2005, Human Heredity.

[39]  Ross D. Shachter Evaluating Influence Diagrams , 1986, Oper. Res..

[40]  John P. Hayes,et al.  Introduction to Digital Logic Design , 1993 .

[41]  Bart Selman,et al.  A New Approach to Model Counting , 2005, SAT.

[42]  Toniann Pitassi,et al.  DPLL with Caching: A new algorithm for #SAT and Bayesian Inference , 2003, Electron. Colloquium Comput. Complex..

[43]  Henry A. Kautz,et al.  Heuristics for Fast Exact Model Counting , 2005, SAT.

[44]  Adnan Darwiche,et al.  Encoding CNFs to Empower Component Analysis , 2006, SAT.

[45]  Changhe Yuan,et al.  Annealed MAP , 2004, UAI.

[46]  Henry A. Kautz,et al.  Solving Bayesian Networks by Weighted Model Counting , 2005 .

[47]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[48]  Adnan Darwiche,et al.  DPLL with a Trace: From SAT to Knowledge Compilation , 2005, IJCAI.

[49]  Nir Friedman,et al.  Proceedings of the 15th annual conference on uncertainty in artificial intelligence (UAI) , 1999 .

[50]  Adnan Darwiche,et al.  Approximating MAP using Local Search , 2001, UAI.

[51]  Toniann Pitassi,et al.  Combining Component Caching and Clause Learning for Effective Model Counting , 2004, SAT.