Inference in belief networks: A procedural guide

Abstract Belief networks are popular tools for encoding uncertainty in expert systems. These networks rely on inference algorithms to compute beliefs in the context of observed evidence. One established method for exact inference on belief networks is the probability propagation in trees of clusters (PPTC) algorithm, as developed by Lauritzen and Spiegelhalter and refined by Jensen et al. PPTC converts the belief network into a secondary structure, then computes probabilities by manipulating the secondary structure. In this document, we provide a self-contained, procedural guide to understanding and implementing PPTC. We synthesize various optimizations to PPTC that are scattered throughout the literature. We articulate undocumented “open secrets” that are vital to producing a robust and efficient implementation of PPTC. We hope that this document makes probabilistic inference more accessible and affordable to those without extensive prior exposure.

[1]  Finn Verner Jensen,et al.  Cautious Propagation in Bayesian Networks , 1995, UAI.

[2]  A. P. Dawid,et al.  Fast retraction of evidence in a probabilistic expert system , 1992 .

[3]  Dan Geiger,et al.  Identifying independence in bayesian networks , 1990, Networks.

[4]  Derek G. Corneil,et al.  Complexity of finding embeddings in a k -tree , 1987 .

[5]  Gregory M. Provan,et al.  Query DAGs: A practical paradigm for implementing belief-network inference , 1996, UAI.

[6]  Zhaoyu Li,et al.  Efficient inference in Bayes networks as a combinatorial optimization problem , 1994, Int. J. Approx. Reason..

[7]  Eric Horvitz,et al.  A Bayesian analysis of simulation algorithms for inference in belief networks , 1993, Networks.

[8]  Eugene Charniak,et al.  Bayesian Networks without Tears , 1991, AI Mag..

[9]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[10]  Frank Jensen,et al.  Optimal junction Trees , 1994, UAI.

[11]  Bruce D'Ambrosio,et al.  Incremental Probabilistic Inference , 1993, UAI.

[12]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[13]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[14]  Frank Jensen,et al.  Approximations in Bayesian Belief Universe for Knowledge Based Systems , 2013, UAI 1990.

[15]  Robert E. Tarjan,et al.  Simple Linear-Time Algorithms to Test Chordality of Graphs, Test Acyclicity of Hypergraphs, and Selectively Reduce Acyclic Hypergraphs , 1984, SIAM J. Comput..

[16]  Judea Pearl,et al.  A Constraint-Propagation Approach to Probabilistic Reasoning , 1985, UAI.

[17]  M. Golummc.  Algorithmic graph theory and perfect graphs , 1980 .

[18]  Ross D. Shachter,et al.  Fusion and Propagation with Multiple Observations in Belief Networks , 1991, Artif. Intell..

[19]  Steffen L. Lauritzen,et al.  Independence properties of directed markov fields , 1990, Networks.

[20]  Ross D. Shachter,et al.  Symbolic Probabilistic Inference in Belief Networks , 1990, AAAI.

[21]  Ross D. Shachter,et al.  Global Conditioning for Probabilistic Inference in Belief Networks , 1994, UAI.

[22]  Steffen L. Lauritzen,et al.  Bayesian updating in causal probabilistic networks by local computations , 1990 .

[23]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[24]  Uue Kjjrull.  Triangulation of Graphs { Algorithms Giving Small Total State Space Triangulation of Graphs { Algorithms Giving Small Total State Space , 1990 .

[25]  Adnan Darwiche,et al.  Conditioning Algorithms for Exact and Approximate Inference in Causal Networks , 1995, UAI.

[26]  M. Golumbic.  Algorithmic graph theory and perfect graphs , 1980 .

[27]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[28]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .