A General Algorithm for Approximate Inference and Its Application to Hybrid Bayes Nets

The clique tree algorithm is the standard method for doing inference in Bayesian networks. It works by manipulating clique potentials - distributions over the variables in a clique. While this approach works well for many networks, it is limited by the need to maintain an exact representation of the clique potentials. This paper presents a new unified approach that combines approximate inference and the clique tree algorithm, thereby circumventing this limitation. Many known approximate inference algorithms can be viewed as instances of this approach. The algorithm essentially does clique tree propagation, using approximate inference to estimate the densities in each clique. In many settings, the computation of the approximate clique potential can be done easily using statistical importance sampling. Iterations are used to gradually improve the quality of the estimation.

[1]  Michael I. Jordan,et al.  Computing upper and lower bounds on likelihoods in intractable networks , 1996, UAI.

[2]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[3]  Serafín Moral,et al.  Mixing exact and importance sampling propagation algorithms in dependence graphs , 1997, Int. J. Intell. Syst..

[4]  Stuart J. Russell,et al.  Stochastic simulation algorithms for dynamic probabilistic networks , 1995, UAI.

[5]  Prakash P. Shenoy,et al.  Axioms for probability and belief-function proagation , 1990, UAI.

[6]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[7]  Stuart J. Russell,et al.  The BATmobile: Towards a Bayesian Automated Taxi , 1995, IJCAI.

[8]  Daphne Koller,et al.  Nonuniform Dynamic Discretization in Hybrid Networks , 1997, UAI.

[9]  Uffe Kjærulff.  HUGS: Combining Exact Inference and Gibbs Sampling in junction Trees , 1995, UAI.

[10]  Jitendra Malik,et al.  Automatic Symbolic Traffic Scene Analysis Using Belief Networks , 1994, AAAI.

[11]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[12]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[13]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[14]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[15]  Daphne Koller,et al.  Using Learning for Approximation in Stochastic Processes , 1998, ICML.

[16]  S. Lauritzen.  Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[17]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[18]  Steffen L. Lauritzen,et al.  Hybrid Propagation in Junction Trees , 1994, IPMU.