Scalable inference for hybrid Bayesian networks with full density estimations

The simplest hybrid Bayesian network is Conditional Linear Gaussian (CLG). It is a hybrid model for which exact inference can be performed by the Junction Tree (JT) algorithm. However, the traditional JT only provides the exact first two moments for hidden continuous variables. In general, the complexity of exact inference algorithms is exponential in the size of the largest clique of the strongly triangulated graph that is usually the one including all of discrete parent nodes for a connected continuous component in the model. Furthermore, for the general nonlinear non-Gaussian hybrid model, it is well-known that no exact inference is possible. This paper introduces a new inference approach by unifying message passing between different types of variables. This algorithm is able to provide an exact solution for polytree CLG, and approximate solution by loopy propagation for general hybrid models. To overcome the exponential complexity, we use Gaussian mixture reduction methods to approximate the original density and make the algorithm scalable. This new algorithm provides not only the first two moments, but full density estimates. Empirically, approximation errors due to reduced Gaussian mixtures and loopy propagation are relatively small, especially for nodes that are far away from the discrete parent nodes. Numerical experiments show encouraging results.

[1]  Changhe Yuan,et al.  Hybrid Loopy Belief Propagation , 2006, Probabilistic Graphical Models.

[2]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  Uwe D. Hanebeck,et al.  A New Approach for Hybrid Bayesian Networks Using Full Densities , 2004 .

[5]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[6]  S. Lauritzen.  Propagation of Probabilities, Means, and Variances in Mixed Graphical Association Models , 1992 .

[7]  William H. Hsu,et al.  A Survey of Algorithms for Real-Time Bayesian Network Inference , 2002 .

[8]  O. C. Schrempf,et al.  Optimal mixture approximation of the product of mixtures , 2005, 2005 7th International Conference on Information Fusion.

[9]  Kuo-Chu Chang,et al.  Unscented Message Passing for Arbitrary Continuous Variables in Bayesian Networks , 2007, AAAI.

[10]  Uri Lerner,et al.  Hybrid Bayesian networks for reasoning about complex systems , 2002 .

[11]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[12]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[13]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[14]  Kuo-Chu Chang,et al.  Hybrid message passing for mixed bayesian networks , 2007, 2007 10th International Conference on Information Fusion.

[15]  Steffen L. Lauritzen,et al.  Stable local computation with conditional Gaussian distributions , 2001, Stat. Comput..

[16]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[17]  Prakash P. Shenoy,et al.  Inference in hybrid Bayesian networks with mixtures of truncated exponentials , 2006, Int. J. Approx. Reason..

[18]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[19]  Wei Sun,et al.  Message Passing for Hybrid Bayesian Networks: Representation, Propagation, and Integration , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[20]  Hd D. Chen,et al.  Constraint optimized weight adaptation for Gaussian mixture reduction , 2010, Defense + Commercial Sensing.

[21]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .