Importance Sampling on Bayesian Networks with Deterministic Causalities

Importance sampling is a powerful approximate inference technique for Bayesian networks. However, the performance tends to be poor when the network exhibits deterministic causalities. Deterministic causalities yield predictable influences between statistical variables. In other words, only a strict subset of the set of all variable states is permissible to sample. Samples inconsistent with the permissible state space do not contribute to the sum estimate and are effectively rejected during importance sampling. Detecting inconsistent samples is NP-hard, since it amounts to calculating the posterior probability of a sample given some evidence. Several methods have been proposed to cache inconsistent samples to improve sampling efficiency. However, cache-based methods do not effectively exploit overlap in the state patterns generated by determinism in a local network structure to compress state information. In this paper, we propose a new algorithm to reduce the overhead of caching by using an adaptive decision tree to efficiently store and detect inconsistent samples. Experimental results show that the proposed approach outperforms existing methods to sample Bayesian networks with deterministic causalities.

[1]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[2]  Geir Storvik,et al.  Simulation and Monte Carlo Methods , 2006 .

[3]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[4]  Serafín Moral,et al.  Dynamic importance sampling in Bayesian networks based on probability trees , 2005, Int. J. Approx. Reason..

[5]  Changhe Yuan,et al.  Importance Sampling in Bayesian Networks: An Influence-Based Approximation Strategy for Importance Functions , 2005, UAI 2005.

[6]  Changhe Yuan,et al.  An Importance Sampling Algorithm Based on Evidence Pre-propagation , 2002, UAI.

[7]  A. Salmerón,et al.  Importance sampling in Bayesian networks using probability trees , 2000 .

[8]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[9]  Robert A. van Engelen,et al.  Refractor Importance Sampling , 2008, UAI.

[10]  Rina Dechter,et al.  Mixtures of Deterministic-Probabilistic Networks and their AND/OR Search Space , 2004, UAI.

[11]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[12]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..

[13]  Vibhav Gogate,et al.  Approximate Inference Algorithms for Hybrid Bayesian Networks with Discrete Constraints , 2005, UAI.

[14]  D. Poole.  Exploiting Contextual Independence and Approximation in Belief Network Inference , 1997 .

[15]  Rina Dechter,et al.  Mixed deterministic and probabilistic networks , 2008, Annals of Mathematics and Artificial Intelligence.

[16]  D. Heckerman,et al.  Toward Normative Expert Systems: Part I The Pathfinder Project , 1992, Methods of Information in Medicine.

[17]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[18]  Vibhav Gogate,et al.  Approximate Counting by Sampling the Backtrack-free Search Space , 2007, AAAI.

[19]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[20]  Cristina Conati,et al.  On-Line Student Modeling for Coached Problem Solving Using Bayesian Networks , 1997 .

[21]  Changhe Yuan,et al.  Importance sampling algorithms for Bayesian networks: Principles and performance , 2006, Math. Comput. Model..

[22]  Vibhav Gogate,et al.  A New Algorithm for Sampling CSP Solutions Uniformly at Random , 2006, CP.

[23]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[24]  David Poole,et al.  Probabilistic Partial Evaluation: Exploiting Rule Structure in Probabilistic Inference , 1997, IJCAI.

[25]  Vibhav Gogate,et al.  SampleSearch: Importance sampling in presence of determinism , 2011, Artif. Intell..

[26]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[27]  Frédéric Benhamou.  Principles and Practice of Constraint Programming - CP 2006, 12th International Conference, CP 2006, Nantes, France, September 25-29, 2006, Proceedings , 2006, CP.

[28]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.