Measuring the Hardness of Stochastic Sampling on Bayesian Networks with Deterministic Causalities: the k-Test

Approximate Bayesian inference is NP-hard. Dagum and Luby defined the Local Variance Bound (LVB) to measure the approximation hardness of Bayesian inference on Bayesian networks, assuming the networks model strictly positive joint probability distributions, i.e. zero probabilities are not permitted. This paper introduces the k-test to measure the approximation hardness of inference on Bayesian networks with deterministic causalities in the probability distribution, i.e. when zero conditional probabilities are permitted. Approximation by stochastic sampling is a widely-used inference method that is known to suffer from inefficiencies due to sample rejection. The k-test predicts when rejection rates of stochastic sampling a Bayesian network will be low, modest, high, or when sampling is intractable.

[1]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[2]  Cristopher Moore,et al.  Random k-SAT: Two Moments Suffice to Cross a Sharp Threshold , 2003, SIAM J. Comput..

[3]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  Leonard M. Adleman,et al.  Two theorems on random polynomial time , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[6]  Changhe Yuan,et al.  An Importance Sampling Algorithm Based on Evidence Pre-propagation , 2002, UAI.

[7]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[8]  John Franco,et al.  Probabilistic analysis of the Davis Putnam procedure for solving the satisfiability problem , 1983, Discret. Appl. Math..

[9]  D. Heckerman,et al.  Toward Normative Expert Systems: Part I The Pathfinder Project , 1992, Methods of Information in Medicine.

[10]  Geir Storvik,et al.  Simulation and Monte Carlo Methods , 2006 .

[11]  Victor R. Lesser,et al.  A survey of research in deliberative real-time artificial intelligence , 1994, Real-Time Systems.

[12]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[13]  Vibhav Gogate,et al.  Approximate Counting by Sampling the Backtrack-free Search Space , 2007, AAAI.

[14]  Finn Verner Jensen,et al.  MUNIN: an expert EMG assistant , 1988 .

[15]  Cristina Conati,et al.  On-Line Student Modeling for Coached Problem Solving Using Bayesian Networks , 1997 .

[16]  Vibhav Gogate,et al.  A New Algorithm for Sampling CSP Solutions Uniformly at Random , 2006, CP.

[17]  Amin Coja-Oghlan.  A Better Algorithm for Random k-SAT , 2010, SIAM J. Comput..

[18]  Robert A. van Engelen,et al.  Refractor Importance Sampling , 2008, UAI.

[19]  Michael Sipser,et al.  Halting space-bounded computations , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[20]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[21]  Michael Luby,et al.  An Optimal Approximation Algorithm for Bayesian Inference , 1997, Artif. Intell..

[22]  Amin Coja-Oghlan.  A Better Algorithm for Random k-SAT , 2009, ICALP.

[23]  Vibhav Gogate,et al.  SampleSearch: Importance sampling in presence of determinism , 2011, Artif. Intell..

[24]  Gregory M. Provan,et al.  Knowledge Engineering for Large Belief Networks , 1994, UAI.

[25]  Rina Dechter,et al.  Mixtures of Deterministic-Probabilistic Networks and their AND/OR Search Space , 2004, UAI.

[26]  William H. Hsu,et al.  A Survey of Algorithms for Real-Time Bayesian Network Inference , 2002 .

[27]  Bruce A. Reed,et al.  Mick gets some (the odds are on his side) (satisfiability) , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[28]  E. Friedgut,et al.  Sharp thresholds of graph properties, and the -sat problem , 1999 .

[29]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[30]  Changhe Yuan,et al.  Theoretical analysis and practical insights on importance sampling in Bayesian networks , 2007, Int. J. Approx. Reason..

[31]  Vibhav Gogate,et al.  Approximate Inference Algorithms for Hybrid Bayesian Networks with Discrete Constraints , 2005, UAI.

[32]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..

[33]  Ming-Te Chao,et al.  Probabilistic analysis of a generalization of the unit-clause literal selection heuristics for the k satisfiability problem , 1990, Inf. Sci..

[34]  Rina Dechter,et al.  Mixed deterministic and probabilistic networks , 2008, Annals of Mathematics and Artificial Intelligence.

[35]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[36]  Serafín Moral,et al.  Dynamic importance sampling in Bayesian networks based on probability trees , 2005, Int. J. Approx. Reason..

[37]  Marco Gavanelli.  The Log-Support Encoding of CSP into SAT , 2007, CP.