Studies in Lower Bounding Probabilities of Evidence using the Markov Inequality

Computing the probability of evidence even with known error bounds is NP-hard. In this paper we address this hard problem by settling on an easier problem. We propose an approximation which provides high confidence lower bounds on probability of evidence but does not have any guarantees in terms of relative or absolute error. Our proposed approximation is a randomized importance sampling scheme that uses the Markov inequality. However, a straight-forward application of the Markov inequality may lead to poor lower bounds. We therefore propose several heuristic measures to improve its performance in practice. Empirical evaluation of our scheme with state-of-the-art lower bounding schemes reveals the promise of our approach.

[1]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[2]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[3]  Harold M. Kaplan A Method of One-sided Nonparametric Inference for the Mean of a Nonnegative Population , 1987 .

[4]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[5]  Gregory M. Provan,et al.  Knowledge Engineering for Large Belief Networks , 1994, UAI.

[6]  Michael Luby,et al.  An Optimal Approximation Algorithm for Bayesian Inference , 1997, Artif. Intell..

[7]  Serafín Moral,et al.  A Monte Carlo algorithm for probabilistic propagation in belief networks based on importance sampling and stratified simulation techniques , 1998, Int. J. Approx. Reason..

[8]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..

[9]  Jian Cheng,et al.  Sampling algorithms for estimating the mean of bounded random variables , 2001, Comput. Stat..

[10]  Rina Dechter,et al.  Mini-buckets: A general scheme for bounded inference , 2003, JACM.

[11]  Dan Geiger,et al.  Optimizing exact genetic linkage computations , 2003, RECOMB '03.

[12]  Hilbert J. Kappen,et al.  Bound Propagation , 2003, J. Artif. Intell. Res..

[13]  Rina Dechter,et al.  Mixtures of Deterministic-Probabilistic Networks and their AND/OR Search Space , 2004, UAI.

[14]  Niklas Een,et al.  MiniSat v1.13 - A SAT Solver with Conflict-Clause Minimization , 2005 .

[15]  Vibhav Gogate,et al.  Approximate Inference Algorithms for Hybrid Bayesian Networks with Discrete Constraints , 2005, UAI.

[16]  Rina Dechter,et al.  Improving Bound Propagation , 2006, ECAI.

[17]  Changhe Yuan,et al.  Importance sampling algorithms for Bayesian networks: Principles and performance , 2006, Math. Comput. Model..

[18]  Rina Dechter,et al.  An Anytime Scheme for Bounding Posterior Beliefs , 2006, AAAI.

[19]  Vibhav Gogate,et al.  SampleSearch: A Scheme that Searches for Consistent Samples , 2007, AISTATS.

[20]  Jörg Hoffmann,et al.  From Sampling to Model Counting , 2007, IJCAI.

[21]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..