On Combining Graph-based Variance Reduction schemes

In this paper, we consider two variance reduction schemes that exploit the structure of the primal graph of the graphical model: Rao-Blackwellised w-cutset sampling and AND/OR sampling. We show that the two schemes are orthogonal and can be combined to further reduce the variance. Our combination yields a new family of estimators which trade time and space with variance. We demonstrate experimentally that the new estimators are superior, often yielding an order of magnitude improvement over previous schemes on several benchmarks.

[1]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[2]  Rina Dechter,et al.  On finding minimal w-cutset problem , 2004, UAI 2004.

[3]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..

[4]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[5]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[6]  Dan Geiger,et al.  Optimizing exact genetic linkage computations , 2003, RECOMB '03.

[7]  Henry A. Kautz,et al.  Performing Bayesian Inference by Weighted Model Counting , 2005, AAAI.

[8]  Rina Dechter,et al.  Iterative Join-Graph Propagation , 2002, UAI.

[9]  Rina Dechter,et al.  AND/OR Cutset Conditioning , 2005, IJCAI.

[10]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[11]  Jörg Hoffmann,et al.  From Sampling to Model Counting , 2007, IJCAI.

[12]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[13]  Rina Dechter,et al.  On Finding Minimal w-cutset , 2004, UAI.

[14]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[15]  Changhe Yuan,et al.  Importance sampling algorithms for Bayesian networks: Principles and performance , 2006, Math. Comput. Model..

[16]  Vibhav Gogate,et al.  SampleSearch: A Scheme that Searches for Consistent Samples , 2007, AISTATS.

[17]  Vibhav Gogate,et al.  Studies in Lower Bounding Probabilities of Evidence using the Markov Inequality , 2007, UAI.

[18]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[19]  Vibhav Gogate,et al.  AND/OR Importance Sampling , 2008, UAI.

[20]  Rina Dechter,et al.  AND/OR search spaces for graphical models , 2007, Artif. Intell..

[21]  Rina Dechter,et al.  Cutset Sampling for Bayesian Networks , 2011, J. Artif. Intell. Res..

[22]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[23]  Vibhav Gogate,et al.  Join-Graph Propagation Algorithms , 2010, J. Artif. Intell. Res..

[24]  Vibhav Gogate,et al.  Approximate Inference Algorithms for Hybrid Bayesian Networks with Discrete Constraints , 2005, UAI.

[25]  Rina Dechter,et al.  Enhancement Schemes for Constraint Processing: Backjumping, Learning, and Cutset Decomposition , 1990, Artif. Intell..