Testing MCMC algorithms with randomly generated Bayesian networks

In this work we show how to generate random Bayesian networks and how to test inference algorithms using these samples. First, we present a new method to generate random networks through Markov chains. We then use random networks to investigate the performance of quasi-random numbers in Gibbs sampling algorithms for inference. We present experimental results and describe code that implements our methods.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[3]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[4]  Fabio Gagliardi Cozman,et al.  Embedded Bayesian Networks: Anyspace, Anytime Probabilistic Inference , 2002 .

[5]  Jian Cheng,et al.  Computational Investigation of Low-Discrepancy Sequences in Simulation Algorithms for Bayesian Networks , 2000, UAI.

[6]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[7]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[8]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[9]  J. Wendelberger.  Adventures in Stochastic Processes , 1993 .

[10]  H. Faure.  Discrépance de suites associées à un système de numération (en dimension s) , 1982 .

[11]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[12]  Takuji Nishimura,et al.  Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator , 1998, TOMC.

[13]  J. E. H. Shaw,et al.  A Quasirandom Approach to Integration in Bayesian Statistics , 1988 .

[14]  J. Liao,et al.  Variance Reduction in Gibbs Sampler Using Quasi Random Numbers , 1998 .

[15]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[16]  Melantjong.  Random Generation Of Dags For Graph Drawing , 2000 .

[17]  Arnold J. Stromberg,et al.  Number-theoretic Methods in Statistics , 1996 .

[18]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[19]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[20]  Michael I. Jordan,et al.  Variational probabilistic inference and the QMR-DT database , 1998 .

[21]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[22]  I. A. Antonov,et al.  An economic method of computing LPτ-sequences , 1979 .

[23]  Bennett L. Fox,et al.  Algorithm 647: Implementation and Relative Efficiency of Quasirandom Sequence Generators , 1986, TOMS.

[24]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[25]  I. Sobol.  On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[26]  G. Melançon,et al.  Random generation of dags for graph drawing , 2000 .

[27]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Tristan Miller,et al.  A Well-behaved Algorithm for Simulating Dependence Structures of Bayesian Networks , 1999 .

[29]  Paul Bratley,et al.  Algorithm 659: Implementing Sobol's quasirandom sequence generator , 1988, TOMS.

[30]  Eric Horvitz,et al.  Bounded Conditioning: Flexible Inference for Decisions under Scarce Resources , 2013, UAI 1989.

[31]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[32]  Fabio Gagliardi Cozman,et al.  Random Generation of Bayesian Networks , 2002, SBIA.