Author's Personal Copy Computational Statistics and Data Analysis Learning Bayesian Networks for Discrete Data

Bayesian networks have received much attention in the recent literature. In this article, we propose an approach to learn Bayesian networks using the stochastic approximation Monte Carlo (SAMC) algorithm. Our approach has two nice features. Firstly, it possesses the self-adjusting mechanism and thus avoids essentially the local-trap problem suffered by conventional MCMC simulation-based approaches in learning Bayesian networks. Secondly, it falls into the class of dynamic importance sampling algorithms; the network features can be inferred by dynamically weighted averaging the samples generated in the learning process, and the resulting estimates can have much lower variation than the single model-based estimates. The numerical results indicate that our approach can mix much faster over the space of Bayesian networks than the conventional MCMC simulation-based approaches.

[1]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[2]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[3]  P. Green,et al.  Decomposable graphical Gaussian model determination , 1999 .

[4]  W H Wong,et al.  Dynamic weighting in Monte Carlo and optimization. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[5]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[6]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[7]  Faming Liang,et al.  A Theory for Dynamic Weighting in Monte Carlo Computation , 2001 .

[8]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  F. Liang.  On the use of stochastic approximation Monte Carlo for Monte Carlo integration , 2009 .

[10]  C. S. Wallace,et al.  Learning Linear Causal Models by MML Sampling , 1999 .

[11]  Doug Fisher,et al.  Learning from Data: Artificial Intelligence and Statistics V , 1996 .

[12]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[13]  Tod S. Levitt,et al.  Uncertainty in artificial intelligence , 1988 .

[14]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[15]  Xiao-Lin Li,et al.  Learning Bayesian Networks Structure with Continuous Variables , 2006, ADMA.

[16]  N. Wermuth,et al.  Graphical and recursive models for contingency tables , 1983 .

[17]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[18]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[19]  Alexander Gammerman,et al.  Causal Models and Intelligent Data Management , 1999, Springer Berlin Heidelberg.

[20]  Han-Fu Chen.  Stochastic approximation and its applications , 2002 .

[21]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[22]  Luis M. de Campos,et al.  A new approach for learning belief networks using independence criteria , 2000, Int. J. Approx. Reason..

[23]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[24]  M. D. Martínez-Miranda,et al.  Computational Statistics and Data Analysis , 2009 .

[25]  F. Liang.  Dynamically Weighted Importance Sampling in Monte Carlo Computation , 2002 .

[26]  Krzysztof J. Cios,et al.  CLIP3: Cover learning using integer programming , 1997 .

[27]  Lukasz A. Kurgan,et al.  Knowledge discovery approach to automated cardiac SPECT diagnosis , 2001, Artif. Intell. Medicine.

[28]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[29]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[30]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[31]  R. Bouckaert.  Bayesian belief networks : from construction to inference , 1995 .

[32]  W. Wong,et al.  Learning Causal Bayesian Network Structures From Experimental Data , 2008 .

[33]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[34]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[35]  Gregory F. Cooper,et al.  Causal Discovery from a Mixture of Experimental and Observational Data , 1999, UAI.

[36]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[37]  R. Carroll,et al.  Stochastic Approximation in Monte Carlo Computation , 2007 .

[38]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[39]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..