The IMAP Hybrid Method for Learning

This paper presents the I-map hybrid algorithm for select- ing, given a data sample, a linear Gaussian model whose structure is a directed graph. The algorithm performs a local search for a model that meets the following criteria: (1) The Markov blankets in the model should be consistent with dependency information from statistical tests. (2) Minimize the number of edges subject to the first constraint. (3) Maximize a given score function subject to the first two constraints. Our local search is based on Graph Equivalence Search (GES); we also apply the recently developed SIN statistical testing strategy to help avoid local minima. Simulation studies with GES search and the BIC score provide evidence that for nets with 10 or more variables, the hybrid method selects simpler graphs whose structure is closer to the target graph.

[1]  D. Edwards.  Introduction to graphical modelling , 1995 .

[2]  Nir Friedman,et al.  Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm , 1999, UAI.

[3]  Wei Luo,et al.  A new hybrid method for Bayesian network learning With dependency constraints , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[4]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[5]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[6]  Mark W. Schmidt,et al.  Learning Graphical Model Structure Using L1-Regularization Paths , 2007, AAAI.

[7]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[8]  Wei Luo,et al.  Mind Change Optimal Learning of Bayes Net Structure , 2007, COLT.

[9]  Mathias Drton,et al.  A SINful approach to Gaussian graphical model selection , 2005 .

[10]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[11]  Ian Witten,et al.  Data Mining , 2000 .

[12]  Luis M. de Campos,et al.  A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests , 2006, J. Mach. Learn. Res..

[13]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[14]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[15]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[16]  R. Bouckaert.  Bayesian belief networks : from construction to inference , 1995 .

[17]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[18]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[19]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[20]  Russell Greiner,et al.  Model Selection Criteria for Learning Belief Nets: An Empirical Comparison , 2000, ICML.

[21]  C. Meek,et al.  Graphical models: selecting causal and statistical models , 1997 .