Two evolutionary methods for learning Bayesian network structures

This paper describes two approaches based on evolutionary algorithms for determining Bayesian networks structures from a database of cases. One major difficulty when tackling the problem of structure learning with evolutionary strategies is to avoid the premature convergence of the population to a local optimum. In this paper, we propose two methods in order to overcome this obstacle. The first method is a hybridization of a genetic algorithm with a tabu search principle whilst the second method consists in the application of a dynamic mutation rate. For both methods, a repair operator based on the mutual information between the variables was defined to ensure the closeness of the genetic operators. Finally, we evaluate the influence of our methods over the search for known networks

[1]  Pedro Larrañaga,et al.  Learning Bayesian networks in the space of structures by estimation of distribution algorithms , 2003, Int. J. Intell. Syst..

[2]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[3]  Pedro Larrañaga,et al.  Learning Bayesian Networks In The Space Of Orderings With Estimation Of Distribution Algorithms , 2004, Int. J. Pattern Recognit. Artif. Intell..

[4]  Kanta Premji Vekaria,et al.  Selective Crossover in Genetic Algorithms: An Empirical Study , 1998, PPSN.

[5]  Uri Lerner,et al.  Exact Inference in Networks with Discrete Children of Continuous Parents , 2001, UAI.

[6]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[7]  Ralph R. Martin,et al.  A Sequential Niche Technique for Multimodal Function Optimization , 1993, Evolutionary Computation.

[8]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[9]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[10]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[11]  Remco R. Bouckaert,et al.  Properties of Bayesian Belief Network Learning Algorithms , 1994, UAI.

[12]  Gerard Lacey,et al.  Context-Aware Shared Control of a Robot Mobility Aid for the Elderly Blind , 2000, Int. J. Robotics Res..

[13]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[14]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[15]  C. Cotta.  On the Learning of Bayesian Network Graph Structures via Evolutionary Programming , 2004 .

[16]  J. York,et al.  Bayesian Graphical Models for Discrete Data , 1995 .

[17]  R. W. Robinson.  Counting unlabeled acyclic digraphs , 1977 .

[18]  Susan T. Dumais,et al.  A Bayesian Approach to Filtering Junk E-Mail , 1998, AAAI 1998.

[19]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  O. Fuentes,et al.  Generic algorithms: what fitness scaling is optimal? , 1993 .

[21]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[22]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[23]  Kenneth de Jong,et al.  Evolutionary computation: a unified approach , 2007, GECCO.

[24]  H. Akaike.  Statistical predictor identification , 1970 .

[25]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[26]  J. Gutiérrez,et al.  Applications of Bayesian Networks in Meteorology , 2004 .

[27]  Michael P. Wellman,et al.  Real-world applications of Bayesian networks , 1995, CACM.

[28]  Haiying Tu,et al.  Detecting, tracking, and counteracting terrorist networks via hidden Markov models , 2004, 2004 IEEE Aerospace Conference Proceedings (IEEE Cat. No.04TH8720).

[29]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[30]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[31]  Dirk Thierens,et al.  On the Use of a Non-redundant Encoding for Learning Bayesian Networks from Data with a GA , 2004, PPSN.

[32]  Philippe Leray,et al.  BNT STRUCTURE LEARNING PACKAGE : Documentation and Experiments , 2004 .

[33]  V. Anne Smith,et al.  Using Bayesian Network Inference Algorithms to Recover Molecular Genetic Regulatory Networks , 2002 .

[34]  Christopher Meek,et al.  Monotone DAG Faithfulness: A Bad Assumption , 2003 .

[35]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[36]  Kwong-Sak Leung,et al.  A Hybrid Data Mining Approach To Discover Bayesian Networks Using Evolutionary Programming , 2002, GECCO.

[37]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[38]  Gary A. Davis,et al.  Bayesian reconstruction of traffic accidents , 2003 .

[39]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[40]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[41]  Dirk Thierens,et al.  A Skeleton-Based Approach to Learning Bayesian Networks from Data , 2003, PKDD.

[42]  Katia Sycara,et al.  Reasons for premature convergence of self-adapting mutation rates , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[43]  Carlos Cotta,et al.  Towards a More Efficient Evolutionary Induction of Bayesian Networks , 2002, PPSN.

[44]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[45]  Kazuo J. Ezawa,et al.  Fraud/Uncollectible Debt Detection Using a Bayesian Network Based Learning System: A Rare Binary Outcome with Mixed Data Structures , 1995, UAI.

[46]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[47]  Pedro Larrañaga,et al.  Analysis of the behaviour of genetic algorithms when learning Bayesian network structure from data , 1997, Pattern Recognit. Lett..

[48]  Gregory F. Cooper,et al.  A Bayesian Network Scoring Metic that Is Based on Globally Uniform Parameter Priors , 2002, UAI.

[49]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Carlos Cotta,et al.  A Primer on the Evolution of Equivalence Classes of Bayesian-Network Structures , 2004, PPSN.

[51]  Carlos Cotta,et al.  A Study on the Evolution of Bayesian Network Graph Structures , 2007 .

[52]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[53]  Eric Horvitz,et al.  The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users , 1998, UAI.

[54]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[55]  R. P. Wishner,et al.  Battlefield awareness via synergistic SAR and MTI exploitation , 1998 .

[56]  José M. Peña,et al.  On Local Optima in Learning Bayesian Networks , 2003, UAI.

[57]  Paul J. Krause,et al.  Learning probabilistic networks , 1999, The Knowledge Engineering Review.

[58]  Prakash P. Shenoy,et al.  Inference in hybrid Bayesian networks with mixtures of truncated exponentials , 2006, Int. J. Approx. Reason..

[59]  David Heckerman,et al.  Causal independence for probability assessment and inference using Bayesian networks , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[60]  Dirk Thierens,et al.  Building a GA from Design Principles for Learning Bayesian Networks , 2003, GECCO.

[61]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2003, J. Mach. Learn. Res..

[62]  Luis M. de Campos,et al.  Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs , 2011, J. Artif. Intell. Res..

[63]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[64]  D. Thierens.  Adaptive mutation rate control schemes in genetic algorithms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[65]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[66]  W. Jaronski.  Use of Bayesian belief networks to help understand online audience , 2001 .

[67]  Joe Suzuki,et al.  Learning Bayesian Belief Networks Based on the Minimum Description Length Principle: An Efficient Algorithm Using the B & B Technique , 1996, ICML.

[68]  Thomas Bäck,et al.  Optimal Mutation Rates in Genetic Search , 1993, ICGA.