A Distance-Based Mutation Operator for learning Bayesian Network structures using Evolutionary Algorithms

Variable Orderings (VOs) have been used as a restriction in the process of Bayesian Networks (BNs) induction. The VO information can significantly reduce the search space and allow some algorithms to reach good results. Previous works reported in the literature suggest that the combination of Evolutionary Algorithms (EAs) and VOs is worth when learning a Bayesian Network structure from data. However, most works on this area do not explore specific characteristics of the domain, thus, they simply apply classic evolutionary operators. In addition, most works did not report good results when applied to big BNs. This paper proposes a new mutation operator, named Distance-Based Mutation Operator (DMO), to be used with the Variable Ordering Evolutionary Algorithm (VOEA). Experimental results obtained by VOEA are compared to ones achieved by VOGA (Variable Ordering Genetic Algorithm), and indicated improvement in the quality of the obtained VO and in the BN induced structure.

[1]  Xue-wen Chen,et al.  Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm , 2008, IEEE Transactions on Knowledge and Data Engineering.

[2]  Marek J. Druzdzel,et al.  SMILE: Structural Modeling, Inference, and Learning Engine and GeNIE: A Development Environment for Graphical Decision-Theoretic Models , 1999, AAAI/IAAI.

[3]  William H. Hsu,et al.  Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning , 2004, Inf. Sci..

[4]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[5]  William H. Hsu,et al.  A Permutation Genetic Algorithm For Variable Ordering In Learning Bayesian Networks From Data , 2002, GECCO.

[6]  Eli Faulkner,et al.  K2GA: Heuristically Guided Evolution of Bayesian Network Structures from Data , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[7]  J. S. Maritz.  Distribution-Free Statistical Methods , 1981 .

[8]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[9]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[10]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[11]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[12]  Estevam R. Hruschka,et al.  Conditional independence based learning of bayesian classifiers guided by a variable ordering genetic search , 2007, 2007 IEEE Congress on Evolutionary Computation.

[13]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  Estevam R. Hruschka,et al.  VOGA: Variable Ordering Genetic Algorithm for Learning Bayesian Classifiers , 2006, 2006 Sixth International Conference on Hybrid Intelligent Systems (HIS'06).

[16]  Pedro Larrañaga,et al.  Learning Bayesian Networks In The Space Of Orderings With Estimation Of Distribution Algorithms , 2004, Int. J. Pattern Recognit. Artif. Intell..

[17]  Estevam R. Hruschka,et al.  Variable Ordering for Bayesian Networks Learning from Data , 2003 .

[18]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.