Towards a method for automatically evolving bayesian network classifiers

When faced with a new machine learning problem, selecting which classifier is the best to perform the task at hand is a very hard problem. Most solutions proposed in the literature are based on meta-learning, and use meta-data about the problem to recommend an effective algorithm to solve the task. This paper proposes a new approach to this problem: to build an algorithm tailored to the application problem at hand. More specifically, we propose an evolutionary algorithm (EA) to automatically evolve Bayesian Network Classifiers (BNCs). The method receives as input a list of the main components of BNC algorithms, and uses an EA to encode these components. Given an input dataset, the method tests different combinations of components to that specific application domain. The method was tested in 10 UCI datasets, and compared to three classical BNCs and a greedy search algorithm. Results show that the current algorithms can indeed be improved, but that the EA is currently outperformed by the greedy search.

[1]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[2]  Peter Reutemann,et al.  WEKA Manual for Version 3-6-10 , 2008 .

[3]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[4]  Krzysztof J. Cios,et al.  New synthesis of bayesian network classifiers and cardiac spect image interpretation , 1999 .

[5]  Nuno Lourenço,et al.  Evolving evolutionary algorithms , 2012, GECCO '12.

[6]  Gisele L. Pappa,et al.  Automating the Design of Data Mining Algorithms: An Evolutionary Computation Approach , 2009 .

[7]  Russell Greiner,et al.  Learning Bayesian Belief Network Classifiers: Algorithms and System , 2001, Canadian Conference on AI.

[8]  Joe Suzuki,et al.  Learning Bayesian Belief Networks Based on the MDL Principle : An Efficient Algorithm Using the Branch and Bound Technique , 1999 .

[9]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[11]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[12]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[13]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[14]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[15]  Alex Alves Freitas,et al.  A hyper-heuristic evolutionary algorithm for automatically designing decision-tree algorithms , 2012, GECCO '12.

[16]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[17]  Qiang Shen,et al.  Learning Bayesian networks: approaches and issues , 2011, The Knowledge Engineering Review.

[18]  David Heckerman,et al.  Bayesian Networks for Data Mining , 2004, Data Mining and Knowledge Discovery.

[19]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[20]  Krzysztof J. Cios,et al.  Bayesian learning for cardiac SPECT image interpretation , 2002, Artif. Intell. Medicine.

[21]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[22]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[23]  Concha Bielza,et al.  A review on evolutionary algorithms in Bayesian network learning and inference tasks , 2013, Inf. Sci..

[24]  Alex Alves Freitas,et al.  Towards the automatic design of decision tree induction algorithms , 2011, GECCO.

[25]  Ian Witten,et al.  Data Mining , 2000 .

[26]  Estevam R. Hruschka,et al.  Bayesian network classifiers: Beyond classification accuracy , 2011, Intell. Data Anal..

[27]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[28]  Pat Langley,et al.  An Analysis of Bayesian Classifiers , 1992, AAAI.