Learning Semi Naïve Bayes Structures by Estimation of Distribution Algorithms

Recent work in supervised learning has shown that a surprisingly simple Bayesian classifier called naive Bayes is competitive with state of the art classifiers. This simple approach stands from assumptions of conditional independence among features given the class. Improvements in accuracy of naive Bayes has been demonstrated by a number of approaches, collectively named semi naive Bayes classifiers. Semi naive Bayes classifiers are usually based on the search of specific values or structures. The learning process of these classifiers is usually based on greedy search algorithms. In this paper we propose to learn these semi naive Bayes structures through estimation of distribution algorithms, which are non-deterministic, stochastic heuristic search strategies. Experimental tests have been done with 21 data sets from the UCI repository.

[1]  Ron Kohavi,et al.  Improving simple Bayes , 1997 .

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  João Gama,et al.  Iterative Bayes , 2000, Intell. Data Anal..

[5]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[6]  María S. Pérez-Hernández,et al.  Interval Estimation Naïve Bayes , 2003, IDA.

[7]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[8]  Igor Kononenko,et al.  Semi-Naive Bayesian Classifier , 1991, EWSL.

[9]  Ron Kohavi,et al.  MLC++: a machine learning library in C++ , 1994, Proceedings Sixth International Conference on Tools with Artificial Intelligence. TAI 94.

[10]  Pat Langley,et al.  Induction of Recursive Bayesian Classifiers , 1993, ECML.

[11]  Geoffrey I. Webb,et al.  Adjusted Probability Naive Bayesian Induction , 1998, Australian Joint Conference on Artificial Intelligence.

[12]  Pat Langley,et al.  Induction of Selective Bayesian Classifiers , 1994, UAI.

[13]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[14]  Pedro M. Domingos,et al.  Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier , 1996, ICML.

[15]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[16]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[17]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[18]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[19]  Irving John Good,et al.  The Estimation of Probabilities: An Essay on Modern Bayesian Methods , 1965 .

[20]  Víctor Robles,et al.  Interval Estimation Na¨ ive Bayes , 2003 .

[21]  D. Hand,et al.  Idiot's Bayes—Not So Stupid After All? , 2001 .

[22]  Pedro Larrañaga,et al.  Optimization in Continuous Domains by Learning and Simulation of Gaussian Networks , 2000 .

[23]  Michael J. Pazzani,et al.  Searching for Dependencies in Bayesian Classifiers , 1995, AISTATS.

[24]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.