Naïve bayes variants in classification learning

Naïve Bayesian classifier is one of the most effective and efficient classification algorithms. The elegant simplicity and apparent accuracy of naive Bayes (NB) even when the independence assumption is violated, fosters the on-going interest in the model. This paper discusses issues on NB along with its advantages and disadvantages. We also present an overview of NB variants and provide a categorization of those methods based on four dimensions. These include manipulating the set of attributes, allowing interdependencies, employing local learning and adjusting the probabilities by numeric weights. Examples for each category are discussed based on 18 variants reviewed in this paper.

[1]  Nada Lavrač,et al.  Induction of Decision Trees and Bayesian Classification Applied to Diagnosis of Sport Injuries , 1997, Journal of Medical Systems.

[2]  Zengchang Qin,et al.  Naive Bayes Classification Given Probability Estimation Trees , 2006, 2006 5th International Conference on Machine Learning and Applications (ICMLA'06).

[3]  Hooman Tahayori,et al.  RoughTree A Classifier with Naive-Bayes and Rough Sets Hybrid in Decision Tree Representation , 2007 .

[4]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[5]  Geoffrey I. Webb,et al.  Lazy Learning of Bayesian Rules , 2000, Machine Learning.

[6]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[7]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[8]  Marcin Szczuka,et al.  Rough Sets in KDD , 2005 .

[9]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[10]  Harry Zhang,et al.  Learning weighted naive Bayes with accurate ranking , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[11]  Liangxiao Jiang,et al.  Learning lazy naive Bayesian classifiers for ranking , 2005, 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'05).

[12]  S. S. Iyengar,et al.  Medical Datamining with a New Algorithm for Feature Selection and Naive Bayesian Classifier , 2007, 10th International Conference on Information Technology (ICIT 2007).

[13]  Geoffrey I. Webb,et al.  On Why Discretization Works for Naive-Bayes Classifiers , 2003, Australian Conference on Artificial Intelligence.

[14]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[15]  Mark A. Hall,et al.  A decision tree-based attribute weighting filter for naive Bayes , 2006, Knowl. Based Syst..

[16]  Liangxiao Jiang,et al.  Dynamic K-Nearest-Neighbor Naive Bayes with Attribute Weighted , 2006, FSKD.

[17]  Albert Sutojo,et al.  Concept Mining using Association Rules and Combinatorial Topology , 2007 .

[18]  S. S. Iyengar,et al.  A comparative analysis of discretization methods for Medical Datamining with Naive Bayesian classifier , 2006, 9th International Conference on Information Technology (ICIT'06).

[19]  Sotiris B. Kotsiantis,et al.  Machine learning: a review of classification and combining techniques , 2006, Artificial Intelligence Review.

[20]  Mong-Li Lee,et al.  SNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning , 2002, PAKDD.

[21]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[22]  D. Gunopulos,et al.  Scaling up the Naive Bayesian Classifier : Using Decision Trees for Feature Selection , 2002 .

[23]  Mehran Sahami,et al.  Learning Limited Dependence Bayesian Classifiers , 1996, KDD.

[24]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[25]  Limin Wang,et al.  Combining decision tree and Naive Bayes for classification , 2006, Knowl. Based Syst..

[26]  Bing Liu,et al.  Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data , 2006, Data-Centric Systems and Applications.

[27]  Geoffrey I. Webb,et al.  A Heuristic Lazy Bayesian Rule Algorithm , 2002, AusDM.

[28]  Geoffrey I. Webb,et al.  Not So Naive Bayes: Aggregating One-Dependence Estimators , 2005, Machine Learning.

[29]  Pat Langley,et al.  Induction of Selective Bayesian Classifiers , 1994, UAI.

[30]  Geoffrey I. Webb,et al.  A comparative study of Semi-naive Bayes methods in classification learning , 2005 .

[31]  Geoffrey I. Webb,et al.  Efficient lazy elimination for averaged one-dependence estimators , 2006, ICML.

[32]  Zdzislaw Pawlak,et al.  Rough sets and intelligent data analysis , 2002, Inf. Sci..