Learning lazy naive Bayesian classifiers for ranking

Naive Bayes (simply NB) has been well-known as an effective and efficient classification algorithm. However, it is based on the conditional independence assumption that it is often violated in applications. In addition, in many real-world data mining applications, however, an accurate ranking of instances is often required rather than an accurate classification. For example, a ranking of customers in terms of the likelihood that they buy one's products is useful in direct marketing. In this paper, we firstly investigate the ranking performance of some lazy learning algorithms for extending naive Bayes. The ranking performance is measured by Hand and Till (2001) and Bradley (1997). We observe that they can not significantly improve naive Bayes' ranking performance. Motivated by this fact and aiming at improving naive Bayes with accurate ranking, we present a new lazy learning algorithm, called lazy naive Bayes (simply LNB), to extend naive Bayes for ranking. We experimentally tested our algorithm, using the whole 36 UCI data sets (Blake and Merz, 2000) recommended by Weka, and compared it to NB and C4.4 (Provost and Domingos, 2003) measured by AUC. The experimental results show that our algorithm significantly outperforms both NB and C4.4

[1]  Mong-Li Lee,et al.  SNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning , 2002, PAKDD.

[2]  David W. Aha,et al.  Lazy Learning , 1997, Springer Netherlands.

[3]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[6]  Pedro M. Domingos,et al.  Tree Induction for Probability-Based Ranking , 2003, Machine Learning.

[7]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[8]  Geoffrey I. Webb,et al.  Lazy Learning of Bayesian Rules , 2000, Machine Learning.

[9]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[10]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[11]  Mehryar Mohri,et al.  AUC Optimization vs. Error Rate Minimization , 2003, NIPS.

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[14]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[15]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[16]  Paul N. Bennett.  Assessing the Calibration of Naive Bayes Posterior Estimates , 2000 .