Learning decision tree for ranking

Decision tree is one of the most effective and widely used methods for classification. However, many real-world applications require instances to be ranked by the probability of class membership. The area under the receiver operating characteristics curve, simply AUC, has been recently used as a measure for ranking performance of learning algorithms. In this paper, we present two novel class probability estimation algorithms to improve the ranking performance of decision tree. Instead of estimating the probability of class membership using simple voting at the leaf where the test instance falls into, our algorithms use similarity-weighted voting and naive Bayes. We design empirical experiments to verify that our new algorithms significantly outperform the recent decision tree ranking algorithm C4.4 in terms of AUC.

[1]  Harry Zhang,et al.  Learning probabilistic decision trees for AUC , 2006, Pattern Recognit. Lett..

[2]  Ian Witten,et al.  Data Mining , 2000 .

[3]  C. Ling,et al.  Decision Tree with Better Ranking , 2003, ICML.

[4]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[5]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[6]  Peter I. Cowling,et al.  Knowledge and Information Systems , 2006 .

[7]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[8]  Zoubin Ghahramani,et al.  Proceedings of the 24th international conference on Machine learning , 2007, ICML 2007.

[9]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[10]  Liangxiao Jiang,et al.  Instance Cloning Local Naive Bayes , 2005, Canadian Conference on AI.

[11]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[12]  Pedro M. Domingos,et al.  Tree Induction for Probability-Based Ranking , 2003, Machine Learning.

[13]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[14]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[17]  Bianca Zadrozny,et al.  Ranking-based evaluation of regression models , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[18]  Harry Zhang,et al.  Exploring Conditions For The Optimality Of Naïve Bayes , 2005, Int. J. Pattern Recognit. Artif. Intell..