Optimization of the Area under the ROC Curve

In this paper, we propose a new binary classification algorithm (AUCtron), based on gradient descent learning, that directly optimizes AUC (area under the ROC curve). We compare it with a linear classifier and with AUCsplit proposed. The AUCtron algorithm implicitly considers class prior probabilities in the decision criteria. Our results demonstrated that AUC is a sensitive enough metric that when used in small and imbalanced data sets may lead to a better separation.

[1]  Luis A. Sarabia,et al.  On Pareto-optimal fronts for deciding about sensitivity and specificity in class-modelling problems , 2005 .

[2]  Bhavani Raskutti,et al.  Optimising area under the ROC curve using gradient descent , 2004, ICML.

[3]  Mehryar Mohri,et al.  AUC Optimization vs. Error Rate Minimization , 2003, NIPS.

[4]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[5]  Katharina Morik,et al.  Combining Statistical Learning with a Knowledge-Based Approach - A Case Study in Intensive Care Monitoring , 1999, ICML.

[6]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[7]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[8]  Gustavo E. A. P. A. Batista,et al.  Learning with Skewed Class Distributions , 2002 .

[9]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[10]  Michael C. Mozer,et al.  Prodding the ROC Curve: Constrained Optimization of Classifier Performance , 2001, NIPS.

[11]  Michael C. Mozer,et al.  Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic , 2003, ICML.

[12]  Bhavani Raskutti,et al.  Area Under ROC Optimisation using a Ramp Approximation , 2006, SDM.

[13]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[14]  Yi Lin,et al.  Support Vector Machines for Classification in Nonstandard Situations , 2002, Machine Learning.

[15]  Bernhard Sendhoff,et al.  Generalization Improvement in Multi-Objective Learning , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[16]  Peter A. Flach,et al.  Learning Decision Trees Using the Area Under the ROC Curve , 2002, ICML.