Pairwise Optimization of Bayesian Classifiers for Multi-class Cost-Sensitive Learning

In this paper, we present a new approach to enhance the performance of Bayesian classifiers. Our method relies on the combination of two ideas: pairwise classification on the one hand, and threshold optimization on the other hand. Introducing one threshold per pair of classes increases the expressivity of the model, therefore its performance on complex problems such as cost-sensitive problems increases as well. Indeed a comparison of our algorithm to other cost-sensitive approaches shows that it reduces the total misclassification cost.

[1]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[2]  Robert P. W. Duin,et al.  Approximating the multiclass ROC by pairwise analysis , 2007, Pattern Recognit. Lett..

[3]  Peter D. Turney.  Types of Cost in Inductive Concept Learning , 2002, ArXiv.

[4]  Ulf Brefeld,et al.  Support Vector Machines with Example Dependent Costs , 2003, ECML.

[5]  Victor S. Sheng,et al.  Cost-Sensitive Learning , 2009, Encyclopedia of Data Warehousing and Mining.

[6]  Nicolas Lachiche.  Receiver Operating Characteristic (ROC) Analysis , 2009, Encyclopedia of Data Warehousing and Mining.

[7]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[8]  John Langford,et al.  Cost-sensitive learning by cost-proportionate example weighting , 2003, Third IEEE International Conference on Data Mining.

[9]  Bianca Zadrozny,et al.  Learning and making decisions when costs and probabilities are both unknown , 2001, KDD '01.

[10]  Laura Maruster,et al.  Encyclopedia of data warehousing and mining , 2008 .

[11]  Eyke Hüllermeier,et al.  On Pairwise Naive Bayes Classifiers , 2007, ECML.

[12]  Robert E. Schapire,et al.  On reoptimizing multi-class classifiers , 2008, Machine Learning.

[13]  Johannes Fürnkranz,et al.  Round Robin Classification , 2002, J. Mach. Learn. Res..

[14]  Victor S. Sheng,et al.  Cost-Sensitive Learning , 2009, Encyclopedia of Data Warehousing and Mining.

[15]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[16]  Peter A. Flach,et al.  Improving Accuracy and Cost of Two-class and Multi-class Probabilistic Classifiers Using ROC Curves , 2003, ICML.

[17]  H. Keselman,et al.  Multiple Comparison Procedures , 2005 .