Thresholding for Making Classifiers Cost-sensitive

In this paper we propose a very simple, yet general and effective method to make any cost-insensitive classifiers (that can produce probability estimates) cost-sensitive. The method, called Thresholding, selects a proper threshold from training instances according to the misclassification cost. Similar to other cost-sensitive meta-learning methods, Thresholding can convert any existing (and future) costinsensitive learning algorithms and techniques into costsensitive ones. However, by comparing with the existing cost sensitive meta-learning methods and the direct use of the theoretical threshold, Thresholding almost always produces the lowest misclassification cost. Experiments also show that Thresholding has the least sensitivity on the misclassification cost ratio. Thus, it is recommended to use when the difference on misclassification costs is large.

[1]  Chris. Drummond,et al.  C 4 . 5 , Class Imbalance , and Cost Sensitivity : Why Under-Sampling beats OverSampling , 2003 .

[2]  Peter D. Turney.  Types of Cost in Inductive Concept Learning , 2002, ArXiv.

[3]  Robert C. Holte,et al.  Exploiting the Cost (In)sensitivity of Decision Tree Splitting Criteria , 2000, ICML.

[4]  Pedro M. Domingos.  MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[5]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[6]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[7]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[8]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[9]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[10]  Nitesh V. Chawla,et al.  Editorial: special issue on learning from imbalanced data sets , 2004, SKDD.

[11]  Peter D. Turney.  Cost-Sensitive Classification: Empirical Evaluation of a Hybrid Genetic Decision Tree Induction Algorithm , 1994, J. Artif. Intell. Res..

[12]  Foster J. Provost,et al.  Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction , 2003, J. Artif. Intell. Res..

[13]  Russell Greiner,et al.  Budgeted Learning of Naive-Bayes Classifiers , 2003, UAI.

[14]  John Langford,et al.  Cost-sensitive learning by cost-proportionate example weighting , 2003, Third IEEE International Conference on Data Mining.

[15]  P. Bühlmann,et al.  Analyzing Bagging , 2001 .

[16]  Robert C. Holte,et al.  C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling , 2003 .

[17]  Kai Ming Ting,et al.  Inducing Cost-Sensitive Trees via Instance Weighting , 1998, PKDD.

[18]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[19]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[20]  Bianca Zadrozny,et al.  Learning and making decisions when costs and probabilities are both unknown , 2001, KDD '01.

[21]  Ian Witten,et al.  Data Mining , 2000 .

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  Russell Greiner,et al.  Budgeted learning of nailve-bayes classifiers , 2002, UAI 2002.