Estimating the ROC Curve of Linearly Combined Dichotomizers

A well established technique to improve the classification performances is to combine more classifiers. In the binary case, an effective instrument to analyze the dichotomizers under different class and cost distributions providing a description of their performances at different operating points is the Receiver Operating Characteristic (ROC) curve. To generate a ROC curve, the outputs of the dichotomizers have to be processed. An alternative way that makes this analysis more tractable with mathematical tools is to use a parametric model and, in particular, the binormal model that gives a good approximation to many empirical ROC curves. Starting from this model, we propose a method to estimate the ROC curve of the linear combination of two dichotomizers given the ROC curves of the single classifiers. A possible application of this approach has been successfully tested on real data set.

[1]  C. Metz,et al.  Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. , 1998, Statistics in medicine.

[2]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[3]  C. Metz ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[4]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  M. Pepe The Statistical Evaluation of Medical Tests for Classification and Prediction , 2003 .

[6]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[7]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[8]  Tom Fawcett,et al.  Robust Classification for Imprecise Environments , 2000, Machine Learning.

[9]  Barak A. Pearlmutter,et al.  Differentiating Functions of the Jacobian with Respect to the Weights , 1999, NIPS.

[10]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[11]  Mehryar Mohri,et al.  AUC Optimization vs. Error Rate Minimization , 2003, NIPS.

[12]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[13]  C. Metz,et al.  "Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation. , 1999, Journal of mathematical psychology.

[14]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[15]  Francesco Tortorella,et al.  A ROC-based reject rule for dichotomizers , 2005, Pattern Recognit. Lett..

[16]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Data Mining Researchers , 2003 .