Twin SVM with a reject option through ROC curve

Abstract This paper proposes a new method which embeds a reject option in twin support vector machine (RO-TWSVM) through the Receiver Operating Characteristic (ROC) curve for binary classification. The proposed RO-TWSVM enhances the classification robustness through inclusion of an effective rejection rule for potentially misclassified samples. The method is formulated based on a cost-sensitive framework which follows the principle of minimization of the expected cost of classification. Extensive experiments are conducted on synthetic and real-world data sets to compare the proposed RO-TWSVM with the original TWSVM without a reject option (TWSVM-without-RO) and the existing SVM with a reject option (RO-SVM). The experimental results demonstrate that our RO-TWSVM significantly outperforms TWSVM-without-RO, and in general, performs better than RO-SVM.

[1]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[2]  Carla M. Santos-Pereira,et al.  On optimal reject rules and ROC curves , 2005, Pattern Recognit. Lett..

[3]  Dimitris N. Metaxas,et al.  RO-SVM: Support Vector Machine with Reject Option for Image Categorization , 2006, BMVC.

[4]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[5]  Yves Grandvalet,et al.  Support Vector Machines with a Reject Option , 2008, NIPS.

[6]  Ivan Flores,et al.  An Optimum Character Recognition System Using Decision Functions , 1958, IRE Trans. Electron. Comput..

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  Kar-Ann Toh,et al.  Between Classification-Error Approximation and Weighted Least-Squares Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Francesco Tortorella,et al.  Reducing the classification cost of support vector classifiers through an ROC-based reject rule , 2004, Pattern Analysis and Applications.

[10]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[11]  Kar-Ann Toh,et al.  Exploiting the relationships among several binary classifiers via data transformation , 2014, Pattern Recognit..

[12]  Robert Sabourin,et al.  The Multiclass ROC Front method for cost-sensitive classification , 2016, Pattern Recognit..

[13]  Francesco Tortorella,et al.  A ROC-based reject rule for dichotomizers , 2005, Pattern Recognit. Lett..

[14]  Ran El-Yaniv,et al.  On the Foundations of Noise-free Selective Classification , 2010, J. Mach. Learn. Res..

[15]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..

[16]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[17]  Adel Belouchrani,et al.  Heartbeat Classification Using Support Vector Machines (SVMs) with an Embedded Reject Option , 2012, Int. J. Pattern Recognit. Artif. Intell..

[18]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[19]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[21]  Matteo Golfarelli,et al.  On the Error-Reject Trade-Off in Biometric Verification Systems , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Fabio Roli,et al.  Support Vector Machines with Embedded Reject Option , 2002, SVM.

[24]  Divya Tomar,et al.  A comparison on multi-class classification methods based on least squares twin support vector machine , 2015, Knowl. Based Syst..

[25]  Jean Mercklé,et al.  Optimizing the Classification Cost using SVMs with a Double Hinge Loss , 2012, Informatica.

[26]  Zhiping Lin,et al.  A center sliding Bayesian binary classifier adopting orthogonal polynomials , 2015, Pattern Recognit..

[27]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[28]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[30]  Tadeusz Pietraszek,et al.  On the use of ROC analysis for the optimization of abstaining classifiers , 2007, Machine Learning.

[31]  Fabio Roli,et al.  Multi-label classification with a reject option , 2013, Pattern Recognit..

[32]  C. M. Bishop,et al.  Improvements on Twin Support Vector Machines , 2011 .