Probabilistic outputs for twin support vector machines

In many cases, the output of a classifier should be a calibrated posterior probability to enable post-processing. However, twin support vector machines (TWSVM) do not provide such probabilities. In this paper, we propose a TWSVM probability model, called PTWSVM, to estimate the posterior probability. Note that our model is quite different from the SVM probability model because of the different mechanism of TWSVM and SVM. In fact, for TWSVM, we first define a new ranking continues output by comparing the distances between the sample and the two non-parallel hyperplanes, and then map this ranking continues output into probability by training the parameters of an additional sigmoid function. Our PTWSVM has been tested on both artificial datasets and several data-mining-style datasets, and the numerical experiments indicate that PTWSVM yields nice results.

[1]  Fei-Yue Wang,et al.  Posterior probability support vector Machines for unbalanced data , 2005, IEEE Transactions on Neural Networks.

[2]  Anirban Mukherjee,et al.  Newton's method for nonparallel plane proximal classifier with unity norm hyperplanes , 2010, Signal Process..

[3]  Yuan-Hai Shao,et al.  A novel margin-based twin support vector machine with unity norm hyperplanes , 2012, Neural Computing and Applications.

[4]  G. Wahba.  Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[5]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[6]  Bianca Zadrozny,et al.  Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers , 2001, ICML.

[7]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[8]  Anirban Mukherjee,et al.  Cancer Classification from Gene Expression Data by NPPC Ensemble , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[9]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[10]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[12]  Joseph Picone,et al.  Applications of support vector machines to speech recognition , 2004, IEEE Transactions on Signal Processing.

[13]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[14]  G. Shafer,et al.  Algorithmic Learning in a Random World , 2005 .

[15]  Johan A. K. Suykens,et al.  Multiclass LS-SVMs: Moderated Outputs and Coding-Decoding Schemes , 2002, Neural Processing Letters.

[16]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[17]  Ganesh R. Naik,et al.  Twin SVM for Gesture Classification Using the Surface Electromyogram , 2010, IEEE Transactions on Information Technology in Biomedicine.

[18]  Chunxia Zhao,et al.  Localized twin SVM via convex minimization , 2011, Neurocomputing.

[19]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[20]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Zhiquan Qi,et al.  Online multiple instance boosting for object detection , 2011, Neurocomputing.

[22]  Dominik Olszewski,et al.  A probabilistic approach to fraud detection in telecommunications , 2012, Knowl. Based Syst..

[23]  Hsuan-Tien Lin,et al.  A note on Platt’s probabilistic outputs for support vector machines , 2007, Machine Learning.

[24]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[25]  Michael I. Jordan,et al.  Consistent probabilistic outputs for protein function prediction , 2008, Genome Biology.

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Chih-Fong Tsai,et al.  Simple instance selection for bankruptcy prediction , 2012, Knowl. Based Syst..

[28]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[29]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[30]  Yong Tang,et al.  Learning to rank with document ranks and scores , 2011, Knowl. Based Syst..

[31]  Gang Wang,et al.  A novel bankruptcy prediction model based on an adaptive fuzzy k-nearest neighbor method , 2011, Knowl. Based Syst..

[32]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[33]  David G. Stork,et al.  Pattern Classification , 1973 .

[34]  Yuan-Hai Shao,et al.  A coordinate descent margin based-twin support vector machine for classification , 2012, Neural Networks.

[35]  Chong Jin Ong,et al.  Feature selection via sensitivity analysis of SVM probabilistic outputs , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[36]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[37]  Chih-Jen Lin,et al.  Generalized Bradley-Terry Models and Multi-Class Probability Estimates , 2006, J. Mach. Learn. Res..