A Novel Twin Support-Vector Machine With Pinball Loss

Twin support-vector machine (TSVM), which generates two nonparallel hyperplanes by solving a pair of smaller-sized quadratic programming problems (QPPs) instead of a single larger-sized QPP, works faster than the standard SVM, especially for the large-scale data sets. However, the traditional TSVM adopts hinge loss which easily leads to its sensitivity of the noise and instability for resampling. To enhance the performance of the TSVM, we present a novel TSVM with the pinball loss (Pin-TSVM) which deals with the quantile distance and is less sensitive to noise points. We further investigate its properties, including the noise insensitivity, between-class distance maximization, and within-class scatter minimization. In addition, we compare our Pin-TSVM with the twin parametric-margin SVM and the SVM with the pinball loss in theory. Numerical experiments on a synthetic data set and 14 benchmark data sets with different noises demonstrate the feasibility and validity of our proposed method.

[1]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[2]  Yuanyuan Wang,et al.  A rough margin based support vector machine , 2008, Inf. Sci..

[3]  Johan A. K. Suykens,et al.  Fixed-size Pegasos for hinge and pinball loss SVM , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[4]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[5]  Wenjie Hu,et al.  Robust support vector machine with bullet hole image classification , 2002 .

[6]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[7]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[8]  Anirban Mukherjee,et al.  Newton's method for nonparallel plane proximal classifier with unity norm hyperplanes , 2010, Signal Process..

[9]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[10]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[11]  Ingo Steinwart,et al.  Estimating conditional quantiles with the help of the pinball loss , 2011, 1102.2101.

[12]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[13]  Andreas Christmann,et al.  How SVMs can estimate quantiles and the median , 2007, NIPS.

[14]  Johan A. K. Suykens,et al.  Sequential minimal optimization for SVM with pinball loss , 2015, Neurocomputing.

[15]  Isabelle Guyon,et al.  Discovering Informative Patterns and Data Cleaning , 1996, Advances in Knowledge Discovery and Data Mining.

[16]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[17]  Alexander J. Smola,et al.  Second Order Cone Programming Approaches for Handling Missing and Uncertain Data , 2006, J. Mach. Learn. Res..

[18]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[19]  PengXinjun.  A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms , 2010 .

[20]  Pei-Yi Hao,et al.  New support vector algorithms with parametric insensitive/margin model , 2010, Neural Networks.

[21]  Ping Zhong,et al.  A rough margin-based ν-twin support vector machine , 2011, Neural Computing and Applications.

[22]  Johan A. K. Suykens,et al.  Support Vector Machine Classifier With Pinball Loss , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[24]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[25]  R. Koenker.  Quantile Regression: Name Index , 2005 .

[26]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[28]  Reshma Khemchandani,et al.  Fuzzy multi-category proximal support vector classification via generalized eigenvalues , 2007, Soft Comput..

[29]  Jinbo Bi,et al.  Support Vector Classification with Input Data Uncertainty , 2004, NIPS.

[30]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[31]  Ralf Herbrich,et al.  Adaptive margin support vector machines for classification , 1999 .