Ramp loss nonparallel support vector machine for pattern classification

In this paper, we propose a novel sparse and robust nonparallel hyperplane classifier, named Ramp loss Nonparallel Support Vector Machine (RNPSVM), for binary classification. By introducing the Ramp loss function and also proposing a new non-convex and non-differentiable loss function based on the e-insensitive loss function, RNPSVM can explicitly incorporate noise and outlier suppression in the training process, has less support vectors and the increased sparsity leads to its better scaling properties. The non-convexity of RNPSVM can be efficiently solved by the Concave-Convex Procedure and experimental results on benchmark datasets confirm the effectiveness of the proposed algorithm.

[1]  Yong Shi,et al.  Laplacian twin support vector machine for semi-supervised classification , 2012, Neural Networks.

[2]  Ying-jie Tian,et al.  Improved twin support vector machine , 2013, Science China Mathematics.

[3]  Johan A. K. Suykens,et al.  Ramp loss linear programming support vector machine , 2014, J. Mach. Learn. Res..

[4]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines , 2003, J. Mach. Learn. Res..

[5]  Yong Shi,et al.  Recent advances on support vector machines research , 2012 .

[6]  Yuan-Hai Shao,et al.  A novel margin-based twin support vector machine with unity norm hyperplanes , 2012, Neural Computing and Applications.

[7]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[8]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[9]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[10]  Yong Shi,et al.  Twin support vector machine with Universum data , 2012, Neural Networks.

[11]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[12]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[13]  Zhiquan Qi,et al.  Efficient sparse nonparallel support vector machines for classification , 2014, Neural Computing and Applications.

[14]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent in Function Space , 2007 .

[15]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[16]  Theodore B. Trafalis,et al.  Robust classification and regression using support vector machines , 2006, Eur. J. Oper. Res..

[17]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[18]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[19]  Chih-Jen Lin,et al.  Coordinate Descent Method for Large-scale L2-loss Linear Support Vector Machines , 2008, J. Mach. Learn. Res..

[20]  Jie Li,et al.  Training robust support vector machine with smooth Ramp loss in the primal space , 2008, Neurocomputing.

[21]  Yingjie Tian,et al.  Large-scale linear nonparallel support vector machine solver , 2014, Neurocomputing.

[22]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[23]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[24]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[25]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[28]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[29]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[31]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[32]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Masao Fukushima,et al.  Second-Order Cone Programming Formulations for Robust Multiclass Classification , 2007, Neural Computation.

[34]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[35]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[36]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[37]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[38]  Yuan-Hai Shao,et al.  A coordinate descent margin based-twin support vector machine for classification , 2012, Neural Networks.

[39]  W. Wong,et al.  On ψ-Learning , 2003 .

[40]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[41]  Le Thi Hoai An,et al.  A D.C. Optimization Algorithm for Solving the Trust-Region Subproblem , 1998, SIAM J. Optim..

[42]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .