Building sparse twin support vector machine classifiers in primal space

Abstract Twin support vector machines (TSVM) obtain faster training speeds than classical support vector machines (SVM). However, TSVM augmented vectors lose sparsity. In this paper, a rapid sparse twin support vector machine (STSVM) classifier in primal space is proposed to improve the sparsity and robustness of TSVM. Based on a simple back-fitting strategy, the STSVM iteratively builds each nonparallel hyperplanes by adding one support vector (SV) from the corresponding class at one time. This process is terminated using an adaptive and stable stopping criterion. STSVM learning is implemented by linear equation computing systems through introducing a quadratic function to approximate the empirical risk. The computational results on several synthetic and benchmark datasets indicate that the STSVM obtains a sparse separating hyperplane at a low cost without sacrificing its generalization performance.

[1]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[2]  Hong Qiao,et al.  Associated evolution of a support vector machine-based classifier for pedestrian detection , 2009, Inf. Sci..

[3]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[4]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[5]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Licheng Jiao,et al.  Recursive Finite Newton Algorithm for Support Vector Regression in the Primal , 2007, Neural Computation.

[9]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[10]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[11]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[12]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[13]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[14]  Xinyu Guo,et al.  Pruning Support Vector Machines Without Altering Performances , 2008, IEEE Transactions on Neural Networks.

[15]  Licheng Jiao,et al.  Fast Sparse Approximation for Least Squares Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[16]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[17]  Johan A. K. Suykens,et al.  Least squares support vector machine classifiers: a large scale algorithm , 1999 .

[18]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[19]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[21]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[22]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[23]  Jue Wang,et al.  A generalized S-K algorithm for learning v-SVM classifiers , 2004, Pattern Recognit. Lett..

[24]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[25]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[26]  Frank Weber,et al.  Optimal Reduced-Set Vectors for Support Vector Machines with a Quadratic Kernel , 2004, Neural Computation.

[27]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[28]  D Haussler,et al.  Knowledge-based analysis of microarray gene expression data by using support vector machines. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  T. Nishi,et al.  A learning algorithm for improving the classification speed of support vector machines , 2005, Proceedings of the 2005 European Conference on Circuit Theory and Design, 2005..

[31]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[32]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[33]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[34]  Shun-Feng Su,et al.  Support vector interval regression networks for interval regression analysis , 2003, Fuzzy Sets Syst..

[35]  Thorsten Joachims,et al.  Sparse kernel SVMs via cutting-plane training , 2009, Machine Learning.

[36]  Madan Gopal,et al.  Knowledge based Least Squares Twin support vector machines , 2010, Inf. Sci..

[37]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  S. Sathiya Keerthi,et al.  A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs , 2005, J. Mach. Learn. Res..

[39]  Licheng Jiao,et al.  Selecting a Reduced Set for Building Sparse Support Vector Regression in the Primal , 2007, PAKDD.

[40]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[41]  Qing Li,et al.  Adaptive simplification of solution for support vector machine , 2007, Pattern Recognit..

[42]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.