Robust twin support vector machine for pattern classification

In this paper, we proposed a new robust twin support vector machine (called R-TWSVM) via second order cone programming formulations for classification, which can deal with data with measurement noise efficiently. Preliminary experiments confirm the robustness of the proposed method and its superiority to the traditional robust SVM in both computation time and classification accuracy. Remarkably, since there are only inner products about inputs in our dual problems, this makes us apply kernel trick directly for nonlinear cases. Simultaneously we does not need to solve the extra inverse of matrices, which is totally different with existing TWSVMs. In addition, we also show that the TWSVMs are the special case of our robust model and simultaneously give a new dual form of TWSVM by degenerating R-TWSVM, which successfully overcomes the existing shortcomings of TWSVM.

[1]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[2]  Jinbo Bi,et al.  Support Vector Classification with Input Data Uncertainty , 2004, NIPS.

[3]  Koby Crammer,et al.  Robust Support Vector Machine Training via Convex Outlier Ablation , 2006, AAAI.

[4]  C. M. Bishop,et al.  Improvements on Twin Support Vector Machines , 2011 .

[5]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Masao Fukushima,et al.  Second-Order Cone Programming Formulations for Robust Multiclass Classification , 2007, Neural Computation.

[7]  Glenn Fung,et al.  Multicategory Proximal Support Vector Machine Classifiers , 2005, Machine Learning.

[8]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[9]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[10]  Theodore B. Trafalis,et al.  Robust classification and regression using support vector machines , 2006, Eur. J. Oper. Res..

[11]  Isabelle Guyon,et al.  Statistical Learning and Kernel Methods in Bioinformatics , 2003 .

[12]  Constantine Caramanis,et al.  Theory and Applications of Robust Optimization , 2010, SIAM Rev..

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[16]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[17]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[18]  Donald Goldfarb,et al.  Robust Portfolio Selection Problems , 2003, Math. Oper. Res..

[19]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[20]  Donald Goldfarb,et al.  Robust convex quadratically constrained programs , 2003, Math. Program..

[21]  Boubakeur Boufama,et al.  A novel SVM+NDA model for classification with an application to face recognition , 2012, Pattern Recognit..

[22]  Yue-Shi Lee,et al.  Robust and efficient multiclass SVM models for phrase pattern recognition , 2008, Pattern Recognit..

[23]  Rujie Liu,et al.  SVM-based active feedback in image retrieval using clustering and unlabeled data , 2008, Pattern Recognit..

[24]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Bernhard Schölkopf,et al.  Support Vector Machine Applications in Computational Biology , 2004 .

[26]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[27]  William Stafiord Noble,et al.  Support vector machine applications in computational biology , 2004 .

[28]  Dong Ming,et al.  Infrared gait recognition based on wavelet transform and support vector machine , 2010, Pattern Recognit..

[29]  A. Martínez,et al.  The AR face databasae , 1998 .

[30]  Ting Wang,et al.  Color image segmentation using pixel wise support vector machine classification , 2011, Pattern Recognit..

[31]  Shuang-Hong Yang,et al.  A Stagewise Least Square Loss Function for Classification , 2008, SDM.

[32]  Shie Mannor,et al.  Robust Regression and Lasso , 2008, IEEE Transactions on Information Theory.

[33]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[34]  Shie Mannor,et al.  Risk sensitive robust support vector machines , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[35]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[36]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[37]  Alexander J. Smola,et al.  A Second Order Cone programming Formulation for Classifying Missing Data , 2004, NIPS.

[38]  Shie Mannor,et al.  Robustness and generalization , 2010, Machine Learning.

[39]  Takashi Tsuchiya,et al.  Primal-dual algorithms and infinite-dimensional Jordan algebras of finite rank , 2003, Math. Program..

[40]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[41]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[42]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[43]  Mohamed Cheriet,et al.  Model selection for the LS-SVM. Application to handwriting recognition , 2009, Pattern Recognit..

[44]  Yaonan Wang,et al.  Texture classification using the support vector machines , 2003, Pattern Recognit..

[45]  Alexander J. Smola,et al.  Second Order Cone Programming Approaches for Handling Missing and Uncertain Data , 2006, J. Mach. Learn. Res..