A Fast and Robust TSVM for Pattern Classification

Twin support vector machine~(TSVM) is a powerful learning algorithm by solving a pair of smaller SVM-type problems. However, there are still some specific issues such as low efficiency and weak robustness when it is faced with some real applications. In this paper, we propose a Fast and Robust TSVM~(FR-TSVM) to deal with the above issues. In order to alleviate the effects of noisy inputs, we propose an effective fuzzy membership function and reformulate the TSVMs such that different input instances can make different contributions to the learning of the separating hyperplanes. To further speed up the training procedure, we develop an efficient coordinate descent algorithm with shirking to solve the involved a pair of quadratic programming problems (QPPs). Moreover, theoretical foundations of the proposed model are analyzed in details. The experimental results on several artificial and benchmark datasets indicate that the FR-TSVM not only obtains a fast learning speed but also shows a robust classification performance. Code has been made available at: this https URL.

[1]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[4]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  S. Abe,et al.  Fuzzy support vector machines for pattern classification , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[7]  Yuji Matsumoto,et al.  Chunking with Support Vector Machines , 2001, NAACL.

[8]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[9]  Dong Xu,et al.  A twin projection support vector machine for data regression , 2014, Neurocomputing.

[10]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[11]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[12]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Yong Shi,et al.  Laplacian twin support vector machine for semi-supervised classification , 2012, Neural Networks.

[14]  Nikolas P. Galatsanos,et al.  A support vector machine approach for detection of microcalcifications , 2002, IEEE Transactions on Medical Imaging.

[15]  Ping Zhong,et al.  A rough margin-based ν-twin support vector machine , 2011, Neural Computing and Applications.

[16]  Feng-Li Lian,et al.  Robust classifier learning with fuzzy class labels for large-margin support vector machines , 2013, Neurocomputing.

[17]  Qi Chao,et al.  Review on Data-based Decision Making Methodologies , 2009 .

[18]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..

[19]  Suresh Chandra,et al.  Robust Parametric Twin Support Vector Machine for Pattern Classification , 2018, Neural Processing Letters.

[20]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[21]  Jalal A. Nasiri,et al.  Least squares twin multi-class classification support vector machine , 2015, Pattern Recognit..

[22]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[23]  Chih-Jen Lin,et al.  A dual coordinate descent method for large-scale linear SVM , 2008, ICML '08.

[24]  Lan Bai,et al.  Weighted Lagrange ε-twin support vector regression , 2016, Neurocomputing.

[25]  Xianli Pan,et al.  A Novel Twin Support-Vector Machine With Pinball Loss , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Yao Wang,et al.  Coordinate Descent Fuzzy Twin Support Vector Machine for Classification , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[27]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[28]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[29]  Wan Mei Tang.  Fuzzy SVM with a New Fuzzy Membership Function to Solve the Two-Class Problems , 2011, Neural Processing Letters.

[30]  Sheng-De Wang,et al.  Training algorithms for fuzzy support vector machines with noisy data , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[31]  Chih-Jen Lin,et al.  Coordinate Descent Method for Large-scale L2-loss Linear Support Vector Machines , 2008, J. Mach. Learn. Res..

[32]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.

[33]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[34]  Tomaso A. Poggio,et al.  Face recognition with support vector machines: global versus component-based approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Muhammad Tanveer,et al.  A robust fuzzy least squares twin support vector machine for class imbalance learning , 2018, Appl. Soft Comput..

[37]  Reshma Khemchandani,et al.  TWSVR: Regression via Twin Support Vector Machine , 2016, Neural Networks.

[38]  Jian-xiong Dong,et al.  Fast SVM training algorithm with decomposition on very large data sets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Achille Messac,et al.  Optimization in Practice with MATLAB®: For Engineering Students and Professionals , 2015 .

[40]  Suresh Chandra,et al.  A ν-twin support vector machine based regression with automatic accuracy control , 2017, Applied Intelligence.

[41]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[42]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[43]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[45]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[46]  Bo-Suk Yang,et al.  Support vector machine in machine condition monitoring and fault diagnosis , 2007 .

[47]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[48]  Yuan-Hai Shao,et al.  A coordinate descent margin based-twin support vector machine for classification , 2012, Neural Networks.