A bilateral-truncated-loss based robust support vector machine for classification problems

Support vector machine (SVM) is sensitive to outliers or noise in the training dataset. Fuzzy SVM (FSVM) and the bilateral-weighted FSVM (BW-FSVM) can partly overcome this shortcoming by assigning different fuzzy membership degrees to different training samples. However, it is a difficult task to set the fuzzy membership degrees of the training samples. To avoid setting fuzzy membership degrees, from the beginning of the BW-FSVM model, this paper outlines the construction of a bilateral-truncated-loss based robust SVM (BTL-RSVM) model for classification problems with noise. Based on its equivalent model, we theoretically analyze the reason why the robustness of BTL-RSVM is higher than that of SVM and BW-FSVM. To solve the proposed BTL-RSVM model, we propose an iterative algorithm based on the concave–convex procedure and the Newton–Armijo algorithm. A set of experiments is conducted on ten real world benchmark datasets to test the robustness of BTL-RSVM. The statistical tests of the experimental results indicate that compared with SVM, FSVM and BW-FSVM, the proposed BTL-RSVM can significantly reduce the effects of noise and provide superior robustness.

[1]  Zhang Yi,et al.  Fuzzy SVM with a new fuzzy membership function , 2006, Neural Computing & Applications.

[2]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[3]  Xuegong Zhang,et al.  Using class-center vectors to build support vector machines , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[4]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[5]  Kin Keung Lai,et al.  A new fuzzy support vector machine to evaluate credit risk , 2005, IEEE Transactions on Fuzzy Systems.

[6]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[7]  James M. Keller,et al.  Incorporating Fuzzy Membership Functions into the Perceptron Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xiaowei Yang,et al.  A Kernel Fuzzy c-Means Clustering-Based Fuzzy Support Vector Machine Algorithm for Classification Problems With Outliers or Noises , 2011, IEEE Transactions on Fuzzy Systems.

[9]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[10]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.

[11]  Jue Wang,et al.  A New Fuzzy Support Vector Machine Based on the Weighted Margin , 2004, Neural Processing Letters.

[12]  Vasile Palade,et al.  FSVM-CIL: Fuzzy Support Vector Machines for Class Imbalance Learning , 2010, IEEE Transactions on Fuzzy Systems.

[13]  Reshma Khemchandani,et al.  Fast and robust learning through fuzzy linear proximal support vector machines , 2004, Neurocomputing.

[14]  T.A. Jilani,et al.  Multiclass Bilateral-Weighted Fuzzy Support Vector Machine to Evaluate Financial Strength Credit Rating , 2008, 2008 International Conference on Computer Science and Information Technology.

[16]  Kin Keung Lai,et al.  A Least Squares Bilateral-Weighted Fuzzy SVM Method to Evaluate Credit Risk , 2008, 2008 Fourth International Conference on Natural Computation.

[17]  Koby Crammer,et al.  Robust Support Vector Machine Training via Convex Outlier Ablation , 2006, AAAI.

[18]  J.M. Leski,et al.  An /spl epsiv/-margin nonlinear classifier based on fuzzy if-then rules , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Yufeng Liu,et al.  Multicategory ψ-Learning and Support Vector Machine: Computational Tools , 2005 .

[20]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[21]  Xindong Wu.  Knowledge Acquisition from Databases , 1995 .

[22]  Yi Lin,et al.  Support Vector Machines and the Bayes Rule in Classification , 2002, Data Mining and Knowledge Discovery.

[23]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[24]  Jie Li,et al.  Training robust support vector machine with smooth Ramp loss in the primal space , 2008, Neurocomputing.

[25]  Wenjie Hu,et al.  Robust support vector machine with bullet hole image classification , 2002 .

[26]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[27]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[28]  Yufeng Liu,et al.  Adaptively Weighted Large Margin Classifiers , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[29]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[30]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[31]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[32]  C. Lee Giles,et al.  Nonconvex Online Support Vector Machines , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[34]  Yufeng Liu,et al.  Multicategory ψ-Learning , 2006 .

[35]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .