Robust L1-norm multi-weight vector projection support vector machine with efficient algorithm

Abstract The recently proposed multi-weight vector projection support vector machine (EMVSVM) is an excellent multi-projections classifier. However, the formulation of MVSVM is based on the L2-norm criterion, which makes it prone to be affected by outliers. To alleviate this issue, in this paper, we propose a robust L1-norm MVSVM method, termed as MVSVM L 1 . Specifically, our MVSVM L 1 aims to seek a pair of multiple projections such that, for each class, it maximizes the ratio of the L1-norm between-class dispersion and the L1-norm within-class dispersion. To optimize such L1-norm ratio problem, a simple but efficient iterative algorithm is further presented. The convergence of the algorithm is also analyzed theoretically. Extensive experimental results on both synthetic and real-world datasets confirm the feasibility and effectiveness of the proposed MVSVM L 1 .

[1]  Yuan-Hai Shao,et al.  Improved Generalized Eigenvalue Proximal Support Vector Machine , 2013, IEEE Signal Processing Letters.

[2]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[3]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.

[4]  Zhaowei Shang,et al.  Scattering transform and LSPTSVM based fault diagnosis of rotating machinery , 2018 .

[5]  Ning Ye,et al.  Multi-weight vector projection support vector machines , 2010, Pattern Recognit. Lett..

[6]  Tommy W. S. Chow,et al.  Maximum Margin Multisurface Support Tensor Machines with application to image classification and segmentation , 2012, Expert Syst. Appl..

[7]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Francis R. Bach,et al.  Structured Sparse Principal Component Analysis , 2009, AISTATS.

[9]  Jiashu Zhang,et al.  Robust locality preserving projection based on maximum correntropy criterion , 2014, J. Vis. Commun. Image Represent..

[10]  Shifei Ding,et al.  A semi-supervised approximate spectral clustering algorithm based on HMRF model , 2018, Inf. Sci..

[11]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[12]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[13]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[15]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[16]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[17]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[18]  Ning Ye,et al.  Enhanced multi-weight vector projection support vector machine , 2014, Pattern Recognit. Lett..

[19]  Panos M. Pardalos,et al.  A classification method based on generalized eigenvalue problems , 2007, Optim. Methods Softw..

[20]  Yuan-Hai Shao,et al.  Manifold proximal support vector machine for semi-supervised classification , 2013, Applied Intelligence.

[21]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[22]  Yuan-Hai Shao,et al.  Robust L1-norm non-parallel proximal support vector machine , 2016 .

[23]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[24]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..

[25]  Yuan-Hai Shao,et al.  Laplacian least squares twin support vector machine for semi-supervised classification , 2014, Neurocomputing.

[26]  Yuan-Hai Shao,et al.  Sparse Lq-norm least squares support vector machine with feature selection , 2018, Pattern Recognit..

[27]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[28]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[29]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[30]  Abdulhamit Subasi,et al.  Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders , 2013, Comput. Biol. Medicine.

[31]  Yan Zhang,et al.  Semi-supervised local multi-manifold Isomap by linear embedding for feature extraction , 2018, Pattern Recognit..

[32]  Jacek M. Zurada,et al.  Generalized Core Vector Machines , 2006, IEEE Transactions on Neural Networks.

[33]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[34]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[36]  Deyu Meng,et al.  Improve robustness of sparse PCA by L1-norm maximization , 2012, Pattern Recognit..

[37]  He Yan,et al.  Least squares twin bounded support vector machines based on L1-norm distance metric for classification , 2018, Pattern Recognit..

[38]  Hua Xu,et al.  Chinese comments sentiment classification based on word2vec and SVMperf , 2015, Expert Syst. Appl..

[39]  Jiashu Zhang,et al.  Discriminant Locality Preserving Projections Based on L1-Norm Maximization , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Samuel Kotz,et al.  On the Student's t-distribution and the t-statistic , 2007 .

[41]  Shifei Ding,et al.  An overview on nonparallel hyperplane support vector machine algorithms , 2013, Neural Computing and Applications.

[42]  Yuan Yuan,et al.  Outlier-resisting graph embedding , 2010, Neurocomputing.

[43]  Yuan-Hai Shao,et al.  Robust L1-norm two-dimensional linear discriminant analysis , 2015, Neural Networks.

[44]  Zhao Zhang,et al.  Robust image recognition by L1-norm twin-projection support vector machine , 2017, Neurocomputing.

[45]  Bin Fang,et al.  Scene classification based on single-layer SAE and SVM , 2015, Expert Syst. Appl..