Least squares twin bounded support vector machines based on L1-norm distance metric for classification

Abstract In this paper, we construct a least squares version of the recently proposed twin bounded support vector machine (TBSVM) for binary classification. As a valid classification tool, TBSVM attempts to seek two non-parallel planes that can be produced by solving a pair of quadratic programming problems (QPPs), but this is time-consuming. Here, we solve two systems of linear equations rather than two QPPs to avoid this deficiency. Furthermore, the distance in least squares TBSVM (LSTBSVM) is measured by L2-norm, but L1-norm distance is usually regarded as an alternative to L2-norm to improve model robustness in the presence of outliers. Inspired by the advantages of least squares twin support vector machine (LSTWSVM), TBSVM and L1-norm distance, we propose a LSTBSVM based on L1-norm distance metric for binary classification, termed as L1-LSTBSVM, which is specially designed for suppressing the negative effect of outliers and improving computational efficiency in large datasets. Then, we design a powerful iterative algorithm to solve the L1-norm optimal problems, and it is easy to implement and its convergence to an optimum solution is theoretically ensured. Finally, the feasibility and effectiveness of L1-LSTBSVM are validated by extensive experimental results on both UCI datasets and artificial datasets.

[1]  Shangbing Gao,et al.  1-Norm least squares twin support vector machines , 2011, Neurocomputing.

[2]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[3]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[4]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[5]  Jiashu Zhang,et al.  Linear Discriminant Analysis Based on L1-Norm Maximization , 2013, IEEE Transactions on Image Processing.

[6]  Shangbing Gao,et al.  Weighted Twin Support Vector Machines with Local Information and its application , 2012, Neural Networks.

[7]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[8]  Liya Fan,et al.  APPLICATION OF SMOOTHING TECHNIQUE ON PROJECTIVE TSVM , 2015 .

[9]  Zechao Li,et al.  L1-Norm Distance Minimization-Based Fast Robust Twin Support Vector $k$ -Plane Clustering , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Nai-Yang Deng,et al.  Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions , 2012 .

[12]  Yuan-Hai Shao,et al.  Robust L1-norm non-parallel proximal support vector machine , 2016 .

[13]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[14]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jian Yang,et al.  L1-Norm Distance Linear Discriminant Analysis Based on an Effective Iterative Algorithm , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[17]  Hui Xue,et al.  Glocalization pursuit support vector machine , 2011, Neural Computing and Applications.

[18]  Mahesh N. Dumaldar.  A theoretical comparison between the simplex method and the basic line search algorithm , 2016 .

[19]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[20]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[21]  Divya Tomar,et al.  Multiclass Least Squares Twin Support Vector Machine for Pattern Classification , 2015 .

[22]  Zhao Chunxia,et al.  A Feature Selection Method for TWSVM via a Regularization Technique , 2011 .

[23]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Tian Sheng.  DATABASE LEARNING ALGORITHMS BASED ON SUPPORT VECTOR MACHINE , 2000 .

[25]  Bin Chen,et al.  Proximal support vector machine using local information , 2009, Neurocomputing.

[26]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[27]  Yuan-Hai Shao,et al.  Robust L1-norm two-dimensional linear discriminant analysis , 2015, Neural Networks.

[28]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[29]  Ning Ye,et al.  Improved Proximal Support Vector Machine via Generalized Eigenvalues , 2009, 2009 International Joint Conference on Computational Sciences and Optimization.

[30]  Haixian Wang,et al.  Locally principal component analysis based on L1-norm maximisation , 2015, IET Image Process..

[31]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[32]  Shifei Ding,et al.  An overview on nonparallel hyperplane support vector machine algorithms , 2013, Neural Computing and Applications.

[33]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[34]  Liu Xue.  COMPUTER KEYSTROKER VERIFICATION BASED ON SUPPORT VECTOR MACHINES , 2002 .