Ν-projection Twin Support Vector Machine for Pattern Classification

Abstract In this paper, we improve the projection twin support vector machine (PTSVM) to a novel nonparallel classifier, termed as ν-PTSVM. Specifically, our ν-PTSVM aims to seek an optimal projection for each class such that, in each projection direction, instances of their own class are clustered around their class center while keep instances of the other class at least one distance away from such center. Different from PTSVM, our ν-PTSVM enjoys the following characteristics: (i) ν-PTSVM is equipped by a more theoretically sound parameter ν, which can be used to control the bounds of fraction of both support vectors and margin-error instances. (ii) By reformulating the least-square loss of within-class instances in primal problems of ν-PTSVM, its dual problems no longer involve the time-costly matrix inversion. (iii) ν-PTSVM behaves consistent between its linear and nonlinear cases. Namely, the kernel trick can be applied directly to ν-PTSVM for its nonlinear extension. Experimental evaluations on both synthetic and real-world datasets demonstrate the feasibility and effectiveness of the proposed approach.

[1]  Bin Fang,et al.  Scene classification based on single-layer SAE and SVM , 2015, Expert Syst. Appl..

[2]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[3]  Johan A. K. Suykens,et al.  Optimized fixed-size kernel models for large data sets , 2010, Comput. Stat. Data Anal..

[4]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[5]  Yuan-Hai Shao,et al.  Manifold proximal support vector machine for semi-supervised classification , 2013, Applied Intelligence.

[6]  Shifei Ding,et al.  Weighted least squares projection twin support vector machines with local information , 2015, Neurocomputing.

[7]  Lingfeng Niu,et al.  Nonparallel Support Vector Ordinal Regression , 2017, IEEE Transactions on Cybernetics.

[8]  Yingjie Tian,et al.  Large-scale linear nonparallel support vector machine solver , 2014, Neurocomputing.

[9]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[10]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[11]  L. Hao,et al.  Partial discharge source discrimination using a support vector machine , 2010, IEEE Transactions on Dielectrics and Electrical Insulation.

[12]  Yuan-Hai Shao,et al.  A regularization for the projection twin support vector machine , 2013, Knowl. Based Syst..

[13]  Reshma Khemchandani,et al.  Twin Support Vector Machines - Models, Extensions and Applications , 2016, Studies in Computational Intelligence.

[14]  Jane You,et al.  A New Kind of Nonparametric Test for Statistical Comparison of Multiple Classifiers Over Multiple Datasets , 2017, IEEE Transactions on Cybernetics.

[15]  Huiru Wang,et al.  An improved rough margin-based ν-twin bounded support vector machine , 2017, Knowl. Based Syst..

[16]  Yuan-Hai Shao,et al.  Laplacian smooth twin support vector machine for semi-supervised classification , 2013, International Journal of Machine Learning and Cybernetics.

[17]  Johan A. K. Suykens,et al.  Regularization, Optimization, Kernels, and Support Vector Machines , 2014 .

[18]  Samuel Kotz,et al.  On the Student's t-distribution and the t-statistic , 2007 .

[19]  Yuan-Hai Shao,et al.  MLTSVM: A novel twin support vector machine to multi-label learning , 2016, Pattern Recognit..

[20]  Yuan-Hai Shao,et al.  Laplacian least squares twin support vector machine for semi-supervised classification , 2014, Neurocomputing.

[21]  Abdulhamit Subasi,et al.  Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders , 2013, Comput. Biol. Medicine.

[22]  Zhaowei Shang,et al.  Scattering transform and LSPTSVM based fault diagnosis of rotating machinery , 2018 .

[23]  Yong Shi,et al.  Structural twin support vector machine for classification , 2013, Knowl. Based Syst..

[24]  Zechao Li,et al.  L1-Norm Distance Minimization-Based Fast Robust Twin Support Vector $k$ -Plane Clustering , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Xinjun Peng,et al.  PTSVRs: Regression models via projection twin support vector machine , 2018, Inf. Sci..

[27]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[28]  Ning Ye,et al.  Multi-weight vector projection support vector machines , 2010, Pattern Recognit. Lett..

[29]  Yu Xue,et al.  Wavelet twin support vector machines based on glowworm swarm optimization , 2017, Neurocomputing.

[30]  Johan A. K. Suykens,et al.  Non-parallel support vector classifiers with different loss functions , 2014, Neurocomputing.

[31]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[32]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[33]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[34]  Yuan-Hai Shao,et al.  Multiple birth support vector machine for multi-class classification , 2012, Neural Computing and Applications.

[35]  Ning Ye,et al.  Enhanced multi-weight vector projection support vector machine , 2014, Pattern Recognit. Lett..

[36]  Yu Xue,et al.  Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification , 2017, Pattern Recognit..

[37]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[38]  Shifei Ding,et al.  An overview on nonparallel hyperplane support vector machine algorithms , 2013, Neural Computing and Applications.

[39]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[40]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[41]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[42]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..

[43]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[44]  Yuan-Hai Shao,et al.  Improved Generalized Eigenvalue Proximal Support Vector Machine , 2013, IEEE Signal Processing Letters.

[45]  Yuan-Hai Shao,et al.  An efficient weighted Lagrangian twin support vector machine for imbalanced data classification , 2014, Pattern Recognit..

[46]  Yuan-Hai Shao,et al.  Nonparallel hyperplane support vector machine for binary classification problems , 2014, Inf. Sci..

[47]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Isabel Praça,et al.  Support Vector Machines for decision support in electricity markets' strategic bidding , 2016, Neurocomputing.

[49]  Ying-jie Tian,et al.  Improved twin support vector machine , 2014 .

[50]  Yuan-Hai Shao,et al.  2DRLPP: Robust two-dimensional locality preserving projection with regularization , 2019, Knowl. Based Syst..

[51]  Yuan-Hai Shao,et al.  Robust L1-norm multi-weight vector projection support vector machine with efficient algorithm , 2018, Neurocomputing.

[52]  Shifei Ding,et al.  Recursive least squares projection twin support vector machines for nonlinear classification , 2014, Neurocomputing.