Adaptively weighted learning for twin support vector machines via Bregman divergences

Some versions of weighted (twin) support vector machines have been developed to handle the contaminated data. However, the weights of samples are generally obtained from the prior knowledge of data in advance. This article develops an adaptively weighted twin support vector machine via Bregman divergences. To better handle the contaminated data, we employ an insensitive loss function to control the fitting error of the samples in one class and introduce the weight (fuzzy membership) of each sample into the proposed model. The alternating optimization technique is utilized to solve the proposed model due to the characteristics of the model. The accelerated version of first-order methods is used to solve a quadratic programming problem, and the fuzzy membership of each sample is achieved analytically in the case of Bregman divergences. Experiments on some data sets have been conducted to show that our method gains better classification performance than previous methods, especially for the open set experiment.

[1]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[2]  Yuan-Hai Shao,et al.  Least squares recursive projection twin support vector machine for classification , 2012, Pattern Recognit..

[3]  Youfu Li,et al.  Incremental support vector machine learning in the primal and applications , 2009, Neurocomputing.

[4]  Youfu Li,et al.  Adaptive weighted learning for linear regression problems via Kullback-Leibler divergence , 2013, Pattern Recognit..

[5]  Weisi Lin,et al.  Scalable image quality assessment with 2D mel-cepstrum and machine learning approach , 2012, Pattern Recognit..

[6]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[8]  Mangui Liang,et al.  Fuzzy support vector machine based on within-class scatter for classification problems with outliers or noises , 2013, Neurocomputing.

[9]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[10]  Rui Guo,et al.  An improved ν-twin support vector machine , 2013, Applied Intelligence.

[11]  Yuan-Hai Shao,et al.  Probabilistic outputs for twin support vector machines , 2012, Knowl. Based Syst..

[12]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[13]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[14]  KhemchandaniReshma,et al.  Improvements on v -Twin Support Vector Machine , 2016 .

[15]  Yitian Xu,et al.  K-nearest neighbor-based weighted twin support vector regression , 2014, Applied Intelligence.

[16]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[17]  Dong Xu,et al.  Twin Mahalanobis distance-based support vector machines for pattern recognition , 2012, Inf. Sci..

[18]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[19]  Divya Tomar,et al.  Weighted Least Square Twin Support Vector Machine for Imbalanced Dataset , 2014 .

[20]  Reshma Khemchandani,et al.  Improvements on ν-Twin Support Vector Machine , 2016, Neural Networks.

[21]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[23]  Sadiq M. Sait,et al.  Cell assignment in hybrid CMOS/nanodevices architecture using Tabu Search , 2013, Applied Intelligence.

[24]  Reshma Khemchandani,et al.  TWSVR: Regression via Twin Support Vector Machine , 2016, Neural Networks.

[25]  Jing Zhao,et al.  Twin least squares support vector regression , 2013, Neurocomputing.

[26]  Anderson Rocha,et al.  Toward Open Set Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xiaojun Wu,et al.  A Novel Twin Support Vector Machine for Binary Classification Problems , 2016, Neural Processing Letters.

[28]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Reshma Khemchandani,et al.  Twin Support Vector Machines - Models, Extensions and Applications , 2016, Studies in Computational Intelligence.

[30]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[31]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[32]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[33]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[34]  Johan A. K. Suykens,et al.  Support Vector Machine Classifier With Pinball Loss , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[36]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[37]  Yong Shi,et al.  Robust twin support vector machine for pattern classification , 2013, Pattern Recognit..

[38]  Shangbing Gao,et al.  Weighted Twin Support Vector Machines with Local Information and its application , 2012, Neural Networks.

[39]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[40]  Ning Liu,et al.  Efficient Feature Scaling for Support Vector Machines with a Quadratic Kernel , 2013, Neural Processing Letters.

[41]  Zhi-Quan Luo,et al.  A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing , 2015, IEEE Signal Processing Magazine.

[42]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[43]  Shangbing Gao,et al.  1-Norm least squares twin support vector machines , 2011, Neurocomputing.

[44]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[45]  Jian Yang,et al.  Recursive projection twin support vector machine via within-class variance minimization , 2011, Pattern Recognit..