An $$l_1$$l1-norm loss based twin support vector regression and its geometric extension

This paper proposes a novel $$l_1$$l1-norm loss based twin support vector regression ($$l_1$$l1-TSVR) model. The bound functions in this $$l_1$$l1-TSVR are optimized by simultaneously minimizing the $$l_1$$l1-norm based fitting and one-side $$\epsilon$$ϵ-insensitive losses, which results in different dual problems compared with twin support vector regression (TSVR) and $$\epsilon$$ϵ-TSVR. The main advantages of this $$l_1$$l1-TSVR are: First, it does not need to inverse any kernel matrix in dual problems, indicating that it not only can be optimized efficiently, but also has partly sparse bound functions. Second, it has a perfect and practical geometric interpretation. In the spirit of its geometric interpretation, this paper further presents a nearest-points based $$l_1$$l1-TSVR (NP-$$l_1$$l1-TSVR), in which bound functions are constructed by finding the nearest points between the reduced convex/affine hulls of training data and its shifted sets, respectively. Computational results obtained on a number of synthetic and real-world benchmark datasets clearly illustrate the superiority of the proposed $$l_1$$l1-TSVR and NP-$$l_1$$l1-TSVR as comparable generalization performance is achieved in accordance with the other SVR-type algorithms.

[1]  Reshma Khemchandani,et al.  TWSVR: Regression via Twin Support Vector Machine , 2016, Neural Networks.

[2]  Jing Zhao,et al.  Twin least squares support vector regression , 2013, Neurocomputing.

[3]  Xinjun Peng,et al.  Primal twin support vector regression and its sparse approximation , 2010, Neurocomputing.

[4]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[5]  Yuan-Hai Shao,et al.  Improvements on Twin Support Vector Machines , 2011, IEEE Transactions on Neural Networks.

[6]  Yuan-Hai Shao,et al.  A novel parametric-insensitive nonparallel support vector machine for regression , 2016, Neurocomputing.

[7]  Dong Xu,et al.  A twin projection support vector machine for data regression , 2014, Neurocomputing.

[8]  Reshma Khemchandani,et al.  Improvements on ν-Twin Support Vector Machine , 2016, Neural Networks.

[9]  Hakan Cevikalp,et al.  Large margin classifiers based on affine hulls , 2010, Neurocomputing.

[10]  S. Sheather,et al.  Robust Estimation & Testing: Staudte/Robust , 1990 .

[11]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[12]  Douglas M. Bates,et al.  Nonlinear Regression Analysis and Its Applications , 1988 .

[13]  Dong Xu,et al.  L1-norm loss based twin support vector machine for data recognition , 2016, Inf. Sci..

[14]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[15]  Muhammad Tanveer,et al.  An efficient regularized K-nearest neighbor based weighted twin support vector regression , 2016, Knowl. Based Syst..

[16]  Yuan-Hai Shao,et al.  An ε-twin support vector machine for regression , 2012, Neural Computing and Applications.

[17]  S. Weisberg,et al.  Applied Linear Regression (2nd ed.). , 1986 .

[18]  Pei-Yi Hao,et al.  Interval regression analysis using support vector networks , 2009, Fuzzy Sets Syst..

[19]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  S. Balasundaram,et al.  Training Lagrangian twin support vector regression via unconstrained convex minimization , 2014, Knowl. Based Syst..

[21]  Jue Wang,et al.  A general soft method for learning SVM classifiers with L1-norm penalty , 2008, Pattern Recognit..

[22]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[23]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[24]  Xianli Pan,et al.  A Novel Twin Support-Vector Machine With Pinball Loss , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[25]  T. Hassard,et al.  Applied Linear Regression , 2005 .

[26]  Miguel Figueroa,et al.  Competitive learning with floating-gate circuits , 2002, IEEE Trans. Neural Networks.

[27]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[28]  Ali Amiri,et al.  Modified twin support vector regression , 2016, Neurocomputing.

[29]  Lan Bai,et al.  Weighted Lagrange ε-twin support vector regression , 2016, Neurocomputing.

[30]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[31]  Xinjun Peng,et al.  TPMSVM: A novel twin parametric-margin support vector machine for pattern recognition , 2011, Pattern Recognit..

[32]  José R. Dorronsoro,et al.  Clipping algorithms for solving the nearest point problem over reduced convex hulls , 2011, Pattern Recognit..

[33]  Yifei Wang,et al.  Geometric Algorithms to Large Margin Classifier Based on Affine Hulls , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Jinbo Bi,et al.  A geometric approach to support vector regression , 2003, Neurocomputing.

[35]  Qinghua Hu,et al.  Support function machine for set-based classification with application to water quality evaluation , 2017, Inf. Sci..

[36]  Xinjun Peng,et al.  Efficient twin parametric insensitive support vector regression model , 2012, Neurocomputing.

[37]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[38]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[39]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[40]  Chi-Hyuck Jun,et al.  Instance categorization by support vector machines to adjust weights in AdaBoost for imbalanced data classification , 2017, Inf. Sci..

[41]  S. Sathiya Keerthi,et al.  Improvements to the SMO algorithm for SVM regression , 2000, IEEE Trans. Neural Networks Learn. Syst..

[42]  Shangbing Gao,et al.  1-Norm least squares twin support vector machines , 2011, Neurocomputing.

[43]  Xinjun Peng,et al.  A nu-twin support vector machine (nu-TSVM) classifier and its geometric algorithms , 2010, Inf. Sci..

[44]  Pei-Yi Hao,et al.  New support vector algorithms with parametric insensitive/margin model , 2010, Neural Networks.

[45]  KhemchandaniReshma,et al.  Improvements on v -Twin Support Vector Machine , 2016 .