An improved regularization based Lagrangian asymmetric ν-twin support vector regression using pinball loss function

In twin support vector regression (TSVR), one can notice that the samples are having the same importance even they are laying above the up-bound and below the down-bound on the estimation function for regression problem. Instead of giving the same emphasis to the samples, a novel approach Asymmetric ν-twin support vector regression (Asy-ν-TSVR) is suggested in this context where samples are having different influences with the estimation function based on samples distribution. Inspired by this concept, in this paper, we propose a new approach as improved regularization based Lagrangian asymmetric ν-twin support vector regression using pinball loss function (LAsy-ν-TSVR) which is more effective and efficient to deal with the outliers and noise. The solution is obtained by solving the simple linearly convergent approach which reduces the computational complexity of the proposed LAsy-ν-TSVR. Also, the structural risk minimization principle is implemented to make the problem strongly convex and more stable by adding the regularization term in their objective functions. The superiority of proposed LAsy-ν-TSVR is justified by performing the various numerical experiments on artificial generated datasets with symmetric and heteroscedastic structure noise as well as standard real-world datasets. The results of LAsy-ν-TSVR compares with support vector regression (SVR), TSVR, TSVR with Huber loss (HN-TSVR) and Asy-ν-TSVR, regularization on Lagrangian TSVR (RLTSVR) for the linear and Gaussian kernel which clearly demonstrates the efficacy and efficiency of the proposed algorithm LAsy-ν-TSVR.

[1]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[2]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[4]  Cuauhtémoc López Martín,et al.  Support vector regression for predicting the productivity of higher education graduate students from individually developed software projects , 2017, IET Softw..

[5]  K. Koçak,et al.  River flow prediction using hybrid models of support vector regression with the wavelet transform, singular spectrum analysis and chaotic approach , 2018, Meteorology and Atmospheric Physics.

[6]  C.-C. Chuang,et al.  Fuzzy Weighted Support Vector Regression With a Fuzzy Partition , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  D. Estève,et al.  Asymmetric noise probed with a josephson junction. , 2009, Physical review letters.

[8]  Keming Yu,et al.  A partially collapsed Gibbs sampler for Bayesian quantile regression , 2009 .

[9]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[10]  Yuqun Zhang,et al.  A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification , 2016, Knowl. Based Syst..

[11]  Jing Chen,et al.  Twin support vector regression with Huber loss , 2017, J. Intell. Fuzzy Syst..

[12]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[13]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[14]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[15]  James T. Kwok,et al.  Bayesian Support Vector Regression , 2001, AISTATS.

[16]  Muhammad Tanveer,et al.  A regularization on Lagrangian twin support vector regression , 2015, International Journal of Machine Learning and Cybernetics.

[17]  Suresh Chandra,et al.  A ν-twin support vector machine based regression with automatic accuracy control , 2017, Applied Intelligence.

[18]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[19]  Johan A. K. Suykens,et al.  Asymmetric v-tube support vector regression , 2014, Comput. Stat. Data Anal..

[20]  Xiongxin Xiao,et al.  Support vector regression snow-depth retrieval algorithm using passive microwave remote sensing data , 2018, Remote Sensing of Environment.

[21]  Yuh-Jye Lee,et al.  epsilon-SSVR: A Smooth Support Vector Machine for epsilon-Insensitive Regression , 2005, IEEE Trans. Knowl. Data Eng..

[22]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[23]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[24]  Alain Abran,et al.  Support vector regression for predicting software enhancement effort , 2018, Inf. Softw. Technol..

[25]  Xiaoyan Li,et al.  Asymmetric ν-twin support vector regression , 2017, Neural Computing and Applications.

[26]  Johan A. K. Suykens,et al.  Support Vector Machine Classifier With Pinball Loss , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  S. Balasundaram,et al.  Training Lagrangian twin support vector regression via unconstrained convex minimization , 2014, Knowl. Based Syst..

[28]  Zichen Zhang,et al.  A Hybrid Seasonal Mechanism with a Chaotic Cuckoo Search Algorithm with a Support Vector Regression Model for Electric Load Forecasting , 2018 .

[29]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[30]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[31]  Jing Zhao,et al.  Twin least squares support vector regression , 2013, Neurocomputing.

[32]  Abbas S. Milani,et al.  Support vector regression modelling and optimization of energy consumption in carbon fiber production line , 2018, Comput. Chem. Eng..

[33]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[34]  Yitian Xu,et al.  A rough margin-based linear ν support vector regression , 2012 .

[35]  Wenjian Wang,et al.  A heuristic training for support vector regression , 2004, Neurocomputing.

[36]  Ingo Steinwart,et al.  Estimating conditional quantiles with the help of the pinball loss , 2011, 1102.2101.

[37]  Xianli Pan,et al.  A Novel Twin Support-Vector Machine With Pinball Loss , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Jinbo Bi,et al.  A geometric approach to support vector regression , 2003, Neurocomputing.

[39]  Johan A. K. Suykens,et al.  Sequential minimal optimization for SVM with pinball loss , 2015, Neurocomputing.

[40]  Dong Xu,et al.  A twin-hypersphere support vector machine classifier and the fast learning algorithm , 2013, Inf. Sci..

[41]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[42]  Przemysław Rokita,et al.  Predicting Popularity of Online Videos Using Support Vector Regression , 2017, IEEE Transactions on Multimedia.

[43]  A support vector regression method for efficiently determining neutral profiles from laser induced fluorescence data. , 2018, The Review of scientific instruments.

[44]  Ding Sheng-feng An Improved Twin Support Vector Machine , 2012 .

[45]  Wei Chen,et al.  Support vector regression with modified firefly algorithm for stock price forecasting , 2018, Applied Intelligence.

[46]  Yuh-Jye Lee,et al.  2-SSVR : A Smooth Support Vector Machine for 2-insensitive Regression , 2004 .

[47]  David R. Musicant,et al.  Active set support vector regression , 2004, IEEE Transactions on Neural Networks.

[48]  Ricardo Nicolau Nassar Koury,et al.  Prediction of wind speed and wind direction using artificial neural network, support vector regression and adaptive neuro-fuzzy inference system , 2018 .

[49]  David R. Musicant,et al.  Robust Linear and Support Vector Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Raghu Nandan Sengupta,et al.  Use of asymmetric loss functions in sequential estimation problems for multiple linear regression , 2008 .

[51]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[52]  Pei-Yi Hao,et al.  New support vector algorithms with parametric insensitive/margin model , 2010, Neural Networks.

[53]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[54]  S. Balasundaram,et al.  On Lagrangian twin support vector regression , 2012, Neural Computing and Applications.