Twin least squares support vector regression

In this paper, combining the spirit of twin hyperplanes with the fast speed of least squares support vector regression (LSSVR) yields a new regressor, termed as twin least squares support vector regression (TLSSVR). As a result, TLSSVR outperforms normal LSSVR in the generalization performance, and as opposed to other algorithms of twin hyperplanes, TLSSVR owns faster computational speed. When coping with large scale problems, this advantage is obvious. To accelerate the testing speed of TLSSVR, TLSSVR is sparsified using a simple mechanism, thus obtaining STLSSVR. In addition to introducing these algorithms above, a lot of experiments including a toy problem, several small and large scale data sets, and a gas furnace example are done. These applications demonstrate the effectiveness and efficiency of the proposed algorithms.

[1]  Reshma Khemchandani,et al.  Optimal kernel selection in twin support vector machines , 2009, Optim. Lett..

[2]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[3]  Ping Zhong,et al.  Training twin support vector regression via linear programming , 2012, Neural Computing and Applications.

[4]  Anthony Kuh,et al.  Comments on "Pruning Error Minimization in Least Squares Support Vector Machines" , 2007, IEEE Trans. Neural Networks.

[5]  Shangbing Gao,et al.  Weighted Twin Support Vector Machines with Local Information and its application , 2012, Neural Networks.

[6]  Suresh Chandra,et al.  Reduced twin support vector regression , 2011, Neurocomputing.

[7]  Yong Shi,et al.  Laplacian twin support vector machine for semi-supervised classification , 2012, Neural Networks.

[8]  Pei-Yi Hao,et al.  New support vector algorithms with parametric insensitive/margin model , 2010, Neural Networks.

[9]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[10]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[11]  Wang Shi-tong.  New feature selection method , 2009 .

[12]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[13]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[14]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[15]  Anirban Mukherjee,et al.  Nonparallel plane proximal classifier , 2009, Signal Process..

[16]  C. M. Bishop,et al.  Improvements on Twin Support Vector Machines , 2011 .

[17]  Chih-Jen Lin,et al.  A study on reduced support vector machines , 2003, IEEE Trans. Neural Networks.

[18]  Senjian An,et al.  Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression , 2007, Pattern Recognit..

[19]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Xinjun Peng,et al.  Efficient twin parametric insensitive support vector regression model , 2012, Neurocomputing.

[21]  Shun-Feng Su,et al.  Robust support vector regression networks for function approximation with outliers , 2002, IEEE Trans. Neural Networks.

[22]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[23]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[24]  Xiaoqin Shan,et al.  Support Vector Machine Method for Multivariate Density Estimation Based on Copulas , 2011, 2011 International Conference on Internet Computing and Information Services.

[25]  Yu Hongyi,et al.  Probability density estimation based on SVM , 2009, 2009 Global Mobile Congress.

[26]  Johan A. K. Suykens,et al.  Sparse approximation using least squares support vector machines , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[27]  Madan Gopal,et al.  Application of smoothing technique on twin support vector machines , 2008, Pattern Recognit. Lett..

[28]  Licheng Jiao,et al.  Fast Sparse Approximation for Least Squares Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[29]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[30]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[31]  Chunxia Zhao,et al.  A feature selection method for nonparallel plane support vector machine classification , 2012, Optim. Methods Softw..

[32]  S. Sathiya Keerthi,et al.  Improvements to the SMO algorithm for SVM regression , 2000, IEEE Trans. Neural Networks Learn. Syst..

[33]  Madan Gopal,et al.  Least squares twin support vector machines for pattern classification , 2009, Expert Syst. Appl..

[34]  Jian Yang,et al.  Robust and Sparse Twin Support Vector Regression via Linear Programming , 2010, 2010 Chinese Conference on Pattern Recognition (CCPR).

[35]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[36]  Sangjun Lee,et al.  Feature selection in the Laplacian support vector machine , 2011, Comput. Stat. Data Anal..

[37]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[38]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[39]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[40]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Jiuyong Li,et al.  Combined Feature Selection and Cancer Prognosis Using Support Vector Machine Regression , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[42]  Sheng Chen,et al.  Particle Swarm Optimization Aided Orthogonal Forward Regression for Unified Data Modeling , 2010, IEEE Transactions on Evolutionary Computation.

[43]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[44]  B. Schölkopf,et al.  Linear programs for automatic accuracy control in regression. , 1999 .

[45]  S. Balasundaram,et al.  On Lagrangian twin support vector regression , 2012, Neural Computing and Applications.

[46]  Thomas Uthmann,et al.  Experiments in Value Function Approximation with Sparse Support Vector Regression , 2004, ECML.

[47]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[48]  Yitian Xu,et al.  A weighted twin support vector regression , 2012, Knowl. Based Syst..

[49]  Jian Yang,et al.  Smooth twin support vector regression , 2010, Neural Computing and Applications.