Robust least squares support vector machine based on recursive outlier elimination

To achieve robust estimation for noisy data set, a recursive outlier elimination-based least squares support vector machine (ROELS-SVM) algorithm is proposed in this paper. In this algorithm, statistical information from the error variables of least squares support vector machine is recursively learned and a criterion derived from robust linear regression is employed for outlier elimination. Besides, decremental learning technique is implemented in the recursive training–eliminating stage, which ensures that the outliers are eliminated with low computational cost. The proposed algorithm is compared with re-weighted least squares support vector machine on multiple data sets and the results demonstrate the remarkably robust performance of the ROELS-SVM.

[1]  Jan-Ming Ho,et al.  Travel time prediction with support vector regression , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[2]  F. E. Grubbs.  Procedures for Detecting Outlying Observations in Samples , 1969 .

[3]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[4]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[5]  T. O. Kvålseth.  Cautionary Note about R 2 , 1985 .

[6]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[7]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[8]  Shun-Feng Su,et al.  Robust support vector regression networks for function approximation with outliers , 2002, IEEE Trans. Neural Networks.

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  Bernhard Schölkopf,et al.  Comparing support vector machines with Gaussian kernels to radial basis function classifiers , 1997, IEEE Trans. Signal Process..

[11]  Corinna Cortes,et al.  Prediction of Generalization Ability in Learning Machines , 1994 .

[12]  Zhang Jiang.  Reweighted Robust Support Vector Regression Method , 2005 .

[13]  David R. Musicant,et al.  Robust Linear and Support Vector Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[15]  Marchese Maurizio,et al.  Support vector machine regression algorithm based on chunking incremental learning , 2006 .

[16]  PETER J. ROUSSEEUW,et al.  Computing LTS Regression for Large Data Sets , 2005, Data Mining and Knowledge Discovery.

[17]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[18]  Kwoh Chee Keong,et al.  Fast leave-one-out evaluation and improvement on inference for LS-SVMs , 2004, ICPR 2004.

[19]  Joseph De Brabanter.  LS-SVM regression modelling and its applications , 2004 .

[20]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[21]  Chunguo Wu,et al.  Support Vector Machine Regression Algorithm Based on Chunking Incremental Learning , 2006, International Conference on Computational Science.

[22]  Xiaowei Yang,et al.  A heuristic weight-setting strategy and iteratively updating algorithm for weighted least-squares support vector regression , 2008, Neurocomputing.

[23]  Gavin C. Cawley,et al.  Fast exact leave-one-out cross-validation of sparse least-squares support vector machines , 2004, Neural Networks.

[24]  Tian Sheng-feng.  A Simplification Algorithm to Support Vector Machines for Regression , 2002 .

[25]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.