Algorithm of ε-SVR Based on a Large-Scale Sample Set: Step-by-Step Search

In view of the support vectors of e-SVR that are not distributed in the e belt and only located on the outskirts of the e belt, a novel algorithm to construct e-SVR of a large-scale training sample set is proposed in this paper. It computes firstly the e-SVR hyper-plane of a small training sample set and the distances d of all samples to the hyper-plane, then deletes the samples not in field e ≤ d ≤ dmax and searches SVs gradually in the scope e ≤ d ≤ dmax, and trains step-by-step the final e-SVR. Finally, it analyzes the time complexity of the algorithm, and verifies its convergence in the theory and tests its efficiency by the simulation.

[1]  Gary William Flake,et al.  Efficient SVM Regression Training with SMO , 2002, Machine Learning.

[2]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[3]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[4]  Yuan Yan Tang,et al.  Analysis of Classification with a Reject Option , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[5]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[6]  Yan Wei,et al.  Research on an Algorithm of Support Vector Stepwise Regression , 2006, First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06).

[7]  Ji Zhu,et al.  Computing the Solution Path for the Regularized Support Vector Regression , 2005, NIPS.

[8]  S. Keerthi,et al.  Improvements to SMO Algorithm for SVM Regression 1 , 1999 .

[9]  Yuan Yan Tang,et al.  Local Learning Estimates by Integral Operators , 2010, Int. J. Wavelets Multiresolution Inf. Process..

[10]  Klaus Obermayer,et al.  An SMO Algorithm for the Potential Support Vector Machine , 2008, Neural Computation.

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .