The robust and efficient adaptive normal direction support vector regression

The recently proposed reduced convex hull support vector regression (RH-SVR) treats support vector regression (SVR) as a classification problem in the dual feature space by introducing an epsilon-tube. In this paper, an efficient and robust adaptive normal direction support vector regression (AND-SVR) is developed by combining the geometric algorithm for support vector machine (SVM) classification. AND-SVR finds a better shift direction for training samples based on the normal direction of output function in the feature space compared with RH-SVR. Numerical examples on several artificial and UCI benchmark datasets with comparisons show that the proposed AND-SVR derives good generalization performance

[1]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[2]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[3]  Jue Wang,et al.  A generalized S-K algorithm for learning v-SVM classifiers , 2004, Pattern Recognit. Lett..

[4]  S. Sheather,et al.  Robust Estimation & Testing: Staudte/Robust , 1990 .

[5]  Jinbo Bi,et al.  A geometric approach to support vector regression , 2003, Neurocomputing.

[6]  R. Eubank.  Nonparametric Regression and Spline Smoothing , 1999 .

[7]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[8]  David J. Crisp,et al.  A Geometric Interpretation of ?-SVM Classifiers , 1999, NIPS 2000.

[9]  王珏,et al.  Kernel Projection Algorithm for Large—Scale SVM Problems , 2002 .

[10]  Wang Jue,et al.  Kernel projection algorithm for large-scale SVM problems , 2002 .

[11]  V. N. Malozemov,et al.  Finding the Point of a Polyhedron Closest to the Origin , 1974 .

[12]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Václav Hlavác,et al.  An iterative algorithm learning the maximal margin classifier , 2003, Pattern Recognit..

[14]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[15]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[16]  Ivor W. Tsang,et al.  Core Vector Machines: Fast SVM Training on Very Large Data Sets , 2005, J. Mach. Learn. Res..

[17]  R. Kass.  Nonlinear Regression Analysis and its Applications , 1990 .

[18]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[19]  T. Hassard,et al.  Applied Linear Regression , 2005 .

[20]  E. Gilbert.  An Iterative Procedure for Computing the Minimum of a Quadratic Form on a Convex Set , 1966 .

[21]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[22]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[23]  Annie Bessot.  Geometry at Work , 2000 .

[24]  S. Weisberg,et al.  Applied Linear Regression (2nd ed.). , 1986 .

[25]  Jue Wang,et al.  A general soft method for learning SVM classifiers with L1-norm penalty , 2008, Pattern Recognit..

[26]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.

[27]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[28]  Robert G. Staudte,et al.  Wiley Series in Probability and Mathematical Statistics , 2011 .