Robust support vector regression in the primal

The classical support vector regressions (SVRs) are constructed based on convex loss functions. Since non-convex loss functions to a certain extent own superiority to convex ones in generalization performance and robustness, we propose a non-convex loss function for SVR, and then the concave-convex procedure is utilized to transform the non-convex optimization to convex one. In the following, a Newton-type optimization algorithm is developed to solve the proposed robust SVR in the primal, which can not only retain the sparseness of SVR but also oppress outliers in the training examples. The effectiveness, namely better generalization, is validated through experiments on synthetic and real-world benchmark data sets.

[1]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[2]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[3]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[4]  Glenn Fung,et al.  Finite Newton method for Lagrangian support vector machine classification , 2003, Neurocomputing.

[5]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[6]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.

[7]  Jie Li,et al.  Training robust support vector machine with smooth Ramp loss in the primal space , 2008, Neurocomputing.

[8]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[9]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[10]  C.-C. Chuang,et al.  Fuzzy Weighted Support Vector Regression With a Fuzzy Partition , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Licheng Jiao,et al.  Recursive Finite Newton Algorithm for Support Vector Regression in the Primal , 2007, Neural Computation.

[12]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[13]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[14]  S. Sathiya Keerthi,et al.  A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs , 2005, J. Mach. Learn. Res..

[15]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[16]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[17]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[18]  S. Sathiya Keerthi,et al.  Improvements to the SMO algorithm for SVM regression , 2000, IEEE Trans. Neural Networks Learn. Syst..

[19]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[20]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .