Two smooth support vector machines for ε-insensitive regression

In this paper, we propose two new smooth support vector machines for εinsensitive regression. According to these two smooth support vector machines, we construct two systems of smooth equations based on two novel families of smoothing functions, from which we seek the solution to ε-support vector regression(ε-SVR). More specifically, using the proposed smoothing functions, we employ the smoothing Newton method to solve the systems of smooth equations. The algorithm is shown to be globally and quadratically convergent without any additional conditions. Numerical comparisons among different values of parameter are also reported.

[1]  Yuh-Jye Lee,et al.  epsilon-SSVR: A Smooth Support Vector Machine for epsilon-Insensitive Regression , 2005, IEEE Trans. Knowl. Data Eng..

[2]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[3]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[4]  Su-Yun Huang,et al.  Reduced Support Vector Machines: A Statistical Theory , 2007, IEEE Transactions on Neural Networks.

[5]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[6]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[7]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[8]  R. Mifflin.  Semismooth and Semiconvex Functions in Constrained Optimization , 1977 .

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[11]  Ying Zhang,et al.  A smoothing-type algorithm for solving system of inequalities , 2008 .

[12]  Paul Tseng,et al.  A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training , 2010, Comput. Optim. Appl..

[13]  R. Palais,et al.  Critical Point Theory and Submanifold Geometry , 1988 .

[14]  D. Basak,et al.  Support Vector Regression , 2008 .

[15]  David R. Musicant,et al.  Active set support vector regression , 2004, IEEE Transactions on Neural Networks.

[16]  YuBo Yuan,et al.  A Polynomial Smooth Support Vector Machine for Classification , 2005, ADMA.

[17]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[18]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[19]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .