2-SSVR : A Smooth Support Vector Machine for 2-insensitive Regression

A new smoothing strategy for solving 2-support vector regression (2-SVR), tolerating a small error in fitting a given dataset linearly or nonlinearly, is proposed in this paper. Conventionally, 2-SVR is formulated as a constrained minimization problem, namely a convex quadratic programming problem. We apply the smoothing techniques that have been used for solving the support vector machine for classification, to replace the 2-insensitive loss function by an accurate smooth approximation. This will allow us to solve 2-SVR as an unconstrained minimization problem directly. We term this reformulated problem as 2-smooth support vector regression (2-SSVR). We also prescribe a NewtonArmijo algorithm that has been shown to be convergent globally and quadratically in finite steps to solve our 2-SSVR. In order to handle the case of nonlinear Department of Computer Science & Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan 106, yuh-jye@mail.ntust.edu.tw. Department of Computer Science & Information Engineering, National Chung Cheng University, Chia-Yi, Taiwan 621, hwf90@cs.ccu.edu.tw. ‡ Department of Computer Science & Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan 106, M9215004@mail.ntust.edu.tw.

[1]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[2]  David R. Musicant,et al.  Robust Linear and Support Vector Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Patrick T. Harker,et al.  Smooth Approximations to Nonlinear Complementarity Problems , 1997, SIAM J. Optim..

[4]  Y. Ye,et al.  On Homotopy-Smoothing Methods for Variational Inequalities , 1999 .

[5]  Olvi L. Mangasarian,et al.  A Finite Newton Method for Classi cation Problems , 2001 .

[6]  P. W. Christensen,et al.  Frictional Contact Algorithms Based on Semismooth Newton Methods , 1998 .

[7]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[8]  David R. Musicant,et al.  Active set support vector regression , 2004, IEEE Transactions on Neural Networks.

[9]  P. Tseng.  Analysis Of A Non-Interior Continuation Method Based On Chen-Mangasarian Smoothing Functions For Com , 1998 .

[10]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[11]  Olvi L. Mangasarian,et al.  A class of smoothing functions for nonlinear and mixed complementarity problems , 1996, Comput. Optim. Appl..

[12]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[13]  Simon Haykin,et al.  Generalized support vector machines , 1999, ESANN.

[14]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[15]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[18]  David R. Musicant,et al.  Large Scale Kernel Regression via Linear Programming , 2002, Machine Learning.

[19]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[20]  Olvi L. Mangasarian,et al.  Smoothing methods for convex inequalities and linear complementarity problems , 1995, Math. Program..

[21]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[22]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[23]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[24]  Chih-Jen Lin,et al.  A study on reduced support vector machines , 2003, IEEE Trans. Neural Networks.

[25]  Xiaojun Chen,et al.  Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities , 1998, Math. Comput..