On finite Newton method for support vector regression

In this paper, we propose a Newton iterative method of solution for solving an ε-insensitive support vector regression formulated as an unconstrained optimization problem. The proposed method has the advantage that the solution is obtained by solving a system of linear equations at a finite number of times rather than solving a quadratic optimization problem. For the case of linear or kernel support vector regression, the finite termination of the Newton method has been proved. Experiments were performed on IBM, Google, Citigroup and Sunspot time series. The proposed method converges in at most six iterations. The results are compared with that of the standard, least squares and smooth support vector regression methods and of the exact solutions clearly demonstrate the effectiveness of the proposed method.

[1]  Theodore B. Trafalis,et al.  Support vector machine for regression and applications to financial forecasting , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[2]  Michael Y. Hu,et al.  A simulation study of artificial neural networks for nonlinear time-series forecasting , 2001, Comput. Oper. Res..

[3]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[4]  Lutgarde M. C. Buydens,et al.  Using support vector machines for time series prediction , 2003 .

[5]  Lei Guo,et al.  Forecasting stock composite index by fuzzy support vector machines regression , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[6]  David R. Musicant,et al.  Active Support Vector Machine Classification , 2000, NIPS.

[7]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[8]  Olvi L. Mangasarian,et al.  A finite newton method for classification , 2002, Optim. Methods Softw..

[9]  O. Mangasarian.  Parallel Gradient Distribution in Unconstrained Optimization , 1995 .

[10]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[11]  Kuan-Yu Chen,et al.  A hybrid SARIMA and support vector machines in forecasting the production values of the machinery industry in Taiwan , 2007, Expert Syst. Appl..

[12]  David R. Musicant,et al.  Active set support vector regression , 2004, IEEE Transactions on Neural Networks.

[14]  Glenn Fung,et al.  Finite Newton method for Lagrangian support vector machine classification , 2003, Neurocomputing.

[15]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[16]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[17]  Paul A. Fishwick,et al.  Feedforward Neural Nets as Models for Time Series Forecasting , 1993, INFORMS J. Comput..

[18]  Lijuan Cao,et al.  Support vector machines experts for time series forecasting , 2003, Neurocomputing.

[19]  Xufa Wang,et al.  A Gene-Prediction Algorithm Based on the Statistical Combination and the Classification in Terms of Gene Characteristics , 2005, 2005 International Conference on Neural Networks and Brain.

[20]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[21]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[22]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[23]  P. Fishwick,et al.  Feed-forward Neural Nets as Models for Time Series Forecasting , 1993 .

[24]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[25]  J. Hiriart-Urruty,et al.  Generalized Hessian matrix and second-order optimality conditions for problems withC1,1 data , 1984 .

[26]  Yuh-Jye Lee,et al.  epsilon-SSVR: A Smooth Support Vector Machine for epsilon-Insensitive Regression , 2005, IEEE Trans. Knowl. Data Eng..

[27]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Gunnar Rätsch,et al.  Using support vector machines for time series prediction , 1999 .

[29]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[30]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[31]  Fang-Mei Tseng,et al.  Combining neural network model with seasonal time series ARIMA model , 2002 .