Variance Minimization Least Squares Support Vector Machines for Time Series Analysis

Here we propose a novel machine learning method for time series forecasting which is based on the widely-used Least Squares Support Vector Machine (LS-SVM) approach. The objective function of our method contains a weighted variance minimization part as well. This modification makes the method more efficient in time series forecasting, as this paper will show. The proposed method is a generalization of the well-known LS-SVM algorithm. It has similar advantages like the applicability of the kernel-trick, it has a linear and unique solution, and a short computational time, but can perform better in certain scenarios. The main purpose of this paper is to introduce the novel Variance Minimization Least Squares Support Vector Machine (VMLS-SVM) method and to show its superiority through experimental results using standard benchmark time series prediction datasets.

[1]  Johan A. K. Suykens,et al.  Financial time series prediction using least squares support vector machines within the evidence framework , 2001, IEEE Trans. Neural Networks.

[2]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[3]  Sven F. Crone,et al.  A Naïve Support Vector Regression Benchmark for the NN3 Forecasting Competition , 2007, 2007 International Joint Conference on Neural Networks.

[4]  Jouko Lampinen,et al.  Time series prediction by Kalman smoother with cross-validated noise density , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[5]  Chris Chatfield,et al.  Time series forecasting with neural networks , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[6]  Theo J. A. de Vries,et al.  Pruning error minimization in least squares support vector machines , 2003, IEEE Trans. Neural Networks.

[7]  Ginés Rubio,et al.  Kernel Methods Applied to Time Series Forecasting , 2007, IWANN.

[8]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[9]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[10]  Jouko Lampinen,et al.  Time series prediction by Kalman smoother with cross-validated noise density , 2004 .

[11]  Julian J. Faraway,et al.  Time Series Forecasting with Neural Networks: A Case Study , 1995 .

[12]  Juan Julián Merelo Guervós,et al.  Evolving RBF neural networks for time-series forecasting with EvRBF , 2004, Inf. Sci..

[13]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[14]  Tomoko Matsui,et al.  A Kernel for Time Series Based on Global Alignments , 2006, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[15]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[16]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[17]  Erkki Oja,et al.  Time series prediction competition: The CATS benchmark , 2007, Neurocomputing.