Dynamic support vector machines for non-stationary time series forecasting

This paper proposes a modified version of support vector machines (SVMs), called dynamic support vector machines (DSVMs), to model non-stationary time series. The DSVMs are obtained by incorporating the problem domain knowledge -- non-stationarity of time series into SVMs. Unlike the standard SVMs which use fixed values of the regularization constant and the tube size in all the training data points, the DSVMs use an exponentially increasing regularization constant and an exponentially decreasing tube size to deal with structural changes in the data. The dynamic regularization constant and tube size are based on the prior knowledge that in the non-stationary time series recent data points could provide more important information than distant data points. In the experiment, the DSVMs are evaluated using both simulated and real data sets. The simulation shows that the DSVMs generalize better than the standard SVMs in forecasting non-stationary time series. Another advantage of this modification is that the DSVMs use fewer support vectors, resulting in a sparser representation of the solution.

[1]  Francis Eng Hock Tay,et al.  Improved financial time series forecasting by combining Support Vector Machines with self-organizing feature map , 2001, Intell. Data Anal..

[2]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[3]  Isabelle Guyon,et al.  Automatic Capacity Tuning of Very Large VC-Dimension Classifiers , 1992, NIPS.

[4]  W. G. Gilchrist,et al.  Methods of Estimation Involving Discounting , 1967 .

[5]  Shouhong Wang,et al.  An Adaptive Approach to Market Development Forecasting , 1999, Neural Computing & Applications.

[6]  Timo Teräsvirta,et al.  Forecasting the consumption of alcoholic beverages in Finland: A box-Jenkins approach , 1976 .

[7]  Vladimir Vapnik,et al.  The Support Vector Method , 1997, ICANN.

[8]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[9]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[10]  Marie Cottrell,et al.  Neural modeling for time series: A statistical stepwise method for weight elimination , 1995, IEEE Trans. Neural Networks.

[11]  David Horn,et al.  Learning the Rule of a Time Series , 1992, Int. J. Neural Syst..

[12]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[13]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[14]  William A. Barnett,et al.  Dynamic Econometric Modeling , 1988 .

[15]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[16]  Claas de Groot,et al.  Analysis of univariate time series with connectionist nets: A case study of two classical examples , 1991, Neurocomputing.

[17]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[18]  Robert Goodell Brown,et al.  Smoothing, forecasting and prediction of discrete time series , 1964 .

[19]  Klaus-Robert Müller,et al.  Annealed Competition of Experts for a Segmentation and Classification of Switching Dynamics , 1996, Neural Computation.

[20]  Francis Eng Hock Tay,et al.  A comparative study of saliency analysis and genetic algorithm for feature selection in support vector machines , 2001, Intell. Data Anal..

[21]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[22]  Masanao Aoki,et al.  State Space Modeling of Time Series , 1987 .

[23]  J. Dziechciarz,et al.  Changing and Random Coefficient Models: A Survey , 1989 .

[24]  Lutgarde M. C. Buydens,et al.  Using support vector machines for time series prediction , 2003 .

[25]  Francis Eng Hock Tay,et al.  Financial Forecasting Using Support Vector Machines , 2001, Neural Computing & Applications.

[26]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[27]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[28]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[29]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[30]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[31]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[32]  Gustavo Deco,et al.  Non-parametric Data Selection for Neural Learning in Non-stationary Time Series , 1997, Neural Networks.

[33]  Junhong Nie.  Nonlinear time-series forecasting: A fuzzy-neural approach , 1997, Neurocomputing.

[34]  Francis Eng Hock Tay,et al.  Modified support vector machines in financial time series forecasting , 2002, Neurocomputing.

[35]  Joydeep Ghosh,et al.  Structurally adaptive modular networks for nonstationary environments , 1999, IEEE Trans. Neural Networks.

[36]  Ruy Luiz Milidiú,et al.  Time-series forecasting through wavelets transformation and a mixture of expert models , 1999, Neurocomputing.

[37]  Krzysztof J. Cios,et al.  Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model , 1996, Neurocomputing.

[38]  A. Harvey.  Time series models , 1983 .

[39]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[40]  Derek W. Bunn,et al.  Financial time series modelling with discounted least squares backpropagation , 1997, Neurocomputing.

[41]  Klaus-Robert Müller,et al.  Hidden Markov gating for prediction of change points in switching dynamical systems , 1999, ESANN.

[42]  Zoran Obradovic,et al.  Regime signaling techniques for non-stationary time series forecasting , 1997, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[43]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[44]  Ian T. Nabney,et al.  Modelling financial time series with switching state space models , 1999, Proceedings of the IEEE/IAFE 1999 Conference on Computational Intelligence for Financial Engineering (CIFEr) (IEEE Cat. No.99TH8408).

[45]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[46]  Gunnar Rätsch,et al.  Using support vector machines for time series prediction , 1999 .