Support vector machines experts for time series forecasting

Abstract This paper proposes using the support vector machines (SVMs) experts for time series forecasting. The generalized SVMs experts have a two-stage neural network architecture. In the first stage, self-organizing feature map (SOM) is used as a clustering algorithm to partition the whole input space into several disjointed regions. A tree-structured architecture is adopted in the partition to avoid the problem of predetermining the number of partitioned regions. Then, in the second stage, multiple SVMs, also called SVM experts, that best fit partitioned regions are constructed by finding the most appropriate kernel function and the optimal free parameters of SVMs. The sunspot data, Santa Fe data sets A, C and D, and the two building data sets are evaluated in the experiment. The simulation shows that the SVMs experts achieve significant improvement in the generalization performance in comparison with the single SVMs models. In addition, the SVMs experts also converge faster and use fewer support vectors.

[1]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[2]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[3]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[4]  Lutgarde M. C. Buydens,et al.  Using support vector machines for time series prediction , 2003 .

[5]  Ke Chen,et al.  Combining linear discriminant functions with neural networks for supervised learning , 2005, Neural Computing & Applications.

[6]  Ashok N. Srivastava,et al.  Nonlinear gated experts for time series: discovering regimes and avoiding overfitting , 1995, Int. J. Neural Syst..

[7]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[8]  Claas de Groot,et al.  Analysis of univariate time series with connectionist nets: A case study of two classical examples , 1991, Neurocomputing.

[9]  Ke Chen,et al.  A self-generating modular neural network architecture for supervised learning , 1997, Neurocomputing.

[10]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[11]  J. Tin-Yau Kwok.  Support vector mixture for classification and regression problems , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[12]  Francis Eng Hock Tay,et al.  Improved financial time series forecasting by combining Support Vector Machines with self-organizing feature map , 2001, Intell. Data Anal..

[13]  Ruy Luiz Milidiú,et al.  Time-series forecasting through wavelets transformation and a mixture of expert models , 1999, Neurocomputing.

[14]  Krzysztof J. Cios,et al.  Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model , 1996, Neurocomputing.

[15]  F. Girosi,et al.  Nonlinear prediction of chaotic time series using support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[16]  Francis Eng Hock Tay,et al.  Financial Forecasting Using Support Vector Machines , 2001, Neural Computing & Applications.

[17]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[18]  Sylvie Thiria,et al.  Sea surface temperature forecasts using on-line local learning algorithm in upwelling regions , 2000, Neurocomputing.

[19]  Gunnar Rätsch,et al.  Using support vector machines for time series prediction , 1999 .

[20]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[21]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[22]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[23]  David Horn,et al.  Learning the Rule of a Time Series , 1992, Int. J. Neural Syst..

[24]  Junhong Nie.  Nonlinear time-series forecasting: A fuzzy-neural approach , 1997, Neurocomputing.

[25]  Francis Eng Hock Tay,et al.  Modified support vector machines in financial time series forecasting , 2002, Neurocomputing.

[26]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[27]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[28]  F. Tay,et al.  Application of support vector machines in financial time series forecasting , 2001 .

[29]  Francis Eng Hock Tay,et al.  A comparative study of saliency analysis and genetic algorithm for feature selection in support vector machines , 2001, Intell. Data Anal..

[30]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[31]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[32]  Marie Cottrell,et al.  Neural modeling for time series: A statistical stepwise method for weight elimination , 1995, IEEE Trans. Neural Networks.

[33]  Klaus-Robert Müller,et al.  Annealed Competition of Experts for a Segmentation and Classification of Switching Dynamics , 1996, Neural Computation.

[34]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .