A ovel Parameters Selection Approach for Support Vector Machines to Predict Time Series

Aimed to solve the problem that there is no structural approach to select the best free parameters for Support Vector Machines when being used in time series prediction, a novel approach is proposed. In this method, best parameters for SVM is not got at the minimal MSE (Mean Squared Error) of validation set, but that the residue of training set is in White Noise form. This conclusion is deduced from the fact that the targets of training set have inherent correlations with each other. This approach is also effective to predict time series with nolinear and non-stationary characteristics. Furthermore, by using this method, confidence interval can be computed under any given confidence degree 1 - alpha which is an important value for many applications. Two algorithms to compute confidence interval are given under different circumstances. Program is also given about how to make dynamic on-line prediction. Experiment was made to predict the annual sunspot number and perfect result was achieved.

[1]  Ravi Sankar,et al.  Time Series Prediction Using Support Vector Machines: A Survey , 2009, IEEE Computational Intelligence Magazine.

[2]  Vladimir Cherkassky,et al.  Learning from Data: Concepts, Theory, and Methods , 1998 .

[3]  R. C. Williamson,et al.  Support vector regression with automatic accuracy control. , 1998 .

[4]  Quan Min Zhu,et al.  A Correlation-Test-Based Validation Procedure for Identified Neural Networks , 2009, IEEE Transactions on Neural Networks.

[5]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[6]  S. A. Billings,et al.  Multi-directional model validity tests for non-linear system identification , 2000 .

[7]  Quan Min Zhu,et al.  A set of novel correlation tests for nonlinear system variables , 2007, Int. J. Syst. Sci..

[8]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[9]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[10]  James T. Kwok.  Linear Dependency between epsilon and the Input Noise in epsilon-Support Vector Regression , 2001, ICANN.

[11]  James A. Bucklew,et al.  Support vector machine techniques for nonlinear equalization , 2000, IEEE Trans. Signal Process..

[12]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[13]  Kurt Hornik,et al.  Artificial Neural Networks — ICANN 2001 , 2001, Lecture Notes in Computer Science.

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.

[17]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[18]  Massimiliano Versace,et al.  Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networks , 2004, Expert Syst. Appl..

[19]  Liva Ralaivola,et al.  Dynamical Modeling with Kernels for Nonlinear Time Series Prediction , 2003, NIPS.

[20]  Lijuan Cao,et al.  Support vector machines experts for time series forecasting , 2003, Neurocomputing.

[21]  J. Weidong.  Selection of Kernel Functions and Parameters for Support Vector Machines in System Identification , 2006 .

[22]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[23]  Quan Min Zhu,et al.  Development of omni-directional correlation functions for nonlinear model validation , 2007, Autom..

[24]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[25]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[26]  Simon Haykin,et al.  Support vector machines for dynamic reconstruction of a chaotic system , 1999 .

[27]  H. Tong,et al.  Threshold Autoregression, Limit Cycles and Cyclical Data , 1980 .

[28]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[29]  J.T. Kwok,et al.  Linear Dependency betweenand the Input Noise in -Support Vector Regression , 2001 .

[30]  Gunnar Rätsch,et al.  Using support vector machines for time series prediction , 1999 .

[31]  B. Schölkopf,et al.  Asymptotically Optimal Choice of ε-Loss for Support Vector Machines , 1998 .

[32]  Stephen A. Billings,et al.  Nonlinear model validation using correlation tests , 1994 .

[33]  Irene Yu-Hua Gu,et al.  Support Vector Machine for Classification of Voltage Disturbances , 2007, IEEE Transactions on Power Delivery.