Multi Reservoir Support Vector Echo State Machine for Multivariate Time Series Prediction

Chaotic time series prediction has received considerable attention in the last few years. Although many studies have been conducted in the field, there is little attention focused on multivariate time series prediction. Considering this problem, a multi reservoir support vector echo state machine(MRSVESM) based on multi kernel learning and echo state networks is proposed in this paper. The single reservoir approach may be ineffective on multivariate time series prediction, as it is not able to character multi time scale dynamics. The MRSVESM use multi different time scale reservoirs to present the dynamics of multivariate time series and replaced the "kernel trick" with "reservoir trick", that is, performed multi kernel learning in the high dimension "reservoir" state space. Two simulation examples, prediction of Lorenz chaotic time series and prediction of sunspots and the Yellow River annual runoff time series are conducted to demonstrate the effectiveness of the proposed method.

[1]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[2]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[3]  A. Mees,et al.  Dynamics from multivariate time series , 1998 .

[4]  Fernando José Von Zuben,et al.  An extended echo state network using Volterra filtering and principal component analysis , 2012, Neural Networks.

[5]  Barbara Caputo,et al.  Multi Kernel Learning with Online-Batch Optimization , 2012, J. Mach. Learn. Res..

[6]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[7]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Danilo P. Mandic,et al.  An Augmented Echo State Network for Nonlinear Adaptive Filtering of Complex Noncircular Signals , 2011, IEEE Transactions on Neural Networks.

[9]  Benjamin Schrauwen,et al.  Recurrent Kernel Machines: Computing with Infinite Echo State Networks , 2012, Neural Computation.

[10]  Yves Grandvalet,et al.  More efficiency in multiple kernel learning , 2007, ICML '07.

[11]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[12]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[13]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[14]  D. Prokhorov,et al.  Echo state networks: appeal and challenges , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[15]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[17]  Sheng Tang,et al.  Localized Multiple Kernel Learning for Realistic Human Action Recognition in Videos , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  G. Sugihara,et al.  Generalized Theorems for Nonlinear State Space Reconstruction , 2011, PloS one.

[19]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[20]  Yiannis Demiris,et al.  Echo State Gaussian Process , 2011, IEEE Transactions on Neural Networks.

[21]  G. Venayagamoorthy,et al.  Neural networks letter Effects of spectral radius and settling time in the performance of echo state networks , 2009 .

[22]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[23]  Sergios Theodoridis,et al.  Adaptive Multiregression in Reproducing Kernel Hilbert Spaces: The Multiaccess MIMO Channel Case , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Vincenzo Piuri,et al.  Hierarchical Approach for Multiscale Support Vector Regression , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[26]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[27]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[28]  Arta A. Jamshidi,et al.  Modeling Multivariate Time Series on Manifolds with Skew Radial Basis Functions , 2011, Neural Computation.

[29]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[30]  T. Lane,et al.  A Framework for Multiple Kernel Support Vector Regression and Its Applications to siRNA Efficacy Prediction , 2009, TCBB.