Stochastic Nonlinear Time Series Forecasting Using Time-Delay Reservoir Computers: Performance and Universality

Reservoir computing is a recently introduced machine learning paradigm that has already shown excellent performances in the processing of empirical data. We study a particular kind of reservoir computers called time-delay reservoirs that are constructed out of the sampling of the solution of a time-delay differential equation and show their good performance in the forecasting of the conditional covariances associated to multivariate discrete-time nonlinear stochastic processes of VEC-GARCH type as well as in the prediction of factual daily market realized volatilities computed with intraday quotes, using as training input daily log-return series of moderate size. We tackle some problems associated to the lack of task-universality for individually operating reservoirs and propose a solution based on the use of parallel arrays of time-delay reservoirs.

[1]  Esther Ruiz,et al.  Bootstrap prediction intervals in state–space models , 2009 .

[2]  Juan Romo,et al.  Effects of parameter estimation on prediction densities: a bootstrap approach , 1999 .

[3]  Eric R. Ziegel,et al.  Analysis of Financial Time Series , 2002, Technometrics.

[4]  Juan Romo,et al.  Bootstrap predictive inference for ARIMA processes , 2004 .

[5]  Stéphane Chrétien,et al.  Multivariate GARCH Estimation via a Bregman-Proximal Trust-Region Method , 2011, Comput. Stat. Data Anal..

[6]  L Pesquera,et al.  Photonic information processing beyond Turing: an optoelectronic implementation of reservoir computing. , 2012, Optics express.

[7]  Paul A. Fishwick,et al.  Time series forecasting using neural networks vs. Box- Jenkins methodology , 1991, Simul..

[8]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[9]  Francis wyffels,et al.  Using reservoir computing in a decomposition approach for time series prediction , 2008 .

[10]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[11]  Juan-Pablo Ortega,et al.  Hybrid Forecasting with Estimated Temporally Aggregated Linear Processes , 2012 .

[12]  Benjamin Schrauwen,et al.  Information Processing Capacity of Dynamical Systems , 2012, Scientific Reports.

[13]  Neil Shephard,et al.  Realising the future: forecasting with high frequency based volatility (HEAVY) models , 2010 .

[14]  Neil Shephard,et al.  Multivariate High-Frequency-Based Volatility (HEAVY) Models , 2012 .

[15]  C. Gouriéroux.  ARCH Models and Financial Applications , 1997 .

[16]  Thomas Kolarik,et al.  Time series forecasting using neural networks , 1994, APL '94.

[17]  Francis X. Diebold,et al.  Modeling and Forecasting Realized Volatility , 2001 .

[18]  Herbert Jaeger,et al.  Optimization and applications of echo state networks with leaky- integrator neurons , 2007, Neural Networks.

[19]  A. Tikhonov.  On the stability of inverse problems , 1943 .

[20]  Jeroen V.K. Rombouts,et al.  Série Scientifique Scientific Series on Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models on Loss Functions and Ranking Forecasting Performances of Multivariate Volatility Models , 2022 .

[21]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[22]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[23]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[24]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[25]  Benjamin Schrauwen,et al.  An experimental unification of reservoir computing methods , 2007, Neural Networks.

[26]  José Manuel Gutiérrez,et al.  Simple reservoirs with chain topology based on a single time-delay nonlinear node , 2012, ESANN.

[27]  L. Appeltant,et al.  Information processing using a single dynamical node as complex system , 2011, Nature communications.

[28]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[29]  Benjamin Schrauwen,et al.  Optoelectronic Reservoir Computing , 2011, Scientific Reports.

[30]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[31]  Leon O. Chua,et al.  Fading memory and the problem of approximating nonlinear operators with volterra series , 1985 .

[32]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[33]  José Manuel Gutiérrez,et al.  Memory and Nonlinear Mapping in Reservoir Computing with Two Uncoupled Nonlinear Delay Nodes , 2013 .

[34]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[35]  Serge Massar,et al.  All-optical Reservoir Computing , 2012, Optics express.

[36]  Fred Collopy,et al.  How effective are neural networks at forecasting and prediction? A review and evaluation , 1998 .

[37]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[38]  Wolfgang Maass,et al.  Liquid State Machines: Motivation, Theory, and Applications , 2010 .

[39]  Sven Leyffer,et al.  Constrained Nonlinear Programming for Volatility Estimation with GARCH Models , 2003, SIAM Rev..

[40]  Milton S. Boyd,et al.  Designing a neural network for forecasting financial and economic time series , 1996, Neurocomputing.

[41]  S.F. Crone,,et al.  Stepwise Selection of Artificial Neural Network Models for Time Series Prediction , 2005 .

[42]  Christian Gouriéroux,et al.  Some Applications of Univariate ARCH Models , 1997 .

[43]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[44]  Eduardo D. Sontag,et al.  Neural Systems as Nonlinear Filters , 2000, Neural Computation.

[45]  Juan Romo,et al.  Bootstrap prediction for returns and volatilities in GARCH models , 2006, Comput. Stat. Data Anal..

[46]  J. Wooldridge,et al.  A Capital Asset Pricing Model with Time-Varying Covariances , 1988, Journal of Political Economy.

[47]  Mario Bertero,et al.  The Stability of Inverse Problems , 1980 .

[48]  E. Ruiz,et al.  Bootstrapping Financial Time Series , 2002 .

[49]  Benjamin Schrauwen,et al.  A comparative study of Reservoir Computing strategies for monthly time series prediction , 2010, Neurocomputing.

[50]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[51]  Juan Romo,et al.  Bootstrap prediction intervals for power-transformed time series , 2005 .

[52]  H. Jaeger,et al.  Stepping forward through echoes of the past : forecasting with Echo State Networks , 2007 .

[53]  James D. Hamilton.  Time Series Analysis , 1994 .

[54]  S. Barry Cooper,et al.  Computability In Context: Computation and Logic in the Real World , 2009 .

[55]  Daniel Brunner,et al.  Parallel photonic information processing at gigabyte per second data rates using transient states , 2013, Nature Communications.

[56]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[57]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[58]  R. Engle.  Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[59]  Nigel Crook.  Nonlinear transient computation , 2007, Neurocomputing.

[60]  J. Keith Ord,et al.  Automatic neural network modeling for univariate time series , 2000 .

[61]  S. Laurent,et al.  On the Forecasting Accuracy of Multivariate GARCH Models , 2010 .