Multi-step-ahead chaotic time series prediction using coevolutionary recurrent neural networks

Multi-step-ahead time series prediction has been one of the greatest challenges for machine learning. Recurrent neural networks (RNN) can efficiently model temporal sequences and have been promising for multi-step time series prediction. Cooperative neuro-evolution has been used for training RNNs with promising performance for single step ahead time series prediction. This paper employs cooperative neuro-evolution of RNNs for multi-step ahead prediction. The RNN recursively predicts the next values in the horizon where the output from the single-step ahead prediction are the input for predicting the next value in the horizon. The performance of cooperative neuro-evolution is compared with back-propagation through time (BPTT) learning algorithm. The results are promising which shows that cooperative neuro-evolution performs better compared to BPTT for most cases.

[1]  Kalyanmoy Deb,et al.  A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization , 2002, Evolutionary Computation.

[2]  Amir F. Atiya,et al.  A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition , 2011, Expert Syst. Appl..

[3]  Amir F. Atiya,et al.  Multi-step-ahead prediction using dynamic recurrent neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4]  M. Gardner,et al.  Neural network modelling and prediction of hourly NOx and NO2 concentrations in urban air in London , 1999 .

[5]  Rohitash Chandra.  Adaptive problem decomposition in cooperative coevolution of recurrent networks for time series prediction , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[6]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[7]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[8]  Floris Takens,et al.  On the numerical determination of the dimension of an attractor , 1985 .

[9]  Mengjie Zhang,et al.  Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction , 2012, Neurocomputing.

[10]  Rob J. Hyndman,et al.  Recursive and direct multi-step forecasting: the best of both worlds , 2012 .

[11]  Yen-Ming Chiang,et al.  Multi-step-ahead neural networks for flood forecasting , 2007 .

[12]  Mohamed Chtourou,et al.  A hybrid approach for training recurrent neural networks: application to multi-step-ahead prediction of noisy and large data sets , 2007, Neural Computing and Applications.

[13]  Amir F. Atiya,et al.  A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[14]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[15]  D. Kugiumtzis.  State space reconstruction parameters in the analysis of chaotic time series—the role of the time window length , 1996, comp-gas/9602002.

[16]  M. Crucianu,et al.  Multi-step-ahead Prediction with Neural Networks : a Review , 2002 .

[17]  Liang Li,et al.  Nonlinear adaptive prediction of nonstationary signals , 1995, IEEE Trans. Signal Process..

[18]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[19]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[20]  C. Rasmussen,et al.  Gaussian Process Priors with Uncertain Inputs - Application to Multiple-Step Ahead Time Series Forecasting , 2002, NIPS.

[21]  C. Lee Giles,et al.  Extraction of rules from discrete-time recurrent neural networks , 1996, Neural Networks.

[22]  K. Obermayer,et al.  Multiple-step ahead prediction for non linear dynamic systems: A Gaussian Process treatment with propagation of the uncertainty , 2003, NIPS 2003.

[23]  Jing Gao,et al.  Multistep-Ahead Time Series Prediction , 2006, PAKDD.

[24]  Zhongyi Hu,et al.  Multi-step-ahead time series prediction using multiple-output support vector regression , 2014, Neurocomputing.

[25]  Li-Chiu Chang,et al.  Reinforced Two-Step-Ahead Weight Adjustment Technique for Online Training of Recurrent Neural Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Rohitash Chandra,et al.  Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[28]  H. B. Sandya,et al.  Feature extraction, classification and forecasting of time series signal using fuzzy and GARCH techniques , 2013 .

[29]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[30]  Peng Hong,et al.  Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network , 2008 .

[31]  Mengjie Zhang,et al.  On the issue of separability for problem decomposition in cooperative neuro-evolution , 2012, Neurocomputing.

[32]  Pei-Chann Chang,et al.  Iterated time series prediction with multiple support vector regression models , 2013, Neurocomputing.

[33]  Antti Sorjamaa,et al.  Multiple-output modeling for multi-step-ahead time series forecasting , 2010, Neurocomputing.