Time series prediction with simple recurrent neural networks

Simple recurrent neural networks are widely used in time series prediction. Most researchers and application developers often choose arbitrarily between Elman or Jordan simple recurrent neural networks for their applications. A hybrid of the two called Elman-Jordan (or Multi-recurrent) neural network is also being used. In this study, we evaluated the performance of these neural networks on three established bench mark time series prediction problems. Results from the experiments showed that Jordan neural network performed significantly better than the others. However, the results indicated satisfactory forecasting performance by the other two neural networks. Key Words : Time Series Prediction, Artificial Neural Network, Recurrent NN, Resilient Propagation.

[1]  Martin A. Riedmiller,et al.  Rprop - Description and Implementation Details , 1994 .

[2]  Grgoire Montavon,et al.  Neural Networks: Tricks of the Trade , 2012, Lecture Notes in Computer Science.

[3]  Paul A. Fishwick,et al.  Time series forecasting using neural networks vs. Box- Jenkins methodology , 1991, Simul..

[4]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[5]  Andries Petrus Engelbrecht,et al.  CIlib: A collaborative framework for Computational Intelligence algorithms - Part II , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[6]  Robert M. Farber,et al.  How Neural Nets Work , 1987, NIPS.

[7]  Ah Chung Tsoi,et al.  Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference , 2001, Machine Learning.

[8]  Kouichi Murakami,et al.  Gesture recognition using recurrent neural networks , 1991, CHI.

[9]  Michael Y. Hu,et al.  A simulation study of artificial neural networks for nonlinear time-series forecasting , 2001, Comput. Oper. Res..

[10]  Lars Kai Hansen,et al.  Adaptive Regularization in Neural Network Modeling , 2012, Neural Networks: Tricks of the Trade.

[11]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[12]  Guoqiang Peter Zhang,et al.  Trend time series modeling and forecasting with neural networks , 2003, 2003 IEEE International Conference on Computational Intelligence for Financial Engineering, 2003. Proceedings..

[13]  Thomas Kolarik,et al.  Time series forecasting using neural networks , 1994, APL '94.

[14]  J. Elman.  Distributed Representations, Simple Recurrent Networks, And Grammatical Structure , 1991 .

[15]  Ramin Yasdi.  Prediction of Road Traffic using a Neural Network Approach , 1999, Neural Computing & Applications.

[16]  Sandiway Fong,et al.  Natural Language Grammatical Inference with Recurrent Neural Networks , 2000, IEEE Trans. Knowl. Data Eng..

[17]  Qing Song.  Robust Jordan network for nonlinear time series prediction , 2011, The 2011 International Joint Conference on Neural Networks.

[18]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .