Evolutionaty training for dynamical recurrent neural networks: an application in finantial time series prediction

Theoretical and experimental studies have shown that traditional training algorithms for Dynamical Recurrent Neural Networks may suffer of local optima solutions, due to the error propagation across the recurrence. In the last years, many researchers have put forward different approaches to solve this problem, most of them being based on heuristic procedures. In this paper, the training capabilities of evolutionary techniques are studied, for Dynamical Recurrent Neural Networks. The performance of the models considered is compared in the experimental section, in real finantial time series prediction problems.

[1]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[2]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[3]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[4]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  J. D. Schaffer,et al.  Real-Coded Genetic Algorithms and Interval-Schemata , 1992, FOGA.

[7]  Jürgen Schmidhuber,et al.  A Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running Networks , 1992, Neural Computation.

[8]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[9]  L. P. J. Veelenturf,et al.  Analysis and applications of artificial neural networks , 1995 .

[10]  Stefan C. Kremer,et al.  On the computational power of Elman-style recurrent networks , 1995, IEEE Trans. Neural Networks.

[11]  Alain Pétrowski,et al.  A clearing procedure as a niching method for genetic algorithms , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[12]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[13]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[14]  Bruno Sareni,et al.  Fitness sharing and niching methods revisited , 1998, IEEE Trans. Evol. Comput..

[15]  Alex Aussem,et al.  Dynamical recurrent neural networks towards prediction and modeling of dynamical systems , 1999, Neurocomputing.

[16]  Armando Blanco,et al.  A genetic algorithm to obtain the optimal recurrent neural network , 2000, Int. J. Approx. Reason..

[17]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[18]  Armando Blanco,et al.  A real-coded genetic algorithm for training recurrent neural networks , 2001, Neural Networks.

[19]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[20]  Marco Tomassini,et al.  Soft computing - integrating evolutionary, neural, and fuzzy systems , 2001 .

[21]  Neil Davey,et al.  Time Series Prediction and Neural Networks , 2001, J. Intell. Robotic Syst..

[22]  L. Schmitt.  Fundamental Study Theory of genetic algorithms , 2001 .

[23]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[24]  Miguel Delgado Calvo-Flores,et al.  Predicción del endeudamiento económico español utilizando modelos bioinspirados , 2003 .

[25]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[26]  María del Carmen Pegalajar Jiménez,et al.  A fir neural network to model the autonomous indebtedness , 2003 .

[27]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[28]  Maria del Carmen Pegalajar Jiménez,et al.  A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference , 2005, Pattern Recognit..

[29]  Pierre Baldi,et al.  Hidden Markov models and neural networks , 2005 .