Multiobjective Evolutionary Optimization of Training and Topology of Recurrent Neural Networks for Time-Series Prediction

This paper provides a new evolutionary multiobjective optimization method for automatically optimizing the network topology of recurrent neural networks. The feature of the proposed method is that it involves a procedure of intensively exploring a region including solutions with small training errors in the Pareto frontier, instead of finding a whole set of the Pareto optimal solutions. Several numerical experiments are executed in order to show the advantage of the proposed method over the existing effective algorithm by Delgado et al. with respect to the capability of time-series prediction.

[1]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[2]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[3]  Manoj Kumar Tiwari,et al.  Interactive Particle Swarm: A Pareto-Adaptive Metaheuristic to Multiobjective Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[4]  Wan-Chi Siu,et al.  Application of a fast real time recurrent learning algorithm to text-to-phoneme conversion , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  M. Ogorzalek,et al.  Time series prediction with ensemble models , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[8]  Thomas Bäck,et al.  Intelligent Mutation Rate Control in Canonical Genetic Algorithms , 1996, ISMIS.

[9]  Manuel P. Cuéllar,et al.  Multiobjective Hybrid Optimization and Training of Recurrent Neural Networks , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[11]  Jonathan E. Fieldsend,et al.  Pareto evolutionary neural networks , 2005, IEEE Transactions on Neural Networks.

[12]  Padhraic Smyth,et al.  Discrete recurrent neural networks for grammatical inference , 1994, IEEE Trans. Neural Networks.

[13]  José Carlos Príncipe,et al.  A Theory for Neural Networks with Time Delays , 1990, NIPS.

[14]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[15]  D. Wunsch,et al.  Time series prediction with recurrent neural networks using a hybrid PSO-EA algorithm , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[16]  A. Nebot,et al.  Multi-resolution time-series prediction using fuzzy inductive reasoning , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[17]  Armando Blanco,et al.  A genetic algorithm to obtain the optimal recurrent neural network , 2000, Int. J. Approx. Reason..

[18]  Erkki Oja,et al.  Time series prediction competition: The CATS benchmark , 2007, Neurocomputing.

[19]  D. Thierens.  Adaptive mutation rate control schemes in genetic algorithms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[20]  Tarik A. Rashid,et al.  Multi-Context Recurrent Neural Network for Time Series Applications , 2007 .

[21]  Ioannis B. Theocharis,et al.  Locally recurrent neural networks for wind speed prediction using spatial correlation , 2007, Inf. Sci..