Multiobjective Evolutionary Optimization of Training and Topology of Recurrent Neural Networks for Time-Series Prediction

This paper provides a new evolutionary multiobjective optimization method for automatically optimizing the network topology of recurrent neural networks. The feature of the proposed method is that it involves a procedure of intensively exploring a region including solutions with small training errors in the Pareto frontier, instead of finding a whole set of the Pareto optimal solutions. Several numerical experiments are executed in order to show the advantage of the proposed method over the existing effective algorithm by Delgado et al. with respect to the capability of time-series prediction.

[1]  D. Thierens.  Adaptive mutation rate control schemes in genetic algorithms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  César Hervás-Martínez,et al.  COVNET: a cooperative coevolutionary model for evolving artificial neural networks , 2003, IEEE Trans. Neural Networks.

[3]  Tarik A. Rashid,et al.  Multi-Context Recurrent Neural Network for Time Series Applications , 2007 .

[4]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[5]  Nian Zhang,et al.  Time series prediction with recurrent neural networks trained by a hybrid PSO-EA algorithm , 2004, Neurocomputing.

[6]  M. Ogorzalek,et al.  Time series prediction with ensemble models , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[8]  A. Nebot,et al.  Multi-resolution time-series prediction using fuzzy inductive reasoning , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[9]  Wan-Chi Siu,et al.  Application of a fast real time recurrent learning algorithm to text-to-phoneme conversion , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[11]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[12]  José Carlos Príncipe,et al.  A Theory for Neural Networks with Time Delays , 1990, NIPS.

[13]  Manuel P. Cuéllar,et al.  Multiobjective Hybrid Optimization and Training of Recurrent Neural Networks , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Jonathan E. Fieldsend,et al.  Pareto evolutionary neural networks , 2005, IEEE Transactions on Neural Networks.

[15]  Thomas Bäck,et al.  Intelligent Mutation Rate Control in Canonical Genetic Algorithms , 1996, ISMIS.

[16]  Armando Blanco,et al.  A genetic algorithm to obtain the optimal recurrent neural network , 2000, Int. J. Approx. Reason..

[17]  Erkki Oja,et al.  Time series prediction competition: The CATS benchmark , 2007, Neurocomputing.

[18]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[19]  Manoj Kumar Tiwari,et al.  Interactive Particle Swarm: A Pareto-Adaptive Metaheuristic to Multiobjective Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Ioannis B. Theocharis,et al.  Locally recurrent neural networks for wind speed prediction using spatial correlation , 2007, Inf. Sci..

[21]  Padhraic Smyth,et al.  Discrete recurrent neural networks for grammatical inference , 1994, IEEE Trans. Neural Networks.

[22]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..