Efficient Evolution of Asymetric Recurrent Neural Networks Using a Two-dimensional Representation

Recurrent neural networks are particularly useful for proc essing time sequences and simulating dynamical systems. However, methods for building r ecu rent architectures have been hindered by the fact that available training algorithms are considerably more complex than those for feedforward networks. In this paper, we present a new met hod to build recurrent neural networks based on evolutionary computation, which combine s a linear chromosome with a twodimensional representation inspired by Parallel Distribu ed Genetic Programming (a form of genetic programming for the evolution of graph-like progra ms) to evolve the architecture and the weights simultaneously. Our method can evolve general asym etric recurrent architectures as well as specialized recurrent architectures. This paper descri bes the method and reports on results of its application.

[1]  Hava T. Siegelmann,et al.  Computational capabilities of recurrent NARX neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[3]  Hans-Paul Schwefel,et al.  Evolutionary Programming and Evolution Strategies: Similarities and Differences , 1993 .

[4]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[5]  C. L. Giles,et al.  Pruning recurrent neural networks for improved generalization performance , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[6]  Donald E. Waagen,et al.  Evolving recurrent perceptrons for time-series modeling , 1994, IEEE Trans. Neural Networks.

[7]  Richard J. Duro,et al.  Evolutionary generation and training of recurrent artificial neural networks , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[8]  A. Rosenfeld,et al.  IEEE TRANSACTIONS ON SYSTEMS , MAN , AND CYBERNETICS , 2022 .

[9]  Mats G. Nordahl,et al.  Evolving Recurrent Neural Networks , 1993 .

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Riccardo Poli,et al.  Some Steps Towards a Form of Parallel Distributed Genetic Programming , 1996 .

[12]  R. Poli,et al.  Discovery of Symbolic, Neuro-Symbolic and Neural Networks with Parallel Distributed Genetic Programming , 1997, ICANNGA.

[13]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[14]  Riccardo,et al.  Evolution of the Topology and the Weightsof Neural Networks using GeneticProgramming with a Dual RepresentationJo , 1997 .

[15]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[16]  M. Mandischer.  Evolving recurrent neural networks with non-binary encoding , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[17]  Stefan Bornholdt,et al.  General asymmetric neural networks and structure design by genetic algorithms: a learning rule for temporal patterns , 1992, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[18]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[19]  Peter Tiño,et al.  Learning long-term dependencies in NARX recurrent neural networks , 1996, IEEE Trans. Neural Networks.

[20]  D.R. Hush,et al.  Progress in supervised neural networks , 1993, IEEE Signal Processing Magazine.

[21]  Xin Yao,et al.  Evolving Artificial Neural Networks through Evolutionary Programming , 1996, Evolutionary Programming.

[22]  Francisco Sandoval Hernández,et al.  Genetic Synthesis of Discrete-Time Recurrent Neural Network , 1993, IWANN.

[23]  Antonette M. Logar,et al.  A comparison of recurrent neural network learning algorithms , 1993, IEEE International Conference on Neural Networks.

[24]  Byoung-Tak Zhang,et al.  Genetic Programming of Minimal Neural Nets Using Occam's Razor , 1993, ICGA.

[25]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.