Eugenic Neuro-Evolution for Reinforcement Learning

In this paper we introduce EuSANE, a novel reinforcement learning algorithm based on the SANE neuro-evolution method. It uses a global genetic search algorithm, the Eugenic Algorithm, to optimize the selection of neurons to the hidden layer of SANE networks. The performance of EuSANE is evaluated in the 2-pole-balancing benchmark task. EuSANE is several times faster than SANE in this task, showing that it is a highly efficient method of reinforcement learning in challenging domains.

[1]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[2]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[3]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[4]  Alexis P. Wieland,et al.  Evolving Controls for Unstable Systems , 1991 .

[5]  David E. Moriarty,et al.  Symbiotic Evolution of Neural Networks in Sequential Decision Tasks , 1997 .

[6]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[7]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[8]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[9]  L. Darrell Whitley,et al.  Cellular Encoding Applied to Neurocontrol , 1995, ICGA.

[10]  Daniel Polani,et al.  Training Kohonen Feature Maps in Different Topologies: An Analysis Using Genetic Algorithms , 1993, ICGA.

[11]  Gilbert Syswerda,et al.  Simulated Crossover in Genetic Algorithms , 1992, FOGA.

[12]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[13]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[14]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[15]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Daniel Polani,et al.  Adaptation of Kohonen Feature Map Topologies by Genetic Algorithms , 1992, PPSN.