Evolving Modular Fast-Weight Networks for Control

In practice, almost all control systems in use today implement some form of linear control. However, there are many tasks for which conventional control engineering methods are not directly applicable because there is not enough information about how the system should be controlled (i.e. reinforcement learning problems). In this paper, we explore an approach to such problems that evolves fast-weight neural networks. These networks, although capable of implementing arbitrary non-linear mappings, can more easily exploit the piecewise linearity inherent in most systems, in order to produce simpler and more comprehensible controllers. The method is tested on 2D mobile robot version of the pole balancing task where the controller must learn to switch between two operating modes, one using a single pole and the other using a jointed pole version that has not before been solved.

[1]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[2]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[3]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[4]  Risto Miikkulainen,et al.  Efficient Non-linear Control Through Neuroevolution , 2006, ECML.

[5]  Geoffrey E. Hinton.  Using fast weights to deblur old memories , 1987 .

[6]  Jürgen Schmidhuber,et al.  Evolino: Hybrid Neuroevolution/Optimal Linear Search for Sequence Learning , 2005, IJCAI.

[7]  Risto Miikkulainen,et al.  Transfer of Neuroevolved Controllers in Unstable Domains , 2004, GECCO.

[8]  Jürgen Schmidhuber,et al.  Modeling systems with internal state using evolino , 2005, GECCO '05.

[9]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[10]  Jürgen Schmidhuber,et al.  Evolino for recurrent support vector machines , 2005, ESANN.

[11]  Jürgen Schmidhuber,et al.  Co-evolving recurrent neurons learn deep memory POMDPs , 2005, GECCO '05.

[12]  Steven Seidman,et al.  A synthesis of reinforcement learning and robust control theory , 2000 .

[13]  Risto Miikkulainen,et al.  2-D Pole Balancing with Recurrent Evolutionary Networks , 1998 .

[14]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[15]  J. Urgen Schmidhuber.  Learning to Control Fast-weight Memories: an Alternative to Dynamic Recurrent Networks , 1991 .

[16]  Jürgen Schmidhuber,et al.  A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Risto Miikkulainen,et al.  A neuro-evolution method for dynamic resource allocation on a chip multiprocessor , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[18]  Wilson J. Rugh,et al.  Research on gain scheduling , 2000, Autom..

[19]  Jürgen Schmidhuber,et al.  Metric State Space Reinforcement Learning for a Vision-Capable Mobile Robot , 2006, IAS.