A robust reinforcement based self constructing neural network

Usually, many high-skilled human resources are required to create sophisticated control systems. Automatic generation of control systems can overcome these requirements. Because of their versatility and flexibility neural networks gained an important role for this task. While evolutionary methods have been relatively successful in generating neural networks, they have some limitations, in addition to being computationally expensive, because they rely on adapting populations instead of individuals. Reinforcement methods on the other hand can improve and adapt the behaviour of an individual; the reinforcement methods that are presented in this paper can grow a neural network during operation. We show that neural networks can be created for various domains without changing any parameters. Additionally, our neural network can learn the action selection policy and the value function locally within the neurons. These features make our neural network highly flexible and distinguish it from other reinforcement based constructive neural networks.

[1]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[2]  Stefano Nolfi,et al.  Evolutionary robotics , 2014, Evolutionary Intelligence.

[3]  David A. Elizondo,et al.  A Constructive Neural Network for Evolving a Machine Controller in Real-Time , 2009, Constructive Neural Networks.

[4]  András Lörincz,et al.  The many faces of optimism: a unifying approach , 2008, ICML '08.

[5]  Kazuyuki Murase,et al.  A Simple Aplysia-Like Spiking Neural Network to Generate Adaptive Behavior in Autonomous Robots , 2008, Adapt. Behav..

[6]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[7]  Michael Schmitt,et al.  Self-organization of spiking neurons using action potential timing , 1998, IEEE Trans. Neural Networks.

[8]  Philippe Collard,et al.  Artificial Neurogenesis: Applications to the Cart-Pole Problem and to an Autonomous Mobile Robot , 1997, Int. J. Artif. Intell. Tools.

[9]  Daoxiong Gong,et al.  A Growing Cognitive Model of Motor Balance Skill: Architecture, Algorithm and Application1 , 2005 .

[10]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[11]  Tim Callinan.  Artificial Neural Network identification and control of the inverted pendulum , 2003 .

[12]  Adam Ponzi,et al.  Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback , 2008, Neural Networks.

[13]  Mance E. Harmon,et al.  Reinforcement Learning: A Tutorial. , 1997 .

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Andrzej Buller,et al.  Self-motivated Learning Agent-Skill-development in a Growing Network Mediated by Pleasure and Tensions , 2006 .

[16]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[17]  Kazuyuki Murase,et al.  A Spiking Neural Network with dynamic memory for a real autonomous mobile robot in dynamic environment , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[18]  Csaba Szepesvári,et al.  Reinforcement Learning Algorithms for MDPs , 2011 .

[19]  André da Motta Salles Barreto,et al.  Restricted gradient-descent algorithm for value-function approximation in reinforcement learning , 2008, Artif. Intell..

[20]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[21]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.