Automatic synthesis of working memory neural networks with neuroevolution methods

Evolutionary Robotics is a research field focused on autonomous design of robots based on evolutionary algorithms. In this field, neuroevolution methods aims in particular at designing both structure and parameters of neural networks that make a robot exhibit a desired behavior. While such methods have shown their efficiency to generate reactive behaviors, they hardly scale to more cognitive behaviors. One of the reasons of such a limitation might be in the properties of the encoding, i.e. the neural network representation explored by the genetic operators. This work considers EvoNeuro encoding, an encoding directly inspired from computational neuroscience and tests its efficiency on a working memory task, namely the AX-CPT task. Neural networks able to solve this task are generated and compared to neural networks evolved with a simpler direct encoding. The task is solved in both cases, but while direct encodings tend to generate networks whose structure is adapted to a particular instance of AX-CPT, networks generated with EvoNeuro encoding are more versatile and can adapt to the new task through a simple parameter optimization. Such versatile neural networks might facilitate the evolution of robot controllers for tasks requiring a working memory.

[1]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[2]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[3]  Nicolas Tabareau,et al.  Where neuroscience and dynamic system theory meet autonomous robotics: A contracting basal ganglia model for action selection , 2008, Neural Networks.

[4]  Kenneth O. Stanley A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks , 2009 .

[5]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[6]  T. Braver,et al.  A theory of cognitive control, aging cognition, and neuromodulation , 2002, Neuroscience & Biobehavioral Reviews.

[7]  David Zipser,et al.  Recurrent Network Model of the Neural Mechanism of Short-Term Active Memory , 1991, Neural Computation.

[8]  David Zipser,et al.  Sequential memory-guided saccades and target selection: a neural model of the frontal eye fields , 2003, Vision Research.

[9]  T. Sejnowski,et al.  Neurocomputational models of working memory , 2000, Nature Neuroscience.

[10]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[11]  Tom Ziemke,et al.  Neuromodulation of Reactive Sensorimotor Mappings as a Short-Term Memory Mechanism in Delayed Response Tasks , 2002, Adapt. Behav..

[12]  Michel Kerszberg,et al.  Acquisition and performance of delayed-response tasks: a neural network model. , 2005, Cerebral cortex.

[13]  Stéphane Doncieux,et al.  Sferesv2: Evolvin' in the multi-core world , 2010, IEEE Congress on Evolutionary Computation.

[14]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[15]  Stéphane Doncieux,et al.  Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia , 2010, GECCO '10.

[16]  H E ROSVOLD,et al.  A continuous performance test of brain damage. , 1956, Journal of consulting psychology.

[17]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[18]  Jonathan D. Cohen,et al.  Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function , 1999, Biological Psychiatry.

[19]  Hannah S. Locke,et al.  Flexible neural mechanisms of cognitive control within human prefrontal cortex , 2009, Proceedings of the National Academy of Sciences.

[20]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[21]  Stéphane Doncieux,et al.  Evolving modular neural-networks through exaptation , 2009, 2009 IEEE Congress on Evolutionary Computation.

[22]  Ranulfo Romo,et al.  Flexible Control of Mutual Inhibition: A Neural Model of Two-Interval Discrimination , 2005, Science.

[23]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[24]  Jonathan D. Cohen,et al.  A Computational Model of Prefrontal Cortex Function , 1994, NIPS.

[25]  Jean-Arcady Meyer,et al.  Evolving modular neural networks to solve challenging control problems , 2004 .