Artificial Evolution of Plastic Neural Networks: A Few Key Concepts

This chapter introduces a hierarchy of concepts to classify the goals and the methods used in articles that mix neuro-evolution and synaptic plasticity. We propose definitions of “behavioral robustness” and oppose it to “reward-based behavioral changes”; we then distinguish the switch between behaviors and the acquisition of new behaviors. Last, we formalize the concept of “synaptic General Learning Abilities” (sGLA) and that of “synaptic Transitive learning Abilities (sTLA)”. For each concept, we review the literature to identify the main experimental setups and the typical studies.

[1]  Stéphane Doncieux,et al.  Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia , 2010, GECCO '10.

[2]  Sebastian Risi,et al.  Indirectly Encoding Neural Plasticity as a Pattern of Local Rules , 2010, SAB.

[3]  E. Kandel,et al.  Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory , 2000, Nature Reviews Neuroscience.

[4]  Dario Floreano,et al.  Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios , 2008, ALIFE.

[5]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[6]  Stéphane Doncieux,et al.  Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study , 2012, Evolutionary Computation.

[7]  Jean-Baptiste Mouret,et al.  On the relationships between synaptic plasticity and generative systems , 2011, GECCO '11.

[8]  B. Skinner.  Operant Behavior , 2021, Encyclopedia of Evolutionary Psychological Science.

[9]  Kenneth O. Stanley,et al.  On the Performance of Indirect Encoding Across the Continuum of Regularity , 2011, IEEE Transactions on Evolutionary Computation.

[10]  D. Parisi,et al.  Phenotypic plasticity in evolving neural networks , 1994, Proceedings of PerAc '94. From Perception to Action.

[11]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[12]  Toshiyuki Kondo,et al.  Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control , 2007, Appl. Soft Comput..

[13]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[14]  Isaac Meilijson,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002, Adapt. Behav..

[15]  Anthony Kulis,et al.  Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies , 2009, Scalable Comput. Pract. Exp..

[16]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[17]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[18]  Stéphane Doncieux,et al.  Using behavioral exploration objectives to solve deceptive problems in neuro-evolution , 2009, GECCO.

[19]  P. Ulinski.  Fundamentals of Computational Neuroscience , 2007 .

[20]  Risto Miikkulainen,et al.  Evolving adaptive neural networks with and without adaptive synapses , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[21]  Dario Floreano,et al.  Levels of dynamics and adaptive behavior in evolutionary neural controllers , 2002 .

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Jean-Baptiste Mouret,et al.  On the Relationships between Generative Encodings, Regularity, and Learning Abilities when Evolving Plastic Artificial Neural Networks , 2013, PloS one.

[24]  Stefano Nolfi.  How Learning and Evolution Interact: The Case of a Learning Task which Differs from the Evolutionary Task , 1999, Adapt. Behav..

[25]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[26]  Dario Floreano,et al.  Evolution of Adaptive Synapses: Robots with Fast Adaptive Behavior in New Environments , 2001, Evolutionary Computation.

[27]  Ben Jones,et al.  Novelty of behaviour as a basis for the neuro-evolution of operant reward learning , 2009, GECCO.

[28]  Dario Floreano,et al.  Evolving neuromodulatory topologies for reinforcement learning-like problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[29]  Francesco Mondada,et al.  Evolution of Plastic Neurocontrollers for Situated Agents , 1996 .

[30]  Charles E. Hughes,et al.  How novelty search escapes the deceptive trap of learning to learn , 2009, GECCO.

[31]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[32]  Kenneth O. Stanley.  A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks , 2009 .

[33]  Jean-Arcady Meyer,et al.  Evolution and development of neural controllers for locomotion, gradient-following, and obstacle-avoidance in artificial insects , 1998, IEEE Trans. Neural Networks.

[34]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..