On the relationships between synaptic plasticity and generative systems

The present paper analyzes the mutual relationships between generative and developmental systems (GDS) and synaptic plasticity when evolving plastic artificial neural networks (ANNs) in reward-based scenarios. We first introduce the concept of synaptic Transitive Learning Abilities (sTLA), which reflects how well an evolved plastic ANN can cope with learning scenarios not encountered during the evolution process. We subsequently report results of a set of experiments designed to check that (1) synaptic plasticity can help a GDS to fine-tune synaptic weights and (2) that with the investigated generative encoding (EvoNeuro), only a few learning scenarios are necessary to evolve a general learning system, which can adapt itself to reward-based scenarios not tested during the fitness evaluation.

[1]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[2]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[3]  Shigenobu Kobayashi,et al.  An Analysis of Actor/Critic Algorithms Using Eligibility Traces: Reinforcement Learning with Imperfect Value Function , 1998, ICML.

[4]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[5]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[6]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[7]  Dario Floreano,et al.  Evolution of Adaptive Synapses: Robots with Fast Adaptive Behavior in New Environments , 2001, Evolutionary Computation.

[8]  R. Klein,et al.  A Model of Saccade Initiation Based on the Competitive Integration of Exogenous and Endogenous Signals in the Superior Colliculus , 2001, Journal of Cognitive Neuroscience.

[9]  Jordan B. Pollack,et al.  Creating High-Level Components with a Generative Representation for Body-Brain Evolution , 2002, Artificial Life.

[10]  Isaac Meilijson,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002, Adapt. Behav..

[11]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[12]  Dario Floreano,et al.  Levels of dynamics and adaptive behavior in evolutionary neural controllers , 2002 .

[13]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[14]  Risto Miikkulainen,et al.  Evolving adaptive neural networks with and without adaptive synapses , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[15]  Behavioural Plasticity in Autonomous Agents: A Comparison between Two Types of Controller , 2003, EvoWorkshops.

[16]  Gregory Hornby,et al.  Measuring, enabling and comparing modularity, regularity and hierarchy in evolutionary design , 2005, GECCO '05.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  J. Slotine,et al.  Selective amplification using a contracting model of the basal ganglia , 2006 .

[19]  Dario Floreano,et al.  Evolving neuromodulatory topologies for reinforcement learning-like problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[20]  Dario Floreano,et al.  Analog Genetic Encoding for the Evolution of Circuits and Networks , 2007, IEEE Transactions on Evolutionary Computation.

[21]  Dario Floreano,et al.  Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios , 2008, ALIFE.

[22]  Charles E. Hughes,et al.  How novelty search escapes the deceptive trap of learning to learn , 2009, GECCO.

[23]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[24]  Ben Jones,et al.  Novelty of behaviour as a basis for the neuro-evolution of operant reward learning , 2009, GECCO.

[25]  Stéphane Doncieux,et al.  Using behavioral exploration objectives to solve deceptive problems in neuro-evolution , 2009, GECCO.

[26]  Kenneth O. Stanley.  A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks , 2009 .

[27]  Sebastian Risi,et al.  Indirectly Encoding Neural Plasticity as a Pattern of Local Rules , 2010, SAB.

[28]  Stéphane Doncieux,et al.  Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia , 2010, GECCO '10.

[29]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[30]  Kenneth O. Stanley,et al.  On the Performance of Indirect Encoding Across the Continuum of Regularity , 2011, IEEE Transactions on Evolutionary Computation.