Evolving Plastic Neural Networks for Online Learning: Review and Future Directions

Recent years have seen a resurgence of interest in evolving plastic neural networks for online learning. These approaches have an intrinsic appeal --- since, to date, the only working example of general intelligence is the human brain, which has developed through evolution, and exhibits a great capacity to adapt to unfamiliar environments. In this paper we review prior work in this area --- including problem domains and tasks, fitness functions, synaptic plasticity models and neural network encoding schemes. We conclude with a discussion of current findings and promising future directions, including incorporation of functional properties observed in biological neural networks which appear to play a role in learning processes, and addressing the "general" in general intelligence by the introduction of previously unseen tasks during the evolution process.

[1]  Sebastian Risi,et al.  A unified approach to evolving plasticity and neural geometry , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[2]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[3]  Jean-Baptiste Mouret,et al.  Artificial Evolution of Plastic Neural Networks: A Few Key Concepts , 2014, Growing Adaptive Machines.

[4]  T. Bliss,et al.  Plasticity in the human central nervous system. , 2006, Brain : a journal of neurology.

[5]  Risto Miikkulainen,et al.  Evolving adaptive neural networks with and without adaptive synapses , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[6]  Charles E. Hughes,et al.  Evolving plastic neural networks with novelty search , 2010, Adapt. Behav..

[7]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[8]  W. Abraham.  Metaplasticity: tuning synapses and networks for plasticity , 2008, Nature Reviews Neuroscience.

[9]  Dario Floreano,et al.  Neural morphogenesis, synaptic plasticity, and evolution , 2001, Theory in Biosciences.

[10]  H. Seung,et al.  Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[11]  Randall D. Beer,et al.  Center-Crossing Recurrent Neural Networks for the Evolution of Rhythmic Behavior , 2002, Neural Computation.

[12]  F. Gage,et al.  New neurons and new memories: how does adult hippocampal neurogenesis affect learning and memory? , 2010, Nature Reviews Neuroscience.

[13]  Phil Husbands,et al.  Quadrupedal locomotion: GasNets, CTRNNs and Hybrid CTRNN/PNNs compared , 2004 .

[14]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[15]  Sebastian Risi,et al.  Indirectly Encoding Neural Plasticity as a Pattern of Local Rules , 2010, SAB.

[16]  Faustino J. Gomez,et al.  When Novelty Is Not Enough , 2011, EvoApplications.

[17]  Ezequiel A. Di Paolo,et al.  New Models for Old Questions: Evolutionary Robotics and the 'A Not B' Error , 2007, ECAL.

[18]  Francesco Mondada,et al.  Evolution of Plastic Neurocontrollers for Situated Agents , 1996 .

[19]  Dario Floreano,et al.  Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios , 2008, ALIFE.

[20]  Wolfgang Banzhaf,et al.  Advances in Artificial Life , 2003, Lecture Notes in Computer Science.

[21]  Dario Floreano,et al.  Evolving neuromodulatory topologies for reinforcement learning-like problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[22]  Aude Billard,et al.  From Animals to Animats , 2004 .

[23]  Peter G Fuerst,et al.  Adhesion molecules in establishing retinal circuitry , 2009, Current Opinion in Neurobiology.

[24]  L. Abbott,et al.  Synaptic plasticity: taming the beast , 2000, Nature Neuroscience.

[25]  Thierry Hoinville,et al.  Flexible and multistable pattern generation by evolving constrained plastic neurocontrollers , 2011, Adapt. Behav..

[26]  E. D. Di Paolo.  Evolving spike-timing-dependent plasticity for single-trial learning in robots. , 2003, Philosophical transactions. Series A, Mathematical, physical, and engineering sciences.

[27]  Jonathan Baxter.  The evolution of learning algorithms for artificial neural networks , 1993 .

[28]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[29]  Gul Muhammad Khan,et al.  Evolution of Cartesian Genetic Programs for Development of Learning Neural Architecture , 2011, Evolutionary Computation.

[30]  John Hallam,et al.  From Animals to Animats 10 , 2008 .

[31]  Jean-Baptiste Mouret,et al.  On the relationships between synaptic plasticity and generative systems , 2011, GECCO '11.

[32]  Jean-Baptiste Mouret,et al.  Using a map-based encoding to evolve plastic neural networks , 2011, 2011 IEEE Workshop on Evolving and Adaptive Intelligent Systems (EAIS).

[33]  E. D. Di Paolo.  Spike-Timing Dependent Plasticity for Evolved Robots , 2002 .

[34]  Y. Niv,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002 .

[35]  Shimon Whiteson,et al.  Critical factors in the performance of novelty search , 2011, GECCO '11.

[36]  HighWire Press.  Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.