Neuroevolution: from architectures to learning

Artificial neural networks (ANNs) are applied to many real-world problems, ranging from pattern classification to robot control. In order to design a neural network for a particular task, the choice of an architecture (including the choice of a neuron model), and the choice of a learning algorithm have to be addressed. Evolutionary search methods can provide an automatic solution to these problems. New insights in both neuroscience and evolutionary biology have led to the development of increasingly powerful neuroevolution techniques over the last decade. This paper gives an overview of the most prominent methods for evolving ANNs with a special focus on recent advances in the synthesis of learning architectures.

[1]  J. Baldwin A New Factor in Evolution , 1896, The American Naturalist.

[2]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[3]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[4]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[5]  K. Gunderson Paranoia concerning program-resistant aspects of the mind - and let's drop rocks on Turing's toes again , 1981, Behavioral and Brain Sciences.

[6]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[7]  Geoffrey E. Hinton,et al.  Learning representations by back-propagation errors, nature , 1986 .

[8]  Geoffrey E. Hinton,et al.  Learning representations of back-propagation errors , 1986 .

[9]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[10]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[11]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[12]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[13]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[14]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[15]  Sio Carlos,et al.  Evolving a learning algorithm for the binary perceptron , 1991 .

[16]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[17]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[18]  Nicholas J. Radcliffe,et al.  Forma Analysis and Random Respectful Recombination , 1991, ICGA.

[19]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[20]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[21]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[22]  Kemal Oflazer,et al.  Genetic Synthesis of Unsupervised Learning Algorithms , 1993 .

[23]  Jonathan Baxter The evolution of learning algorithms for artificial neural networks , 1993 .

[24]  Randall D. Beer,et al.  Sequential Behavior and Learning in Evolved Dynamical Neural Networks , 1994, Adapt. Behav..

[25]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[26]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[27]  D. Parisi,et al.  Phenotypic plasticity in evolving neural networks , 1994, Proceedings of PerAc '94. From Perception to Action.

[28]  Frédéric Gruau,et al.  Automatic Definition of Modular Neural Networks , 1994, Adapt. Behav..

[29]  Inman Harvey,et al.  The use of genetic algorithms for the development of sensorimotor control systems , 1994, Proceedings of PerAc '94. From Perception to Action.

[30]  O. Hikosaka Models of information processing in the basal Ganglia edited by James C. Houk, Joel L. Davis and David G. Beiser, The MIT Press, 1995. $60.00 (400 pp) ISBN 0 262 08234 9 , 1995, Trends in Neurosciences.

[31]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[32]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[33]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[34]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[35]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[36]  Giles Mayley,et al.  Landscapes, Learning Costs, and Genetic Assimilation , 1996, Evolutionary Computation.

[37]  Francesco Mondada,et al.  Evolution of Plastic Neurocontrollers for Situated Agents , 1996 .

[38]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[39]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[40]  Takahiro Sasaki,et al.  Adaptation toward Changing Environments: Why Darwinian in Nature? , 1997 .

[41]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[42]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[43]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[44]  Wolfgang Maass,et al.  Spiking Neurons , 1998, NC.

[45]  Simon M. Lucas,et al.  A comparison of matrix rewriting versus direct encoding for evolving neural networks , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[46]  Hugo de Garis,et al.  A "Spike Interval Information Coding" Representation for ATR's CAM-Brain Machine (CBM) , 1998, ICES.

[47]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[48]  Phil Husbands,et al.  Evolutionary robotics , 2014, Evolutionary Intelligence.

[49]  Phil Husbands,et al.  Better Living Through Chemistry: Evolving GasNets for Robot Control , 1998, Connect. Sci..

[50]  Jean-Marc Fellous,et al.  Computational Models of Neuromodulation , 1998, Neural Computation.

[51]  Eiji Mizutani,et al.  Totally model-free reinforcement learning by actor-critic Elman networks in non-Markovian domains , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[52]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[53]  Stefano Nolfi,et al.  Learning and Evolution , 1999, Auton. Robots.

[54]  James Kennedy,et al.  Proceedings of the 1998 IEEE International Conference on Evolutionary Computation [Book Review] , 1999, IEEE Transactions on Evolutionary Computation.

[55]  X. Yao Evolving Artificial Neural Networks , 1999 .

[56]  Wulfram Gerstner,et al.  Spiking neurons , 1999 .

[57]  Paul S. Katz,et al.  What are we talking about? Modes of neuronal communication , 1999 .

[58]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[59]  Torsten Reil,et al.  Dynamics of Gene Expression in an Artificial Genome - Implications for Biological and Artificial Ontogeny , 1999, ECAL.

[60]  E. Kandel,et al.  Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory , 2000, Nature Reviews Neuroscience.

[61]  Dario Floreano,et al.  Evolutionary robots with on-line self-organization and behavioral fitness , 2000, Neural Networks.

[62]  J. Nazuno Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[63]  Dario Floreano,et al.  Evolution of Adaptive Synapses: Robots with Fast Adaptive Behavior in New Environments , 2001, Evolutionary Computation.

[64]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[65]  Dario Floreano,et al.  Evolution of Plastic Control Networks , 2001, Auton. Robots.

[66]  Dario Floreano,et al.  Evolution of Spiking Neural Controllers for Autonomous Vision-Based Robots , 2001, EvoRobots.

[67]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[68]  Inman Harvey,et al.  An Evolutionary Ecological Approach to the Study of Learning Behavior Using a Robot-Based Model , 2002, Adapt. Behav..

[69]  Josh Bongard,et al.  Evolving modular genetic regulatory networks , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[70]  Isaac Meilijson,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002, Adapt. Behav..

[71]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[72]  Christian Igel,et al.  Neuroevolution for reinforcement learning using evolution strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[73]  Ezequiel A. Di Paolo,et al.  Evolving spike-timing-dependent plasticity for single-trial learning in robots , 2003 .

[74]  Peter J. Bentley,et al.  On growth, form and computers , 2003 .

[75]  Torsten Reil 14 – Artificial genomes as models of gene regulation , 2003 .

[76]  Janet Wiles,et al.  Structure and dynamics of a gene network model incorporating small RNAs , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[77]  Dario Floreano,et al.  Exploring the T-Maze: Evolving Learning-Like Robot Behaviors Using CTRNNs , 2003, EvoWorkshops.

[78]  Dario Floreano,et al.  Evolution of analog networks using local string alignment on highly reorganizable genomes , 2004, Proceedings. 2004 NASA/DoD Conference on Evolvable Hardware, 2004..

[79]  N. Schraudolph,et al.  Dynamic Parameter Encoding for genetic algorithms , 2004, Machine Learning.

[80]  Aude Billard,et al.  GasNets and other Evolvable Neural Networks applied to Bipedal Locomotion , 2004 .

[81]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[82]  Phil Husbands,et al.  GasNets and other evovalble neural networks applied to bipedal locomotion , 2004 .

[83]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 2004 .

[84]  Eytan Ruppin,et al.  Spikes that count: rethinking spikiness in neurally embedded systems , 2004, Neurocomputing.

[85]  Risto Miikkulainen,et al.  Neuroevolution of an automobile crash warning system , 2005, GECCO '05.

[86]  Radhika Nagpal,et al.  On Growth, Form and Computers , 2005, Genetic Programming and Evolvable Machines.

[87]  Diego Federici,et al.  Evolving developing spiking neural networks , 2005, 2005 IEEE Congress on Evolutionary Computation.

[88]  Risto Miikkulainen,et al.  Real-time Learning in the NERO Video Game , 2005, AIIDE.

[89]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[90]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[91]  J. Shapiro A 21st century view of evolution: genome system architecture, repetitive DNA, and natural genetic engineering. , 2005, Gene.

[92]  Larry Bull,et al.  A Neural Learning Classifier System with Self-Adaptive Constructivism for Mobile Robot Control , 2006, Artificial Life.

[93]  Dario Floreano,et al.  Neuroevolution with Analog Genetic Encoding , 2006, PPSN.

[94]  Andrew Philippides,et al.  GasNets and CTRNNs - A Comparison in Terms of Evolvability , 2006, SAB.

[95]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[96]  Dario Floreano,et al.  Evolving neuromodulatory topologies for reinforcement learning-like problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[97]  Marco Dorigo,et al.  From Solitary to Collective Behaviours: Decision Making and Cooperation , 2007, ECAL.

[98]  Risto Miikkulainen,et al.  Acquiring evolvability through adaptive representations , 2007, GECCO '07.

[99]  Risto Miikkulainen,et al.  Coevolving Strategies for General Game Playing , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[100]  Dario Floreano,et al.  Center of mass encoding: a self-adaptive representation with adjustable redundancy for real-valued parameters , 2007, GECCO '07.

[101]  Dario Floreano,et al.  The Age of Analog Networks , 2008, AI Mag..