Robot coverage control by evolved neuromodulation

An important connection between evolution and learning was made over a century ago and is now termed as the Baldwin effect. Learning acts as a guide for an evolutionary search process. In this study reinforcement learning agents are trained to solve the robot coverage control problem. These agents are improved by evolving neuromodulatory gene regulatory networks (GRN) that influence the learning and memory of agents. Agents trained by these neuromodulatory GRNs can consistently generalize better than agents trained with fixed parameter settings. This work introduces evolutionary GRN models into the context of neuromodulation and illustrates some of the benefits that stem from neuromodulatory GRNs.

[1]  P. Katz.  Intrinsic and extrinsic neuromodulation of motor circuits , 1995, Current Opinion in Neurobiology.

[2]  Eve Marder,et al.  Cellular, synaptic and network effects of neuromodulation , 2002, Neural Networks.

[3]  W. Banzhaf.  Artificial Regulatory Networks and Genetic Programming , 2003 .

[4]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[5]  Jean-Marc Fellous,et al.  Computational Models of Neuromodulation , 1998, Neural Computation.

[6]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[7]  Hervé Luga,et al.  A Synthesis of the Cell2Organ Developmental Model , 2012, Morphogenetic Engineering, Toward Programmable Complex Systems.

[8]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[9]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[10]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  Lenka Pitonakova,et al.  Ultrastable neuroendocrine robot controller , 2013, Adapt. Behav..

[12]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[13]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[14]  Howie Choset,et al.  Coverage for robotics – A survey of recent results , 2001, Annals of Mathematics and Artificial Intelligence.

[15]  Borys Wróbel,et al.  Evo-devo in silico - a Model of a Gene Network Regulating Multicellular Development in 3D Space with Artificial Physics , 2008, ALIFE.

[16]  Borys Wróbel,et al.  Evolving Gene Regulatory Networks for Real Time Control of Foraging Behaviours , 2010, ALIFE.

[17]  Wolfgang Banzhaf,et al.  Evolving Control Metabolisms for a Robot , 2001, Artificial Life.

[18]  Lubica Benuskova,et al.  Modeling brain dynamics using computational neurogenetic approach , 2008, Cognitive Neurodynamics.

[19]  Yves Duthen,et al.  Controlling cooperative and conflicting continuous actions with a Gene Regulatory Network , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[20]  René Doursat,et al.  Facilitating evolutionary innovation by developmental modularity and variability , 2009, GECCO.

[21]  Michael E. Palmer,et al.  Evolved neural network controllers for physically simulated robots that hunt with an artificial visual cortex , 2012, ALIFE.

[22]  W. Fontana,et al.  Plasticity, evolvability, and modularity in RNA. , 2000, The Journal of experimental zoology.

[23]  J. Baldwin.  A New Factor in Evolution , 1896, The American Naturalist.

[24]  E. Marder,et al.  Plasticity in single neuron and circuit computations , 2004, Nature.

[25]  Olaf Sporns,et al.  Neuromodulation and plasticity in an autonomous robot , 2002, Neural Networks.

[26]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Charles E. Hughes,et al.  How novelty search escapes the deceptive trap of learning to learn , 2009, GECCO.

[29]  Vladimir Brezina,et al.  Variability of Motor Neuron Spike Timing Maintains and Shapes Contractions of the Accessory Radula Closer Muscle of Aplysia , 2006, The Journal of Neuroscience.

[30]  Jeffrey L. Krichmar,et al.  The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World , 2008, Adapt. Behav..

[31]  Lee Spector,et al.  What’s in an Evolved Name? The Evolution of Modularity via Tag-Based Reference , 2011 .

[32]  Marc Schoenauer,et al.  Evolving Genes to Balance a Pole , 2010, EuroGP.

[33]  E. Marder.  Neuromodulation of Neuronal Circuits: Back to the Future , 2012, Neuron.

[34]  Dario Floreano,et al.  Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios , 2008, ALIFE.

[35]  A. Wagner.  DOES EVOLUTIONARY PLASTICITY EVOLVE? , 1996, Evolution; international journal of organic evolution.

[36]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  Maria Gini,et al.  Dispersing robots in an unknown environment , 2004, DARS.

[38]  L. Ancel,et al.  Undermining the Baldwin expediting effect: does phenotypic plasticity accelerate evolution? , 2000, Theoretical population biology.

[39]  Sonia Martínez,et al.  Coverage control for mobile sensing networks , 2002, IEEE Transactions on Robotics and Automation.