A single computational model for many learning phenomena

Abstract Simplicity is a basic principle of science and this implies that, if we want to explain the behaviour of animals by constructing robots that behave like real animals, one and the same robot should reproduce as many behaviours and as many behavioural phenomena as possible. In this paper we describe robots that both evolve and learn in their “natural” environment and, in addition, learn in the equivalent of an experimental laboratory and reproduce a variety of results of experiments on learning in animals. We introduce a new model of learning in which the weights of the connections that link the units of the robots’ neural network are genetically inherited and do not change during the robots’ life but what changes during life and makes the robots learn new behaviours is the synaptic receptivity of a special set of network units which we call learning units. The robots evolve in a variety of different environments and they learn in a variety of different ways including imprinting and learning by imitating the behaviour of others. Then we test the robots in the controlled conditions of an artificial laboratory and we reproduce a number of experimental results on both operant learning and classical conditioning, including learning and extinction curves, the role of the temporal interval between conditioned and unconditioned stimuli, and the influence of motivation on learning.

[1]  Marco Mirolli,et al.  Evolution and Learning in an Intrinsically Motivated Reinforcement Learning Robot , 2007, ECAL.

[2]  Aude Billard,et al.  From Animals to Animats , 2004 .

[3]  Leslie Pack Kaelbling,et al.  Ecological Robotics , 1998, Adapt. Behav..

[4]  Stefano Nolfi,et al.  Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines , 2000 .

[5]  Jonathan Baxter.  The evolution of learning algorithms for artificial neural networks , 1993 .

[6]  Christian Balkenius,et al.  Computational models of classical conditioning: a comparative study , 1998 .

[7]  Peter M. Todd,et al.  Exploring adaptive agency II: simulating the evolution of associative learning , 1991 .

[8]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[9]  K. Lorenz,et al.  Der Kumpan in der Umwelt des Vogels , 1935, Journal für Ornithologie.

[10]  U. Frey,et al.  Synaptic tagging and long-term potentiation , 1997, Nature.

[11]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[12]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[13]  Stefano Nolfi,et al.  Evorobot* - A Tool for Running Experiments on the Evolution of Communication , 2010, Evolution of Communication and Language in Embodied Agents.

[14]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..

[15]  Frederic Mery,et al.  A fitness cost of learning ability in Drosophila melanogaster , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[16]  M. Bear,et al.  Synaptic plasticity: LTP and LTD , 1994, Current Opinion in Neurobiology.

[17]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[18]  Isaac Meilijson,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002, Adapt. Behav..

[19]  P. Bateson.  How do sensitive periods arise and what are they for? , 1979, Animal Behaviour.

[20]  Ronald C. Arkin,et al.  Ecological Robotics: A Schema-Theoretic Approach , 1996, Intelligent Robots.

[21]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  H. Atwood,et al.  Silent synapses in neural plasticity: current evidence. , 1999, Learning & memory.

[23]  R. Morris,et al.  Enhanced long-term potentiation and impaired learning in mice with mutant postsynaptic density-95 protein , 1998, Nature.

[24]  Stefano Nolfi,et al.  Econets: Neural networks that learn in an environment , 1990 .

[25]  A. Cangelosi,et al.  Developmental Robotics: From Babies to Robots , 2015 .

[26]  Inman Harvey,et al.  An Evolutionary Ecological Approach to the Study of Learning Behavior Using a Robot-Based Model , 2002, Adapt. Behav..

[27]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[28]  B. Skinner,et al.  The Behavior of Organisms: An Experimental Analysis , 2016 .