Computational modeling of observational learning inspired by the cortical underpinnings of human primates

Recent neuroscientific evidence in human and non-human primates indicates that the regions that become active during motor execution and motor observation overlap extensively in the cerebral cortex. This suggests that to observe an action, these primates employ their motor and somatosensation areas in order to simulate it internally. In line with this finding, in the current paper, we examine relevant neuroscientific evidence in order to design a computational agent that can facilitate observational learning of reaching movements. For this reason, we develop a novel motor control system, inspired from contemporary theories of motor control, and demonstrate how it can be used during observation to facilitate learning, without the active involvement of the agent’s body. Our results show that novel motor skills can be acquired only by observation, by optimizing the peripheral components of the agent’s motion.

[1]  S. Carey,et al.  Understanding other minds: linking developmental psychology and functional neuroimaging. , 2004, Annual review of psychology.

[2]  M. Iacoboni.  Imitation, empathy, and mirror neurons. , 2009, Annual review of psychology.

[3]  J. Saunders,et al.  An experimental comparison of imitation paradigms used in social robotics , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[4]  Chrystopher L. Nehaniv.  Computation for Metaphors, Analogy, and Agents , 2000, Lecture Notes in Computer Science.

[5]  J. Decety,et al.  The cerebellum participates in mental activity: tomographic measurements of regional cerebral blood flow , 1990, Brain Research.

[6]  Helen E. Savaki,et al.  Observation of action: grasping with the mind's hand , 2004, NeuroImage.

[7]  Auke Ijspeert,et al.  Modeling discrete and rhythmic movements through motor primitives: a review , 2010, Biological Cybernetics.

[8]  A. Richardson.  Mental practice: a review and discussion. II. , 1967, Research quarterly.

[9]  Panos E. Trahanias,et al.  Computational modeling of cortical pathways involved in action execution and action observation , 2011, Neurocomputing.

[10]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[11]  B. Kirkcaldy,et al.  Normalities and abnormalities in human movement , 1989 .

[12]  R. S. Sackett,et al.  The Relationship between Amount of Symbolic Rehearsal and Retention of a Maze Habit , 1935 .

[13]  N. A. Borghese,et al.  Different Brain Correlates for Watching Real and Virtual Hand Actions , 2001, NeuroImage.

[14]  A P Georgopoulos,et al.  On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[16]  B. Scassellati.  Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot , 1999 .

[17]  S. Wermter,et al.  An associator network approach to robot learning by imitation through vision, motor control and language , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[18]  F. A. Mussa-lvaldi,et al.  Convergent force fields organized in the frog's spinal cord , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  L. Craighero,et al.  Modulation of spinal excitability during observation of hand actions in humans , 2001, The European journal of neuroscience.

[20]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[21]  Jean Decety,et al.  rCBF landscapes during motor performance and motor ideation of a graphic gesture , 2004, European archives of psychiatry and neurological sciences.

[22]  Yiannis Demiris,et al.  Perceiving the unusual: Temporal properties of hierarchical motor representations for action perception , 2006, Neural Networks.

[23]  Panos E. Trahanias,et al.  Observational Learning Based on Models of Overlapping Pathways , 2011, ICANN.

[24]  R. Byrne,et al.  Machiavellian intelligence : social expertise and the evolution of intellect in monkeys, apes, and humans , 1990 .

[25]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[26]  J. Mazziotta,et al.  Mapping motor representations with positron emission tomography , 1994, Nature.

[27]  M S Gazzaniga,et al.  Cerebrally Lateralized Mental Representations of Hand Shape and Movement , 1998, The Journal of Neuroscience.

[28]  Panos E. Trahanias,et al.  Computational modeling of online reaching , 2011, ECAL.

[29]  B. Touwen,et al.  The brain and development of function , 1998 .

[30]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[31]  A. Meltzoff,et al.  An fMRI study of imitation: action representation and body schema , 2005, Neuropsychologia.

[32]  Peter I. Corke,et al.  A robotics toolbox for MATLAB , 1996, IEEE Robotics Autom. Mag..

[33]  Chrystopher L. Nehaniv,et al.  Imitation as a Dual-Route Process Featuring Predictive and Learning Components: A Biologically Plausible Computational Model , 2002 .

[34]  O. Hikosaka.  Models of information processing in the basal Ganglia edited by James C. Houk, Joel L. Davis and David G. Beiser, The MIT Press, 1995. $60.00 (400 pp) ISBN 0 262 08234 9 , 1995, Trends in Neurosciences.

[35]  V. Raos,et al.  Mental Simulation of Action in the Service of Action Perception , 2007, The Journal of Neuroscience.

[36]  J. Kaas,et al.  Multiple representations of the body within the primary somatosensory cortex of primates. , 1979, Science.

[37]  Herbert Heuer,et al.  A Multiple-Representations’ Approach to Mental Practice of Motor Skills1 , 1989 .

[38]  A. Barto,et al.  Adaptive Critics and the Basal Ganglia , 1994 .

[39]  Aude Billard,et al.  DRAMA, a Connectionist Architecture for Control and Learning in Autonomous Robots , 1999, Adapt. Behav..

[40]  F A Mussa-Ivaldi,et al.  Computations underlying the execution of movement: a biological perspective. , 1991, Science.

[41]  A B Schwartz,et al.  Direct cortical representation of drawing. , 1994, Science.

[42]  G. Rizzolatti,et al.  Grasping objects and grasping action meanings: the dual role of monkey rostroventral premotor cortex (area F5). , 1998, Novartis Foundation symposium.

[43]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB , 2004, Neural Networks.

[44]  M. Jeannerod.  The representing brain: Neural correlates of motor intention and imagery , 1994, Behavioral and Brain Sciences.

[45]  Michael A. Arbib,et al.  Schema design and implementation of the grasp-related mirror neuron system , 2002, Biological Cybernetics.

[46]  J. Hollerman,et al.  Reward processing in primate orbitofrontal cortex and basal ganglia. , 2000, Cerebral cortex.

[47]  M. Denis.  Visual imagery and the use of mental practice in the development of motor skills. , 1985, Canadian journal of applied sport sciences. Journal canadien des sciences appliquees au sport.

[48]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[49]  G. Rizzolatti.  The mirror neuron system and its function in humans , 2005, Anatomy and Embryology.

[50]  P. Roland,et al.  Different cortical areas in man in organization of voluntary movements in extrapersonal space. , 1980, Journal of neurophysiology.

[51]  Angela R. Laird,et al.  ALE meta-analysis of action observation and imitation in the human brain , 2010, NeuroImage.

[52]  G. Rizzolatti,et al.  Mirror neurons and mirror systems in monkeys and humans. , 2008, Physiology.

[53]  R. Finke.  Levels of equivalence in imagery and perception. , 1980 .

[54]  H. Savaki,et al.  How do we understand tHe actions of otHers ? by mental simulation , not mirroring , 2012 .

[55]  S. Baron-Cohen,et al.  Understanding other minds : perspectives from autism , 1994 .