A Neural Architecture for Performing Actual and Mentally Simulated Movements During Self-Intended and Observed Bimanual Arm Reaching Movements

Dexterous reaching, pointing, and grasping play a critical role in human interactions with tools and the environment, and it also allows individuals to interact with one another effectively in social settings. Developing robotic systems with mental simulation and imitation learning abilities for such tasks seems a promising way to enhance robot performance as well as to enable interactions with humans in a social context. In spite of important advances in artificial intelligence and smart robotics, current robotic systems lack the flexibility and adaptability that humans so naturally exhibit. Here we present and study a neural architecture that captures some critical visuo-spatial transformations that are required for the cognitive processes of mental simulation and imitation. The results show that our neural model can perform accurate, flexible and robust 3D unimanual and bimanual actual/imagined reaching movements while avoiding extreme joint positions and generating kinematics similar to those observed with humans. In addition, using visuo-spatial transformations, the neural model was able to observe/imitate bimanual arm reaching movements independently of the viewpoint, distance and anthropometry between the demonstrator and imitator. Our model is a first step towards developing a more advanced neurally-inspired hierarchical architecture that integrates mental simulation and sensorimotor processing as it learns to imitate dexterous bimanual arm movements.

[1]  Marc Toussaint,et al.  Learning a World Model and Planning with a Self-Organizing, Dynamic Neural System , 2003, NIPS.

[2]  J C Mazziotta,et al.  Reafferent copies of imitated actions in the right superior temporal cortex , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Iwona Stepniewska,et al.  Cortical connections of functional zones in posterior parietal cortex and frontal cortex motor regions in new world monkeys. , 2011, Cerebral cortex.

[4]  Mian Muhammad Awais,et al.  Learning from demonstration in robots: Experimental comparison of neural architectures , 2011 .

[5]  Marc Toussaint,et al.  A Sensorimotor Map: Modulating Lateral Interactions for Anticipation and Planning , 2006, Neural Computation.

[6]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[7]  Masao Ito.  Error detection and representation in the olivo-cerebellar system , 2013, Front. Neural Circuits.

[8]  A Diamond,et al.  Reaching control of a full-torso, modelled musculoskeletal robot using muscle synergies emergent under reinforcement learning , 2014, Bioinspiration & biomimetics.

[9]  Rüdiger Dillmann,et al.  Learning of Planning Models for Dexterous Manipulation Based on Human Demonstrations , 2012, International Journal of Social Robotics.

[10]  Yiannis Demiris,et al.  Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning , 2003, Connect. Sci..

[11]  M. Herrero,et al.  Functional anatomy of thalamus and basal ganglia , 2002, Child’s Nervous System.

[12]  Aude Billard,et al.  Learning human arm movements by imitation: : Evaluation of a biologically inspired connectionist architecture , 2000, Robotics Auton. Syst..

[13]  G. Rizzolatti,et al.  View-Based Encoding of Actions in Mirror Neurons of Area F5 in Macaque Premotor Cortex , 2011, Current Biology.

[14]  Trent J. Bradberry,et al.  Reconstructing Three-Dimensional Hand Movements from Noninvasive Electroencephalographic Signals , 2010, The Journal of Neuroscience.

[15]  M. Jeannerod.  The representing brain: Neural correlates of motor intention and imagery , 1994, Behavioral and Brain Sciences.

[16]  Daniel M. Wolpert,et al.  Forward Models for Physiological Motor Control , 1996, Neural Networks.

[17]  José Santos-Victor,et al.  Visual learning by imitation with motor representations , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  A. Guillot,et al.  The neurophysiological foundations of mental and motor imagery , 2010 .

[19]  M. Arbib,et al.  Role of the cerebellum in reaching movements in humans. I. Distributed inverse dynamics control , 1998, The European journal of neuroscience.

[20]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[21]  Rajesh P. N. Rao,et al.  Robotic imitation from human motion capture using Gaussian processes , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[22]  G. Logan,et al.  Converging Evidence for a Fronto-Basal-Ganglia Network for Inhibitory Control of Action and Cognition , 2007, The Journal of Neuroscience.

[23]  Andreas Birk,et al.  Interdisciplinary approaches to robot learning , 2000 .

[24]  M. Matarić,et al.  Task Learning Through Imitation and Human-Robot Interaction , 2004 .

[25]  Nicolas Schweighofer,et al.  Motor learning without doing: trial-by-trial improvement in motor performance during mental training. , 2010, Journal of neurophysiology.

[26]  Pyung Hun Chang,et al.  Relative Impedance Control for Dual-Arm Robots Performing Asymmetric Bimanual Tasks , 2014, IEEE Transactions on Industrial Electronics.

[27]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[28]  José Luis Contreras-Vidal,et al.  A neural model of basal ganglia-thalamocortical relations in normal and parkinsonian movement , 1995, Biological Cybernetics.

[29]  Giulio Sandini,et al.  A biomimetic, force-field based computational model for motion planning and bimanual coordination in humanoid robots , 2009, Auton. Robots.

[30]  Ferdinand Freudenstein,et al.  Kinematic Synthesis of Linkages , 1965 .

[31]  Iroise Dumontheil,et al.  The gateway hypothesis of rostral prefrontal cortex (area 10) function , 2007, Trends in Cognitive Sciences.

[32]  M. Jeannerod.  Mental imagery in the motor context , 1995, Neuropsychologia.

[33]  S. Schaal.  The Computational Neurobiology of Reaching and Pointing — A Foundation for Motor Learning by Reza Shadmehr and Steven P. Wise , 2007 .

[34]  Stefano Caselli,et al.  Trajectory clustering and stochastic approximation for robot programming by demonstration , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  J A Reggia,et al.  Controlling working memory with learned instructions. , 2013, Neural networks : the official journal of the International Neural Network Society.

[36]  J. Fiala,et al.  Neural network models of motor timing and coordination , 1996 .

[37]  Kerstin Dautenhahn,et al.  Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation a , 2000 .

[38]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[39]  A. Liegeois,et al.  Automatic supervisory control of the configuration and behavior of multi-body mechanisms , 1977 .

[40]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[41]  Shigeki Sugano,et al.  Imitating others by composition of primitive actions: A neuro-dynamic model , 2012, Robotics Auton. Syst..

[42]  K. Doya,et al.  A unifying computational framework for motor control and social interaction. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[43]  S. Blakemore,et al.  Action prediction in the cerebellum and in the parietal lobe , 2003, Experimental Brain Research.

[44]  M. Shanahan.  A cognitive architecture that combines internal simulation with a global workspace , 2006, Consciousness and Cognition.

[45]  D. Wolpert,et al.  Maintaining internal representations: the role of the human superior parietal lobe , 1998, Nature Neuroscience.

[46]  Pietro G. Morasso,et al.  Self-Organization, Computational Maps, and Motor Control , 1997 .

[47]  Aymeric Guillot,et al.  Autonomic nervous system activities during imagined movements , 2010 .

[48]  Angelo Cangelosi,et al.  Towards a Bio-inspired Cognitive Architecture for Short-Term Memory in Humanoid Robots , 2012, TAROS.

[49]  M. Jeannerod,et al.  Mental simulation of an action modulates the excitability of spinal reflex pathways in man. , 1997, Brain research. Cognitive brain research.

[50]  Jun Tani,et al.  Model-based learning for mobile robot navigation from the dynamical systems perspective , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[51]  W. Hauber.  Involvement of basal ganglia transmitter systems in movement initiation , 1998, Progress in Neurobiology.

[52]  G. Fink,et al.  REVIEW: The functional organization of the intraparietal sulcus in humans and monkeys , 2005, Journal of anatomy.

[53]  J. Hollerbach,et al.  Programming and control of kinematically redundant manipulators , 1984, The 23rd IEEE Conference on Decision and Control.

[54]  M. Erb,et al.  Activation of Cortical and Cerebellar Motor Areas during Executed and Imagined Hand Movements: An fMRI Study , 1999, Journal of Cognitive Neuroscience.

[55]  Rodolphe J. Gentili,et al.  Integration of Gravitational Torques in Cerebellar Pathways Allows for the Dynamic Inverse Computation of Vertical Pointing Movements of a Robot Arm , 2009, PloS one.

[56]  M. Jeannerod,et al.  Mentally simulated movements in virtual reality: does Fitt's law hold in motor imagery? , 1995, Behavioural Brain Research.

[57]  E. G. Cruz,et al.  Stereotypical fingertip trajectories during grasp. , 2003, Journal of neurophysiology.

[58]  Juan López Coronado,et al.  A modular neural network linking Hyper RBF and AVITE models for reaching moving objects , 2005, Robotica.

[59]  J. Mazziotta,et al.  Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[60]  Philip E. Martin,et al.  Muscular activity during real and imagined movements: a test of inflow explanations. , 2002 .

[61]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[62]  P Jissendi,et al.  Brain areas involved in the control of speed during a motor sequence of the foot: real movement versus mental imagery. , 2013, Journal of neuroradiology. Journal de neuroradiologie.

[63]  J. Gordon,et al.  Accuracy of planar reaching movements , 1994, Experimental Brain Research.

[64]  Gordon Cheng,et al.  Dexterous Skills Transfer by Extending Human Body Schema to a Robotic Hand , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[65]  S. Grossberg,et al.  A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm , 1993, Journal of Cognitive Neuroscience.

[66]  Dana H. Ballard,et al.  Recognizing teleoperated manipulations , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[67]  K. Dautenhahn,et al.  The correspondence problem , 2002 .

[68]  Jun Tani,et al.  Development of hierarchical structures for actions and motor imagery: a constructivist view from synthetic neuro-robotics study , 2009, Psychological research.

[69]  Aude Billard,et al.  Reaching with multi-referential dynamical systems , 2008, Auton. Robots.

[70]  F. Guenther,et al.  Neural Models for Flexible Control of Redundant Systems , 1997 .

[71]  Michael Arbib,et al.  Extending the mirror neuron system model, I. Audible actions and invisible grasps. , 2007, Biological cybernetics.

[72]  James A. Reggia,et al.  Modeling of visuospatial perspectives processing and modulation of the fronto-parietal network activity during action imitation , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[73]  M. Arbib,et al.  Infant grasp learning: a computational model , 2004, Experimental Brain Research.

[74]  E. Bizzi,et al.  Human arm trajectory formation. , 1982, Brain : a journal of neurology.

[75]  Victor Frak,et al.  A dissociation between real and simulated movements in Parkinson's disease , 2004, Neuroreport.

[76]  R. Passingham,et al.  Attention to Intention , 2004, Science.

[77]  R. Miall,et al.  Connecting mirror neurons and forward models. , 2003, Neuroreport.

[78]  James A. Reggia,et al.  Cortex inspired model for inverse kinematics computation for a humanoid robotic finger , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[79]  F. P. de Lange,et al.  Motor imagery of gait: a quantitative approach , 2007, Experimental Brain Research.

[80]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[81]  E. Chao,et al.  Passive motion of the elbow joint. , 1976, The Journal of bone and joint surgery. American volume.

[82]  P. Morasso.  Spatial control of arm movements , 2004, Experimental Brain Research.

[83]  J. Gregory Trafton,et al.  Integrating cognition, perception and action through mental simulation in robots , 2004, Robotics Auton. Syst..

[84]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[85]  Giuseppe Cotugno,et al.  Dynamic Motion Control: Adaptive Bimanual Grasping for a Humanoid Robot , 2011, Fundam. Informaticae.

[86]  Aude Billard,et al.  Discriminative and adaptive imitation in uni-manual and bi-manual tasks , 2006, Robotics Auton. Syst..

[87]  K. Zentgraf,et al.  Cognitive motor processes: The role of motor imagery in the study of motor representations , 2009, Brain Research Reviews.

[88]  C. Collet,et al.  Modulation of EMG power spectrum frequency during motor imagery , 2008, Neuroscience Letters.

[89]  Aude Billard,et al.  Biologically Inspired Multimodal Integration: Interferences in a Human-Robot Interaction Game , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[90]  Ron Chrisley,et al.  Cognitive Map Construction and Use: A Parallel Distributed Processing Approach , 1991 .

[91]  Raymond H. Cuijpers,et al.  Goals and means in action observation: A computational approach , 2006, Neural Networks.

[92]  Y Uno,et al.  Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. , 1999, Journal of neurophysiology.

[93]  M. Arbib,et al.  Mirror neurons: Functions, mechanisms and models , 2013, Neuroscience Letters.

[94]  Marco Schieppati,et al.  Imagined and actual arm movements have similar durations when performed under different conditions of direction and mass , 2002, Experimental Brain Research.

[95]  Monica N. Nicolescu,et al.  Task Learning through Imitation and Human-robot Interaction , 2005 .

[96]  Fumiya Iida,et al.  The challenges ahead for bio-inspired 'soft' robotics , 2012, CACM.

[97]  Juan López Coronado,et al.  A modular neural network architecture for step-wise learning of grasping tasks , 2007, Neural Networks.

[98]  Tom Ziemke,et al.  Internal simulation of perception: a minimal neuro-robotic model , 2005, Neurocomputing.

[99]  J. Tani.  On the Interactions Between Top-Down Anticipation and Bottom-Up Regression , 2007, Frontiers in neurorobotics.

[100]  P. Jackson,et al.  The neural network of motor imagery: An ALE meta-analysis , 2013, Neuroscience & Biobehavioral Reviews.

[101]  James A. Reggia,et al.  Learning of spatial relationships between observed and imitated actions allows invariant inverse computation in the frontal mirror neuron system , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[102]  Iwona Stepniewska,et al.  Thalamocortical connections of functional zones in posterior parietal cortex and frontal cortex motor regions in New World monkeys. , 2010, Cerebral cortex.

[103]  G. Logan,et al.  In search of the point of no return: the control of response processes. , 1990, Journal of experimental psychology. Human perception and performance.

[104]  T. Pozzo,et al.  Improvement and generalization of arm motor performance through motor imagery practice , 2006, Neuroscience.

[105]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[106]  Giorgio Metta,et al.  Actions and Imagined Actions in Cognitive Robots , 2011 .

[107]  Robert O. Ambrose,et al.  Robonaut 2 - The first humanoid robot in space , 2011, 2011 IEEE International Conference on Robotics and Automation.

[108]  M. Lotze,et al.  Motor imagery , 2006, Journal of Physiology-Paris.

[109]  S. K. Tso,et al.  Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories , 1996, Proceedings of the IEEE International Conference on Industrial Technology (ICIT'96).

[110]  Aaron M. Dollar,et al.  A Hand-Centric Classification of Human and Robot Dexterous Manipulation , 2013, IEEE Transactions on Haptics.

[111]  A. Guillot,et al.  Imagining is Not Doing but Involves Specific Motor Commands: A Review of Experimental Data Related to Motor Inhibition , 2012, Front. Hum. Neurosci..

[112]  Deb Roy,et al.  Mental imagery for a conversational robot , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[113]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[114]  M. Jeannerod.  Neural Simulation of Action: A Unifying Mechanism for Motor Cognition , 2001, NeuroImage.

[115]  E. Visalberghi,et al.  Acting and understanding: Tool use revisited through the minds of capuchin monkeys , 1996 .

[116]  Michael A. Arbib,et al.  Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons , 2010, Biological Cybernetics.

[117]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[118]  C Collet,et al.  Muscular responses during motor imagery as a function of muscle contraction types. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[119]  Yiannis Demiris,et al.  Imitation as a dual-route process featuring prediction and learning components: A biologically plaus , 2002 .

[120]  Alain Berthoz,et al.  Comparison of actual and imagined execution of whole-body movements after a long exposure to microgravity , 2003, Neuroscience Letters.

[121]  F. Lacquaniti,et al.  Eye-hand coordination during reaching. I. Anatomical relationships between parietal and frontal cortex. , 2001, Cerebral cortex.

[122]  R. Andersen,et al.  The posterior parietal cortex: Sensorimotor interface for the planning and online control of visually guided movements , 2006, Neuropsychologia.