Generalisation, decision making, and embodiment effects in mental rotation: A neurorobotic architecture tested with a humanoid robot

Mental rotation, a classic experimental paradigm of cognitive psychology, tests the capacity of humans to mentally rotate a seen object to decide if it matches a target object. In recent years, mental rotation has been investigated with brain imaging techniques to identify the brain areas involved. Mental rotation has also been investigated through the development of neural-network models, used to identify the specific mechanisms that underlie its process, and with neurorobotics models to investigate its embodied nature. Current models, however, have limited capacities to relate to neuro-scientific evidence, to generalise mental rotation to new objects, to suitably represent decision making mechanisms, and to allow the study of the effects of overt gestures on mental rotation. The work presented in this study overcomes these limitations by proposing a novel neurorobotic model that has a macro-architecture constrained by knowledge held on brain, encompasses a rather general mental rotation mechanism, and incorporates a biologically plausible decision making mechanism. The model was tested using the humanoid robot iCub in tasks requiring the robot to mentally rotate 2D geometrical images appearing on a computer screen. The results show that the robot gained an enhanced capacity to generalise mental rotation to new objects and to express the possible effects of overt movements of the wrist on mental rotation. The model also represents a further step in the identification of the embodied neural mechanisms that may underlie mental rotation in humans and might also give hints to enhance robots' planning capabilities.

[1]  Giulio Sandini,et al.  The iCub Cognitive Humanoid Robot: An Open-System Research Platform for Enactive Cognition , 2006, 50 Years of Artificial Intelligence.

[2]  S. Kosslyn,et al.  Motor processes in mental rotation , 1998, Cognition.

[3]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[4]  Hang Zhang,et al.  Planning Routes Across Economic Terrains: Maximizing Utility, Following Heuristics , 2010, Front. Psychology.

[5]  D. Parisi,et al.  Integrating reinforcement learning, equilibrium points, and minimum variance to understand the development of reaching: a computational model. , 2014, Psychological review.

[6]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[7]  Olivier Sigaud,et al.  Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems , 2003 .

[8]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[9]  Carlo Miniussi,et al.  Parietal Lobe Contribution to Mental Rotation Demonstrated with rTMS , 2003, Journal of Cognitive Neuroscience.

[10]  A. Clark Being There: Putting Brain, Body, and World Together Again , 1996 .

[11]  Gianluca Baldassarre,et al.  Forward and Bidirectional Planning Based on Reinforcement Learning and Neural Networks in a Simulated Robot , 2003, ABiALS.

[12]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[13]  K. Gurney,et al.  The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis , 2013, Front. Behav. Neurosci..

[14]  Francesco Mannella,et al.  The Hierarchical Organisation of Cortical and Basal-Ganglia Systems: A Computationally-Informed Review and Integrated Hypothesis , 2013, Computational and Robotic Models of the Hierarchical Organization of Behavior.

[15]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[16]  Tom Ziemke,et al.  Internal simulation of perception: a minimal neuro-robotic model , 2005, Neurocomputing.

[17]  S. Kita,et al.  The nature of gestures ' beneficial role in spatial problem solving , 2013 .

[18]  A. Wohlschläger,et al.  Mental and manual rotation. , 1998, Journal of experimental psychology. Human perception and performance.

[19]  Angelo Cangelosi,et al.  A cognitive robotic model of mental rotation , 2013, 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB).

[20]  N. Thacker,et al.  Functional contribution of medial premotor cortex to visuo-spatial transformation in humans , 2004, Neuroscience Letters.

[21]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Richard A. Andersen,et al.  Separate body- and world-referenced representations of visual space in parietal cortex , 1998, Nature.

[23]  A. D. Kulkarni,et al.  Neural networks for invariant object recognition , 1990, Proceedings of the 1990 Symposium on Applied Computing.

[24]  Loredana Zollo,et al.  Interplay of Rhythmic and Discrete Manipulation Movements During Development: A Policy-Search Reinforcement-Learning Robot Model , 2016, IEEE Transactions on Cognitive and Developmental Systems.

[25]  R. Passingham,et al.  Functional anatomy of the mental representation of upper extremity movements in healthy subjects. , 1995, Journal of neurophysiology.

[26]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1986 .

[27]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[28]  Gianluca Baldassarre,et al.  A modular neural-network model of the basal ganglia’s role in learning and selecting motor behaviours , 2002, Cognitive Systems Research.

[29]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[30]  A. Wohlschläger Mental object rotation and the planning of hand movements , 2001, Perception & psychophysics.

[31]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[32]  Dimitri Ognibene,et al.  Ecological Active Vision: Four Bioinspired Principles to Integrate Bottom–Up and Adaptive Top–Down Attention Tested With a Simple Camera-Arm Robot , 2015, IEEE Transactions on Autonomous Mental Development.

[33]  G. Rizzolatti,et al.  The organization of the cortical motor system: new concepts. , 1998, Electroencephalography and clinical neurophysiology.

[34]  S. Kosslyn Image and brain: the resolution of the imagery debate , 1994 .

[35]  Martin V. Butz,et al.  Anticipatory Behavior in Adaptive Learning Systems , 2003, Lecture Notes in Computer Science.

[36]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[37]  J. T. Massey,et al.  Mental rotation of the neuronal population vector. , 1989, Science.

[38]  R. McLaren,et al.  Interaction between perceived and imagined rotation. , 1982, Journal of experimental psychology. Human perception and performance.

[39]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[40]  O. Braddick,et al.  Brain Areas Sensitive to Coherent Visual Motion , 2001, Perception.

[41]  R. Bogacz Optimal decision-making theories: linking neurobiology with behaviour , 2007, Trends in Cognitive Sciences.

[42]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  R. Andersen,et al.  Intentional maps in posterior parietal cortex. , 2002, Annual review of neuroscience.

[44]  Shinichi Tamura,et al.  Mental Rotation by Neural Network , 2009, 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[45]  D. Parisi,et al.  TRoPICALS: a computational embodied neuroscience model of compatibility effects. , 2010, Psychological review.

[46]  Mark S. Cohen,et al.  Localization of brain function using magnetic resonance imaging , 1994, Trends in Neurosciences.

[47]  Gianluca Baldassarre,et al.  Planning with neural networks and reinforcement learning , 2001 .

[48]  R. Ellis,et al.  The potentiation of grasp types during visual object categorization , 2001 .

[49]  Jeffrey M. Zacks,et al.  Lateral somatotopic organization during imagined and prepared movements. , 2006, Journal of neurophysiology.

[50]  S. Kosslyn Image and Brain: The Resolution of the Imagery Debate , 1994, Journal of Cognitive Neuroscience.

[51]  Jean-Arcady Meyer,et al.  BIOLOGICALLY BASED ARTIFICIAL NAVIGATION SYSTEMS: REVIEW AND PROSPECTS , 1997, Progress in Neurobiology.

[52]  Toshio Inui,et al.  Temporo-Parietal Network Model for 3D Mental Rotation , 2011 .

[53]  P. Strick,et al.  Basal ganglia and cerebellar loops: motor and cognitive circuits , 2000, Brain Research Reviews.

[54]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[55]  A. Cangelosi,et al.  Developmental Robotics: From Babies to Robots , 2015 .

[56]  Angelo Cangelosi,et al.  Modelling mental rotation in cognitive robots , 2013, Adapt. Behav..

[57]  S. Kosslyn,et al.  Mental rotation of objects versus hands: neural mechanisms revealed by positron emission tomography. , 1998, Psychophysiology.

[58]  David Filliat,et al.  Map-based navigation in mobile robots: II. A review of map-learning and path-planning strategies , 2003, Cognitive Systems Research.

[59]  Angelo Cangelosi,et al.  An open-source simulator for cognitive robotics research: the prototype of the iCub humanoid robot simulator , 2008, PerMIS.

[60]  Alexander V. Terekhov,et al.  Space as an invention of biological organisms , 2013, ArXiv.

[61]  A. Cangelosi,et al.  How affordances associated with a distractor object affect compatibility effects: A study with the computational model TRoPICALS , 2013, Psychological research.

[62]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[63]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[64]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[65]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[66]  Minoru Fukumi,et al.  Rotation-invariant neural pattern recognition system estimating a rotation angle , 1997, IEEE Trans. Neural Networks.

[67]  G. Schöner,et al.  Dynamic Field Theory of Movement Preparation , 2022 .

[68]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[69]  J. Hochberg,et al.  The effect of landmark features on mental rotation times , 1977, Memory & cognition.

[70]  G. Pezzulo,et al.  Neuroscience and Biobehavioral Reviews the Contribution of Brain Sub-cortical Loops in the Expression and Acquisition of Action Understanding Abilities , 2022 .

[71]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[72]  A. Cangelosi,et al.  Mental practice and verbal instructions execution: A cognitive robotics study , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[73]  Jeffrey M. Zacks,et al.  Neuroimaging Studies of Mental Rotation: A Meta-analysis and Review , 2008, Journal of Cognitive Neuroscience.

[74]  D. Hubel Eye, brain, and vision , 1988 .

[75]  Ellips Masehian,et al.  Classic and Heuristic Approaches in Robot Motion Planning A Chronological Review , 2007 .

[76]  Francesco Mannella,et al.  Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. , 2013, Neural networks : the official journal of the International Neural Network Society.

[77]  A. Borghi,et al.  Embodied cognition and beyond: Acting and sensing the body , 2010, Neuropsychologia.

[78]  Eugenio Guglielmelli,et al.  The Role of Learning and Kinematic Features in Dexterous Manipulation: A Comparative Study with Two Robotic Hands , 2013 .

[79]  Scott T. Grafton,et al.  Localization of grasp representations in humans by positron emission tomography , 1996, Experimental Brain Research.

[80]  Christian Windischberger,et al.  Corrigendum to “The functional role of dorso-lateral premotor cortex during mental rotation. An event-related fMRI study separating cognitive processing steps using a novel task paradigm” [NeuroImage 36 (2007) 1374–1386] , 2007, NeuroImage.

[81]  Minoru Fukumi,et al.  Rotation-invariant neural pattern recognition system with application to coin recognition , 1992, IEEE Trans. Neural Networks.

[82]  K. Doya Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[83]  Angelo Rega,et al.  A Model of Reaching that Integrates Reinforcement Learning and Population Encoding of Postures , 2006, SAB.

[84]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[85]  Ravi S. Menon,et al.  Motor Area Activity During Mental Rotation Studied by Time-Resolved Single-Trial fMRI , 2000, Journal of Cognitive Neuroscience.

[86]  Teuvo Kohonen,et al.  Self-Organizing Maps, Third Edition , 2001, Springer Series in Information Sciences.

[87]  Luca Tummolini,et al.  The embodied mind extended: using words as social tools , 2013, Front. Psychol..

[88]  Davide Marocco,et al.  Special issue on artificial mental imagery in cognitive systems and robotics , 2013, Adapt. Behav..

[89]  Rick Grush,et al.  The emulation theory of representation: Motor control, imagery, and perception , 2004, Behavioral and Brain Sciences.

[90]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[91]  Domenico Parisi,et al.  Using Motor Babbling and Hebb Rules for Modeling the Development of Reaching with Obstacles and Grasping , 2008 .

[92]  M. Just,et al.  Graded Functional Activation in the Visuospatial System with the Amount of Task Demand , 1999, Journal of Cognitive Neuroscience.