Getting a handle on virtual tools: An examination of the neuronal activity associated with virtual tool use

&NA; Tool use is associated with three visual streams—dorso‐dorsal, ventro‐dorsal, and ventral visual streams. These streams are involved in processing online motor planning, action semantics, and tool semantics features, respectively. Little is known about the way in which the brain represents virtual tools. To directly assess this question, a virtual tool paradigm was created that provided the ability to manipulate tool components in isolation of one another. During functional magnetic resonance imaging (fMRI), adult participants performed a series of virtual tool manipulation tasks in which vision and movement kinematics of the tool were manipulated. Reaction time and hand movement direction were monitored while the tasks were performed. Functional imaging revealed that activity within all three visual streams was present, in a similar pattern to what would be expected with physical tool use. However, a previously unreported network of right‐hemisphere activity was found including right inferior parietal lobule, middle and superior temporal gyri and supramarginal gyrus – regions well known to be associated with tool processing within the left hemisphere. These results provide evidence that both virtual and physical tools are processed within the same brain regions, though virtual tools recruit bilateral tool processing regions to a greater extent than physical tools. HighlightsPhysical tool use is supported by three visual streams within the brain.Virtual tools appear to rely on many of the same brain systems as physical tool use.A more bilateral recruitment of tool use areas is observed during virtual tool use.

[1]  Sarah H. Creem-Regehr,et al.  Neural representations of graspable objects: are tools special? , 2005, Brain research. Cognitive brain research.

[2]  J. Haxby,et al.  fMRI Responses to Video and Point-Light Displays of Moving Humans and Manipulable Objects , 2003, Journal of Cognitive Neuroscience.

[3]  Daniel M. Corcos,et al.  Three-dimensional locations and boundaries of motor and premotor cortices as defined by functional brain imaging: A meta-analysis , 2006, NeuroImage.

[4]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[5]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[6]  D. Brooks,et al.  Motor sequence learning: a study with positron emission tomography , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[7]  Ashwin G Ramayya,et al.  A DTI investigation of neural substrates supporting tool use. , 2010, Cerebral cortex.

[8]  Christopher R. Madan,et al.  Tool selection and the ventral‐dorsal organization of tool‐related knowledge , 2017, Physiological reports.

[9]  Edoardo Battaglia,et al.  ThimbleSense: An individual-digit wearable tactile sensor for experimental grasp studies , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Gereon R. Fink,et al.  Neural basis of pantomiming the use of visually presented objects , 2004, NeuroImage.

[11]  Guy A. Orban,et al.  The neural basis of human tool use , 2014, Front. Psychol..

[12]  R. Passingham,et al.  Temporary interference in human lateral premotor cortex suggests dominance for the selection of movements. A study using transcranial magnetic stimulation. , 1998, Brain : a journal of neurology.

[13]  Ryo Ishibashi,et al.  The neural network for tool-related cognition: An activation likelihood estimation meta-analysis of 70 neuroimaging contrasts , 2016, Cognitive neuropsychology.

[14]  Y. Rossetti,et al.  No double-dissociation between optic ataxia and visual agnosia: Multiple sub-streams for multiple visuo-manual integrations , 2006, Neuropsychologia.

[15]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[16]  Ravi S. Menon,et al.  A comparison of frontoparietal fMRI activation during anti-saccades and anti-pointing. , 2000, Journal of neurophysiology.

[17]  Jonathan S. Cant,et al.  Independent Processing of Form, Colour, and Texture in Object Perception , 2008, Perception.

[18]  G. Króliczak,et al.  Planning Functional Grasps of Simple Tools Invokes the Hand-independent Praxis Representation Network: An fMRI Study , 2017, Journal of the International Neuropsychological Society.

[19]  D. Boussaoud,et al.  Projections of the claustrum to the primary motor, premotor, and prefrontal cortices in the macaque monkey , 2002, The Journal of comparative neurology.

[20]  Ravi S. Menon,et al.  Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas , 2003, Experimental Brain Research.

[21]  M. Goodale,et al.  Two visual systems re-viewed , 2008, Neuropsychologia.

[22]  S. Frey What Puts the How in Where? Tool Use and the Divided Visual Streams Hypothesis , 2007, Cortex.

[23]  M. Brett,et al.  Actions Speak Louder Than Functions: The Importance of Manipulability and Action in Tool Representation , 2003, Journal of Cognitive Neuroscience.

[24]  P. Roland,et al.  Supplementary motor area and other cortical areas in organization of voluntary movements in man. , 1980, Journal of neurophysiology.

[25]  G. Rizzolatti,et al.  Two different streams form the dorsal visual system: anatomy and functions , 2003, Experimental Brain Research.

[26]  L. Buxbaum,et al.  Action knowledge, visuomotor activation, and embodiment in the two action systems , 2010, Annals of the New York Academy of Sciences.

[27]  Scott T. Grafton,et al.  Premotor Cortex Activation during Observation and Naming of Familiar Tools , 1997, NeuroImage.

[28]  Scott T. Grafton,et al.  A distributed left hemisphere network active during planning of everyday tool use skills. , 2004, Cerebral cortex.

[29]  M. Goodale,et al.  The visual brain in action , 1995 .

[30]  R. Passingham,et al.  Objects automatically potentiate action: an fMRI study of implicit processing , 2003, The European journal of neuroscience.

[31]  Umberto Castiello,et al.  Human inferior parietal cortex ‘programs’ the action class of grasping , 2000, Cognitive Systems Research.

[32]  Jody C Culham,et al.  Behavioral / Systems / Cognitive Functional Magnetic Resonance Imaging Reveals the Neural Substrates of Arm Transport and Grip Formation in Reach-to-Grasp Actions in Humans , 2010 .

[33]  J. Hermsdörfer,et al.  It takes the whole brain to make a cup of coffee: the neuropsychology of naturalistic actions involving technical devices , 2005, Neuropsychologia.

[34]  Jonathan S. Cant,et al.  Scratching Beneath the Surface: New Insights into the Functional Properties of the Lateral Occipital Area and Parahippocampal Place Area , 2011, The Journal of Neuroscience.

[35]  Jody C Culham,et al.  Is That within Reach? fMRI Reveals That the Human Superior Parieto-Occipital Cortex Encodes Objects Reachable by the Hand , 2009, The Journal of Neuroscience.

[36]  S. Zeki,et al.  A visuo‐somatomotor pathway through superior parietal cortex in the macaque monkey: cortical connections of areas V6 and V6A , 1998, The European journal of neuroscience.

[37]  Sheng Zhang,et al.  Functional Clustering of the Human Inferior Parietal Lobule by Whole-Brain Connectivity Mapping of Resting-State Functional Magnetic Resonance Imaging Signals , 2014, Brain Connect..

[38]  Volkmar Glauche,et al.  The ventral fiber pathway for pantomime of object use , 2015, NeuroImage.

[39]  G. Vingerhoets,et al.  Neural correlates of pantomiming familiar and unfamiliar tools: Action semantics versus mechanical problem solving? , 2011, Human brain mapping.

[40]  Bradford Z. Mahon,et al.  Action-Related Properties Shape Object Representations in the Ventral Stream , 2007, Neuron.

[41]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[42]  Jonathan S. Cant,et al.  Cerebral Cortex Advance Access published April 28, 2006 Attention to Form or Surface Properties Modulates Different Regions of Human , 2022 .

[43]  F. Osiurak,et al.  A cognitive-based model of tool use in normal aging , 2017, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[44]  Michela Gamberini,et al.  The Most Direct Visual Pathway to the Frontal Cortex , 2004, Cortex.

[45]  C. Galletti,et al.  The cortical connections of area V6: an occipito‐parietal network processing visual information , 2001, The European journal of neuroscience.

[46]  Jonathan S. Cant,et al.  Asymmetric interference between the perception of shape and the perception of surface properties. , 2009, Journal of vision.

[47]  M. Kawato,et al.  Modular organization of internal models of tools in the human cerebellum , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Afra Wohlschläger,et al.  The Neural Correlates of Planning and Executing Actual Tool Use , 2014, The Journal of Neuroscience.

[49]  J. Haxby,et al.  Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects , 1999, Nature Neuroscience.

[50]  Emanuelle Reynaud,et al.  On the neurocognitive origins of human tool use : A critical review of neuroimaging data , 2016, Neuroscience & Biobehavioral Reviews.

[51]  T. Schormann,et al.  Activation in the Ipsilateral Posterior Parietal Cortex during Tool Use: A PET Study , 2001, NeuroImage.

[52]  G. Orban,et al.  The Representation of Tool Use in Humans and Monkeys: Common and Uniquely Human Features , 2009, The Journal of Neuroscience.

[53]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[54]  Scott T. Grafton,et al.  Graspable objects grab attention when the potential for action is recognized , 2003, Nature Neuroscience.

[55]  Lee A Baugh,et al.  Hand-held tools with complex kinematics are efficiently incorporated into movement planning and online control. , 2012, Journal of neurophysiology.

[56]  Hiroshi Imamizu,et al.  Human cerebellar activity reflecting an acquired internal model of a new tool , 2000, Nature.

[57]  Scott T. Grafton,et al.  Cortical topography of human anterior intraparietal cortex active during visually guided grasping. , 2005, Brain research. Cognitive brain research.

[58]  Leslie G. Ungerleider Two cortical visual systems , 1982 .

[59]  Andrea Serino,et al.  Everyday use of the computer mouse extends peripersonal space representation , 2010, Neuropsychologia.

[60]  R. E Passingham,et al.  Cerebral dominance for action in the human brain: the selection of actions , 2001, Neuropsychologia.

[61]  Mark Drakesmith,et al.  Posterior middle temporal gyrus is involved in verbal and non-verbal semantic cognition: Evidence from rTMS , 2012 .

[62]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[63]  H. Onoe,et al.  Functional Brain Mapping of Monkey Tool Use , 2001, NeuroImage.

[64]  K. Zilles,et al.  Neural consequences of acting in near versus far space: a physiological basis for clinical dissociations. , 2000, Brain : a journal of neurology.

[65]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[66]  Jacqueline C. Snow,et al.  What Role Does “Elongation” Play in “Tool-Specific” Activation and Connectivity in the Dorsal and Ventral Visual Streams? , 2018, Cerebral cortex.

[67]  P. Matthews,et al.  The role of ipsilateral premotor cortex in hand movement after stroke , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[68]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[69]  Scott H. Johnson-Frey The neural bases of complex tool use in humans , 2004, Trends in Cognitive Sciences.

[70]  G Rizzolatti,et al.  When pliers become fingers in the monkey motor system , 2008, Proceedings of the National Academy of Sciences.

[71]  Anjali Krishnan,et al.  Cluster-extent based thresholding in fMRI analyses: Pitfalls and recommendations , 2014, NeuroImage.

[72]  E. DeYoe,et al.  Distinct Cortical Pathways for Processing Tool versus Animal Sounds , 2005, The Journal of Neuroscience.

[73]  R. Klatzky,et al.  Cognitive representations of hand posture in ideomotor apraxia , 2003, Neuropsychologia.

[74]  Jody C. Culham,et al.  Observing Learned Object-specific Functional Grasps Preferentially Activates the Ventral Stream , 2010, Journal of Cognitive Neuroscience.

[75]  J. Decety,et al.  Does visual perception of object afford action? Evidence from a neuroimaging study , 2002, Neuropsychologia.

[76]  Jason B. Mattingley,et al.  Motor role of human inferior parietal lobe revealed in unilateral neglect patients , 1998, Nature.

[77]  M. Mishkin,et al.  OCCIPITOTEMPORAL CORTICOCORTICAL CONNECTIONS IN THE RHESUS MONKEY. , 1965, Experimental neurology.

[78]  Agnès Roby-Brami,et al.  The use of a tool requires its incorporation into the movement: Evidence from stick-pointing in apraxia , 2009, Cortex.

[79]  R. J. Seitz,et al.  A parieto-premotor network for object manipulation: evidence from neuroimaging , 1999, Experimental Brain Research.

[80]  D. Na,et al.  Functional magnetic resonance imaging during pantomiming tool-use gestures , 2001, Experimental Brain Research.

[81]  F. Binkofski,et al.  Two action systems in the human brain , 2013, Brain and Language.

[82]  R. J. Seitz,et al.  A fronto‐parietal circuit for object manipulation in man: evidence from an fMRI‐study , 1999, The European journal of neuroscience.

[83]  Jody C Culham,et al.  Decoding the neural mechanisms of human tool use , 2013, eLife.