Decoding the neural mechanisms of human tool use

Sophisticated tool use is a defining characteristic of the primate species but how is it supported by the brain, particularly the human brain? Here we show, using functional MRI and pattern classification methods, that tool use is subserved by multiple distributed action-centred neural representations that are both shared with and distinct from those of the hand. In areas of frontoparietal cortex we found a common representation for planned hand- and tool-related actions. In contrast, in parietal and occipitotemporal regions implicated in hand actions and body perception we found that coding remained selectively linked to upcoming actions of the hand whereas in parietal and occipitotemporal regions implicated in tool-related processing the coding remained selectively linked to upcoming actions of the tool. The highly specialized and hierarchical nature of this coding suggests that hand- and tool-related actions are represented separately at earlier levels of sensorimotor processing before becoming integrated in frontoparietal cortex. DOI: http://dx.doi.org/10.7554/eLife.00425.001

[1]  N. Kanwisher,et al.  Domain specificity in visual cortex. , 2006, Cerebral cortex.

[2]  H. Sakata,et al.  Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. , 2000, Journal of neurophysiology.

[3]  P. Downing,et al.  The role of occipitotemporal body-selective regions in person perception , 2011, Cognitive neuroscience.

[4]  G. Luppino,et al.  Visual responses in the dorsal premotor area F2 of the macaque monkey , 1999, Experimental Brain Research.

[5]  Jody C Culham,et al.  Is That within Reach? fMRI Reveals That the Human Superior Parieto-Occipital Cortex Encodes Objects Reachable by the Hand , 2009, The Journal of Neuroscience.

[6]  C. Sherrington Integrative Action of the Nervous System , 1907 .

[7]  L. Fogassi,et al.  Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. , 2006, Journal of neurophysiology.

[8]  Flavia Filimon Human Cortical Control of Hand Movements: Parietofrontal Networks for Reaching, Grasping, and Pointing , 2010, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[9]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[10]  A. Berti,et al.  When Far Becomes Near: Remapping of Space by Tool Use , 2000, Journal of Cognitive Neuroscience.

[11]  Paul Cisek,et al.  Neural activity in primary motor and dorsal premotor cortex in reaching tasks with the contralateral versus ipsilateral arm. , 2003, Journal of neurophysiology.

[12]  Chih-Jen Lin,et al.  A Comparison of Methods for Multi-class Support Vector Machines , 2015 .

[13]  Scott T. Grafton,et al.  Evidence for a distributed hierarchy of action representation in the brain. , 2007, Human movement science.

[14]  P. Strick,et al.  Imaging the premotor areas , 2001, Current Opinion in Neurobiology.

[15]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[16]  Paul Cisek,et al.  Cortical mechanisms of action selection: the affordance competition hypothesis , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[17]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[18]  Bradford Z. Mahon,et al.  Concepts and categories: a cognitive neuropsychological perspective. , 2009, Annual review of psychology.

[19]  Nikolaus Kriegeskorte,et al.  Pattern-information analysis: From stimulus decoding to computational-model testing , 2011, NeuroImage.

[20]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[21]  R. Hinde,et al.  Advances in the study of behavior , 1966 .

[22]  R. Andersen,et al.  Intentional maps in posterior parietal cortex. , 2002, Annual review of neuroscience.

[23]  K Tsutsui,et al.  Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[24]  N. Kanwisher Functional specificity in the human brain: A window into the functional architecture of the mind , 2010, Proceedings of the National Academy of Sciences.

[25]  Alex Martin,et al.  Grounding Object Concepts in Perception and Action: Evidence from FMRI Studies of Tools , 2007, Cortex.

[26]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[27]  Fraser W. Smith,et al.  Decoding Effector-Dependent and Effector-Independent Movement Intentions from Human Parieto-Frontal Brain Activity , 2011, The Journal of Neuroscience.

[28]  Ehud Zohary,et al.  Multiple Reference Frames for Saccadic Planning in the Human Parietal Cortex , 2011, The Journal of Neuroscience.

[29]  Talma Hendler,et al.  Center–periphery organization of human object areas , 2001, Nature Neuroscience.

[30]  D. Wolpert,et al.  Motor prediction , 2001, Current Biology.

[31]  I. Toni,et al.  Spatial and effector processing in the human parietofrontal network for reaches and saccades. , 2009, Journal of neurophysiology.

[32]  G Rizzolatti,et al.  When pliers become fingers in the monkey motor system , 2008, Proceedings of the National Academy of Sciences.

[33]  M. Arbib,et al.  Grasping objects: the cortical mechanisms of visuomotor transformation , 1995, Trends in Neurosciences.

[34]  N. Kanwisher,et al.  The Human Body , 2001 .

[35]  Fraser W. Smith,et al.  Nonstimulated early visual areas carry information about surrounding context , 2010, Proceedings of the National Academy of Sciences.

[36]  M. Brass,et al.  Unconscious determinants of free decisions in the human brain , 2008, Nature Neuroscience.

[37]  Anwar R. Padhani,et al.  Diffusion-Weighted Imaging , 2009 .

[38]  Scott T. Grafton,et al.  A distributed left hemisphere network active during planning of everyday tool use skills. , 2004, Cerebral cortex.

[39]  M. Goodale,et al.  The visual brain in action , 1995 .

[40]  Jonathan D. Nelson,et al.  Multiple Parietal Reach Regions in Humans: Cortical Representations for Visual and Proprioceptive Feedback during On-Line Reaching , 2009, The Journal of Neuroscience.

[41]  J. Culham,et al.  The role of parietal cortex in visuomotor control: What have we learned from neuroimaging? , 2006, Neuropsychologia.

[42]  H. Alkadhi,et al.  Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. , 1997, Brain : a journal of neurology.

[43]  W. Smith The Integrative Action of the Nervous System , 1907, Nature.

[44]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[45]  Gereon R. Fink,et al.  Neural basis of pantomiming the use of visually presented objects , 2004, NeuroImage.

[46]  A. Caramazza,et al.  Closely overlapping responses to tools and hands in left lateral occipitotemporal cortex. , 2012, Journal of neurophysiology.

[47]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[48]  L. Fogassi,et al.  Functional properties of grasping-related neurons in the dorsal premotor area F2 of the macaque monkey. , 2004, Journal of neurophysiology.

[49]  Jody C. Culham,et al.  Observing Learned Object-specific Functional Grasps Preferentially Activates the Ventral Stream , 2010, Journal of Cognitive Neuroscience.

[50]  Polina Golland,et al.  Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies , 2003, IPMI.

[51]  J. Decety,et al.  Does visual perception of object afford action? Evidence from a neuroimaging study , 2002, Neuropsychologia.

[52]  Timothy Edward John Behrens,et al.  Diffusion-Weighted Imaging Tractography-Based Parcellation of the Human Lateral Premotor Cortex Identifies Dorsal and Ventral Subregions with Anatomical and Functional Specializations , 2007, The Journal of Neuroscience.

[53]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[54]  Nadim Joni Shah,et al.  The extrastriate cortex distinguishes between the consequences of one's own and others' behavior , 2007, NeuroImage.

[55]  Clayton E. Curtis,et al.  Maintenance of Spatial and Motor Codes during Oculomotor Delayed Response Tasks , 2004, The Journal of Neuroscience.

[56]  Jun Tanji,et al.  Differential involvement of neurons in the dorsal and ventral premotor cortex during processing of visual signals for action planning. , 2006, Journal of neurophysiology.

[57]  Maurizio Gentilucci,et al.  Grasping an object naturally or with a tool: are these tasks guided by a common motor representation? , 2004, Experimental Brain Research.

[58]  G. Rizzolatti,et al.  Parietal Lobe: From Action Organization to Intention Understanding , 2005, Science.

[59]  Leslie G. Ungerleider,et al.  Discrete Cortical Regions Associated with Knowledge of Color and Knowledge of Action , 1995, Science.

[60]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[61]  J Randall Flanagan,et al.  Where One Hand Meets the Other: Limb-Specific and Action-Dependent Movement Plans Decoded from Preparatory Signals in Single Human Frontoparietal Brain Areas , 2013, The Journal of Neuroscience.

[62]  M. Tanaka,et al.  Coding of modified body schema during tool use by macaque postcentral neurones. , 1996, Neuroreport.

[63]  David G. Stork,et al.  Pattern Classification , 1973 .

[64]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[65]  A. Maravita,et al.  Tools for the body (schema) , 2004, Trends in Cognitive Sciences.

[66]  S. Frey What Puts the How in Where? Tool Use and the Divided Visual Streams Hypothesis , 2007, Cortex.

[67]  Alessandro Farnè,et al.  Grab an object with a tool and change your body: tool-use-dependent changes of body representation for action , 2012, Experimental Brain Research.

[68]  M. Corbetta,et al.  Extrastriate body area in human occipital cortex responds to the performance of motor actions , 2004, Nature Neuroscience.

[69]  Scott H. Frey,et al.  Human Anterior Intraparietal and Ventral Premotor Cortices Support Representations of Grasping with the Hand or a Novel Tool , 2010, Journal of Cognitive Neuroscience.

[70]  M. Davare,et al.  Behavioral / Systems / Cognitive Dissociating the Role of Ventral and Dorsal Premotor Cortex in Precision Grasping , 2018 .

[71]  J. M. Anderson,et al.  Organ transplantation and the fetal allograft. , 1978, British medical journal.

[72]  Richard S. J. Frackowiak,et al.  A Blueprint for Movement: Functional and Anatomical Representations in the Human Motor System , 1999, The Journal of Neuroscience.

[73]  R. Porter,et al.  Corticospinal Function and Voluntary Movement , 1993 .

[74]  R T Knight,et al.  Neural representations of skilled movement. , 2000, Brain : a journal of neurology.

[75]  Lloyd T. Elliott,et al.  Cortical surface-based searchlight decoding , 2011, NeuroImage.

[76]  Hanna Damasio,et al.  Predicting visual stimuli on the basis of activity in auditory cortices , 2010, Nature Neuroscience.

[77]  G. Rizzolatti,et al.  The Cortical Motor System , 2001, Neuron.

[78]  M. Arbib,et al.  Tool use and the distalization of the end-effector , 2009, Psychological research.

[79]  R. Andersen,et al.  Intention, Action Planning, and Decision Making in Parietal-Frontal Circuits , 2009, Neuron.

[80]  E. DeYoe,et al.  Distinct Cortical Pathways for Processing Tool versus Animal Sounds , 2005, The Journal of Neuroscience.

[81]  Jody C. Culham,et al.  Does tool-related fMRI activity within the intraparietal sulcus reflect the plan to grasp? , 2007, NeuroImage.

[82]  R. Andersen,et al.  Coding of intention in the posterior parietal cortex , 1997, Nature.

[83]  T. Lillicrap,et al.  Preference Distributions of Primary Motor Cortex Neurons Reflect Control Solutions Optimized for Limb Biomechanics , 2013, Neuron.

[84]  Christian Keysers,et al.  Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data , 2008, PloS one.

[85]  G. Orban,et al.  The Representation of Tool Use in Humans and Monkeys: Common and Uniquely Human Features , 2009, The Journal of Neuroscience.

[86]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[87]  Jody C Culham,et al.  Behavioral / Systems / Cognitive Functional Magnetic Resonance Imaging Reveals the Neural Substrates of Arm Transport and Grip Formation in Reach-to-Grasp Actions in Humans , 2010 .

[88]  E. Zohary,et al.  Topographic Representation of the Human Body in the Occipitotemporal Cortex , 2010, Neuron.

[89]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[90]  J. Kalaska,et al.  Neural mechanisms for interacting with a world full of action choices. , 2010, Annual review of neuroscience.

[91]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[92]  Bradford Z. Mahon,et al.  Action-Related Properties Shape Object Representations in the Ventral Stream , 2007, Neuron.

[93]  D. V. Essen,et al.  Surface-Based and Probabilistic Atlases of Primate Cerebral Cortex , 2007, Neuron.

[94]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[95]  Alex Martin,et al.  Representation of Manipulable Man-Made Objects in the Dorsal Stream , 2000, NeuroImage.

[96]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[97]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[98]  Scott T Grafton,et al.  From 'acting on' to 'acting with': the functional anatomy of object-oriented action schemata. , 2003, Progress in brain research.

[99]  Alison J. Wiggett,et al.  Surface-Based Information Mapping Reveals Crossmodal Vision–Action Representations in Human Parietal and Occipitotemporal Cortex , 2010, Journal of neurophysiology.

[100]  P. Downing,et al.  The neural basis of visual body perception , 2007, Nature Reviews Neuroscience.

[101]  Bradford Z. Mahon,et al.  What drives the organization of object knowledge in the brain? , 2011, Trends in Cognitive Sciences.

[102]  Umberto Castiello,et al.  Human inferior parietal cortex ‘programs’ the action class of grasping , 2000, Cognitive Systems Research.

[103]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[104]  Tom M. Mitchell,et al.  Machine learning classifiers and fMRI: A tutorial overview , 2009, NeuroImage.

[105]  Kenneth F. Valyear,et al.  Decoding Action Intentions from Preparatory Brain Activity in Human Parieto-Frontal Networks , 2011, The Journal of Neuroscience.

[106]  J. Kalaska From intention to action: motor cortex and the control of reaching movements. , 2009, Advances in experimental medicine and biology.

[107]  I. Toni,et al.  Integration of target and effector information in the human brain during reach planning. , 2007, Journal of neurophysiology.

[108]  Ravi S. Menon,et al.  Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas , 2003, Experimental Brain Research.

[109]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[110]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[111]  Paul E. Downing,et al.  Is the extrastriate body area involved in motor actions? , 2005, Nature Neuroscience.

[112]  JamesW. Lewis Cortical Networks Related to Human Use of Tools , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[113]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.