Distributed Representation of Single Touches in Somatosensory and Visual Cortex

Multi‐voxel pattern analysis (MVPA) was used to analyze blood–oxygen level dependent functional magnetic resonance imaging (BOLD fMRI) data, which were acquired as human subjects received brief vibrotactile stimulation of their hands and feet. Support vector machines trained and tested on the whole brain fMRI data were able to accurately decode the body site of single touches, with mean performance of 92% in a two‐way discrimination task (chance performance 50%) and 70% in a four‐way discrimination task (chance performance 25%). Primary and secondary somatosensory areas (S1 and S2) alone decoded the touched body site with high accuracy. S1 was more accurate at decoding touches closely spaced on the body surface (different fingers of the same hand) whereas S2 and S1 were equally accurate at decoding widely spaced touches (hand vs. foot). The hand and foot regions of S1 (S1hand and S1foot) were separately examined in a two‐way classification task. S1hand was better able to decode the hand of stimulation (left vs. right), and S1foot was better able to decode the foot of stimulation. In addition to S1 and S2, vibrotactile responses were observed in a region of visual cortex, areas MST and STP (MST/STP) in lateral occipito‐temporal lobe. MST/STP was able to accurately decode the hand but not the foot of stimulation, supporting the idea of a role for MST/STP in eye‐hand coordination. Hum Brain Mapp 2009. © 2009 Wiley‐Liss, Inc.

[1]  B. Argall,et al.  Simplified intersubject averaging on the cortical surface using SUMA , 2006, Human brain mapping.

[2]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[3]  M. Tommerdahl,et al.  Human vibrotactile frequency discriminative capacity after adaptation to 25 Hz or 200 Hz stimulation , 2005, Brain Research.

[4]  H. Alkadhi,et al.  Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. , 1997, Brain : a journal of neurology.

[5]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[6]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[7]  A. Dale,et al.  Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. , 1998, Science.

[8]  Michael S. Beauchamp,et al.  Touch, sound and vision in human superior temporal sulcus , 2008, NeuroImage.

[9]  Nikolaus Kriegeskorte,et al.  Combining the tools: Activation- and information-based fMRI analysis , 2007, NeuroImage.

[10]  Kenith V. Sobel,et al.  PSYCHOLOGICAL SCIENCE Research Article Neural Synergy Between Kinetic Vision and Touch , 2022 .

[11]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[12]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[13]  Tony Ro,et al.  Human MST But Not MT Responds to Tactile Stimulation , 2007, The Journal of Neuroscience.

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  K. Amunts,et al.  The human parietal operculum. II. Stereotaxic maps and correlation with functional imaging results. , 2006, Cerebral cortex.

[16]  G. Rees,et al.  Predicting the Stream of Consciousness from Activity in Human Visual Cortex , 2005, Current Biology.

[17]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[18]  J. Haxby,et al.  The effect of visual experience on the development of functional architecture in hMT+. , 2007, Cerebral cortex.

[19]  Xiaoping P. Hu,et al.  Real‐time fMRI using brain‐state classification , 2007, Human brain mapping.

[20]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[21]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[22]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[23]  M. Sur,et al.  Representations of the body surface in postcentral parietal cortex of Macaca fascicularis , 1980, The Journal of comparative neurology.

[24]  G. Glover.  Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[25]  David Whitney,et al.  Visually guided reaching depends on motion area MT+. , 2007, Cerebral cortex.

[26]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[27]  U. Ilg,et al.  Primate area MST-l is involved in the generation of goal-directed eye and hand movements. , 2007, Journal of neurophysiology.

[28]  G. Essick,et al.  Tactile motion activates the human middle temporal/V5 (MT/V5) complex , 2002, The European journal of neuroscience.

[29]  K. O. Johnson,et al.  Time-course of vibratory adaptation and recovery in cutaneous mechanoreceptive afferents. , 2005, Journal of neurophysiology.

[30]  Gabriel Curio,et al.  Event-related fMRI of the somatosensory system using electrical finger stimulation , 2002, Neuroreport.

[31]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[32]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[33]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[34]  J. Kaas,et al.  Multiple representations of the body within the primary somatosensory cortex of primates. , 1979, Science.

[35]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[36]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[37]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[38]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[39]  Stephen C. Strother,et al.  Support vector machines for temporal classification of block design fMRI data , 2005, NeuroImage.

[40]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.