Multivariate pattern analysis reveals common neural patterns across individuals during touch observation

In a recent study we found that multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data could predict which of several touch-implying video clips a subject saw, only using voxels from primary somatosensory cortex. Here, we re-analyzed the same dataset using cross-individual MVPA to locate patterns of information that were common across participants' brains. In this procedure a classifier learned to distinguish the neural patterns evoked by each stimulus based on the data from a sub-group of the subjects and was then tested on data from an individual that was not part of that sub-group. We found prediction performance to be significantly above chance both when using voxels from the whole brain and when only using voxels from the postcentral gyrus. SVM voxel weight maps established based on the whole-brain analysis as well as a separate searchlight analysis suggested foci of especially high information content in medial and lateral occipital cortex and around the intraparietal sulcus. Classification across individuals appeared to rely on similar brain areas as classification within individuals. These data show that observing touch leads to stimulus-specific patterns of activity in sensorimotor networks and that these patterns are similar across individuals. More generally, the results suggest that cross-individual MVPA can succeed even when applied to restricted regions of interest.

[1]  K Amunts,et al.  Left and right superior parietal lobule in tactile object discrimination , 2004, The European journal of neuroscience.

[2]  G. Fink,et al.  REVIEW: The functional organization of the intraparietal sulcus in humans and monkeys , 2005, Journal of anatomy.

[3]  Alice J. O'Toole,et al.  Theoretical, Statistical, and Practical Perspectives on Pattern-based Classification Approaches to the Analysis of Functional Neuroimaging Data , 2007, Journal of Cognitive Neuroscience.

[4]  L. Fogassi,et al.  A Touching Sight SII/PV Activation during the Observation and Experience of Touch , 2004, Neuron.

[5]  N. Kanwisher,et al.  Cortical Regions Involved in Perceiving Object Shape , 2000, The Journal of Neuroscience.

[6]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[7]  Vittorio Gallese,et al.  The Sense of Touch: Embodied Simulation in a Visuotactile Mirroring Mechanism for Observed Animate or Inanimate Touch , 2008, Journal of Cognitive Neuroscience.

[8]  Stephen José Hanson,et al.  Decoding the Large-Scale Structure of Brain Function by Classifying Mental States Across Individuals , 2009, Psychological science.

[9]  Seung-Schik Yoo,et al.  Human brain mapping of auditory imagery: event-related functional MRI study , 2001, Neuroreport.

[10]  Christian Keysers,et al.  The impact of certain methodological choices on multivariate analysis of fMRI data with support vector machines , 2011, NeuroImage.

[11]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[12]  Alison J. Wiggett,et al.  Crossmodal action representations 1 2 Surface-based information mapping reveals crossmodal vision-action 3 representations in human parietal and occipitotemporal cortex 4 5 6 , 2010 .

[13]  S-J Blakemore,et al.  Somatosensory activations during the observation of touch and a case of vision-touch synaesthesia. , 2005, Brain : a journal of neurology.

[14]  Dinggang Shen,et al.  Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection , 2005, NeuroImage.

[15]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[16]  B. Wandell,et al.  Visual Field Maps in Human Cortex , 2007, Neuron.

[17]  K. Zilles,et al.  Crossmodal Processing of Object Features in Human Anterior Intraparietal Cortex An fMRI Study Implies Equivalencies between Humans and Monkeys , 2002, Neuron.

[18]  Scott A. Huettel,et al.  Within- and cross-participant classifiers reveal different neural coding of information , 2011, NeuroImage.

[19]  Stefan Pollmann,et al.  PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data , 2009, Neuroinformatics.

[20]  Xiaoping Hu,et al.  Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception , 2007, Neuropsychologia.

[21]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[22]  Thomas W. James,et al.  Dorsal and Ventral Cortical Pathways for Visuo-haptic Shape Integration Revealed Using fMRI , 2010 .

[23]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[24]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[25]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[26]  Tom M. Mitchell,et al.  Machine learning classifiers and fMRI: A tutorial overview , 2009, NeuroImage.

[27]  H. C. Dijkerman,et al.  Somatosensory processes subserving perception and action , 2007, Behavioral and Brain Sciences.

[28]  S. Kosslyn,et al.  The role of area 17 in visual imagery: convergent evidence from PET and rTMS. , 1999, Science.

[29]  Tom M. Mitchell,et al.  Commonality of neural representations of words and pictures , 2011, NeuroImage.

[30]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[31]  T. James,et al.  The neural basis of haptic object processing. , 2007, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[32]  Irving Biederman,et al.  Adaptation to objects in the lateral occipital complex (LOC): Shape or semantics? , 2009, Vision Research.

[33]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[34]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[35]  Raymond J. Dolan,et al.  fMRI Activity Patterns in Human LOC Carry Information about Object Exemplars within Category , 2008, Journal of Cognitive Neuroscience.

[36]  Kaspar Meyer,et al.  Behind the looking-glass , 2008, Nature.

[37]  Christophe Habas,et al.  The neural network involved in a bimanual tactile–tactile matching discrimination task: a functional imaging study at 3 T , 2007, Neuroradiology.

[38]  C. Neil Macrae,et al.  Puddles, Parties, and Professors: Linking Word Categorization to Neural Patterns of Visuospatial Coding , 2011, Journal of Cognitive Neuroscience.

[39]  Seung-Schik Yoo,et al.  Neural substrates of tactile imagery: a functional MRI study , 2003, Neuroreport.

[40]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[41]  R. J. Seitz,et al.  A parieto-premotor network for object manipulation: evidence from neuroimaging , 1999, Experimental Brain Research.

[42]  A. Damasio,et al.  Convergence and divergence in a neural architecture for recognition and memory , 2009, Trends in Neurosciences.

[43]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[44]  N. Valenza,et al.  Dissociated active and passive tactile shape recognition: a case study of pure tactile apraxia. , 2001, Brain : a journal of neurology.

[45]  Michael Schaefer,et al.  Effects of different viewing perspectives on somatosensory activations during observation of touch , 2009, Human brain mapping.

[46]  Chris Rorden,et al.  Improving Lesion-Symptom Mapping , 2007, Journal of Cognitive Neuroscience.

[47]  Janaina Mourão Miranda,et al.  Classifying brain states and determining the discriminating activation patterns: Support Vector Machine on functional MRI data , 2005, NeuroImage.

[48]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[49]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[50]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[51]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[52]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[53]  T. Hendler,et al.  Visuo-haptic object-related activation in the ventral visual pathway , 2001, Nature Neuroscience.

[54]  Shy Shoham,et al.  Neural substrates of tactile object recognition: An fMRI study , 2004, Human brain mapping.

[55]  Hanna Damasio,et al.  Predicting visual stimuli on the basis of activity in auditory cortices , 2010, Nature Neuroscience.

[56]  E. Deibert,et al.  Neural pathways in tactile object recognition , 1999, Neurology.

[57]  David J. M. Kraemer,et al.  Musical imagery: Sound of silence activates auditory cortex , 2005, Nature.

[58]  Justin L. Gardner,et al.  Executed and Observed Movements Have Different Distributed Representations in Human aIPS , 2008, The Journal of Neuroscience.

[59]  Tom Michael Mitchell,et al.  From the SelectedWorks of Marcel Adam Just 2008 Using fMRI brain activation to identify cognitive states associated with perception of tools and dwellings , 2016 .

[60]  C. Richards,et al.  Brain activity during visual versus kinesthetic imagery: An fMRI study , 2009, Human brain mapping.

[61]  Niels Birbaumer,et al.  Effective functional mapping of fMRI data with support‐vector machines , 2010, Human brain mapping.

[62]  Hanna Damasio,et al.  Seeing touch is correlated with content-specific activity in primary somatosensory cortex. , 2011, Cerebral cortex.

[63]  S. Kosslyn,et al.  Topographical representations of mental images in primary visual cortex , 1995, Nature.

[64]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[65]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.