fMRI-based Multivariate Pattern Analyses Reveal Imagery Modality and Imagery Content Specific Representations in Primary Somatosensory, Motor and Auditory Cortices.

Previous studies have shown that the early visual cortex contains content-specific representations of stimuli during visual imagery, and that these representational patterns of imagery content have a perceptual basis. To date, there is little evidence for the presence of a similar organization in the auditory and tactile domains. Using fMRI-based multivariate pattern analyses we showed that primary somatosensory, auditory, motor, and visual cortices are discriminative for imagery of touch versus sound. In the somatosensory, motor and visual cortices the imagery modality discriminative patterns were similar to perception modality discriminative patterns, suggesting that top-down modulations in these regions rely on similar neural representations as bottom-up perceptual processes. Moreover, we found evidence for content-specific representations of the stimuli during auditory imagery in the primary somatosensory and primary motor cortices. Both the imagined emotions and the imagined identities of the auditory stimuli could be successfully classified in these regions.

[1]  S. Kosslyn,et al.  Visual Mental Imagery Activates Topographically Organized Visual Cortex: PET Investigations , 1993, Journal of Cognitive Neuroscience.

[2]  Rainer Goebel,et al.  Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: From single‐subject to cortically aligned group general linear model analysis and self‐organizing group independent component analysis , 2006, Human brain mapping.

[3]  Niels Birbaumer,et al.  Overt and imagined singing of an Italian aria , 2007, NeuroImage.

[4]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[5]  Denis Schluppeck,et al.  Decoding Working Memory of Stimulus Contrast in Early Visual Cortex , 2013, The Journal of Neuroscience.

[6]  O. Blanke,et al.  Multisensory Mechanisms in Temporo-Parietal Cortex Support Self-Location and First-Person Perspective , 2011, Neuron.

[7]  Skye McDonald,et al.  Emotion in voice matters: neural correlates of emotional prosody perception. , 2013, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[8]  Cyriel M. A. Pennartz,et al.  Modality-specific and modality-independent components of the human imagery system , 2010, NeuroImage.

[9]  Amir Amedi,et al.  Negative BOLD in Sensory Cortices During Verbal Memory: A Component in Generating Internal Representations? , 2009, Brain Topography.

[10]  I. Toni,et al.  Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex , 2013, Current Biology.

[11]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[12]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[13]  J B Poline,et al.  Transient Activity in the Human Calcarine Cortex During Visual-Mental Imagery: An Event-Related fMRI Study , 2000, Journal of Cognitive Neuroscience.

[14]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[15]  Mark S. Cohen,et al.  Changes in cortical activity during mental rotation. A mapping study using functional MRI. , 1996, Brain : a journal of neurology.

[16]  Claudio Babiloni,et al.  Working memory of somatosensory stimuli: an fMRI study. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[17]  Peter Andersen,et al.  Quantitative relations between parietal activation and performance in mental rotation , 1996, Neuroreport.

[18]  Seung-Schik Yoo,et al.  Human brain mapping of auditory imagery: event-related functional MRI study , 2001, Neuroreport.

[19]  B. Gelder Towards the neurobiology of emotional body language , 2006, Nature Reviews Neuroscience.

[20]  Giancarlo Valente,et al.  Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals , 2016, NeuroImage.

[21]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[22]  F. Aboitiz,et al.  The evolutionary origin of the language areas in the human brain. A neuroanatomical perspective , 1997, Brain Research Reviews.

[23]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[24]  Rainer Goebel,et al.  Integration of “what” and “where” in frontal cortex during visual imagery of scenes , 2012, NeuroImage.

[25]  J. Baron,et al.  Does motor imagery share neural networks with executed movement: a multivariate fMRI analysis , 2013, Front. Hum. Neurosci..

[26]  Hans-Jochen Heinze,et al.  Scanning silence: Mental imagery of complex sounds , 2005, NeuroImage.

[27]  V. van de Ven,et al.  The brain's voices: comparing nonclinical auditory hallucinations and imagery. , 2011, Cerebral cortex.

[28]  S. Kosslyn,et al.  The role of area 17 in visual imagery: convergent evidence from PET and rTMS. , 1999, Science.

[29]  Giancarlo Valente,et al.  Task-Dependent Decoding of Speaker and Vowel Identity from Auditory Cortical Response Patterns , 2014, The Journal of Neuroscience.

[30]  Leslie G. Ungerleider,et al.  ‘What’ and ‘where’ in the human brain , 1994, Current Opinion in Neurobiology.

[31]  J B Poline,et al.  Partially overlapping neural networks for real and imagined hand movements. , 2000, Cerebral cortex.

[32]  G. Schalk,et al.  Identifying the Attended Speaker Using Electrocorticographic (ECoG) Signals. , 2015, Brain computer interfaces.

[33]  Gottfried Schlaug,et al.  Shared and distinct neural correlates of singing and speaking , 2006, NeuroImage.

[34]  R. Goebel,et al.  Tracking the Mind's Image in the Brain I Time-Resolved fMRI during Visuospatial Mental Imagery , 2002, Neuron.

[35]  R. Goebel,et al.  Matching two imagined clocks: the functional anatomy of spatial analysis in the absence of visual stimulation. , 2000, Cerebral cortex.

[36]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Emiliano Ricciardi,et al.  Beyond sensory images: Object-based representation in the human ventral pathway. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[38]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[39]  Á. Pascual-Leone,et al.  Linking Out-of-Body Experience and Self Processing to Mental Own-Body Imagery at the Temporoparietal Junction , 2005, The Journal of Neuroscience.

[40]  Rupert Lanzenberger,et al.  The suppressive influence of SMA on M1 in motor imagery revealed by fMRI and dynamic causal modeling , 2008, NeuroImage.

[41]  J. Rauschecker,et al.  A PET study of human auditory spatial processing , 1999, Neuroscience Letters.

[42]  P. Morosan,et al.  Human Primary Auditory Cortex: Cytoarchitectonic Subdivisions and Mapping into a Spatial Reference System , 2001, NeuroImage.

[43]  S. Kosslyn,et al.  Brain areas underlying visual mental imagery and visual perception: an fMRI study. , 2004, Brain research. Cognitive brain research.

[44]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[45]  S. Kosslyn,et al.  When is early visual cortex activated during visual mental imagery? , 2003, Psychological bulletin.

[46]  D. Kareken,et al.  Functional brain imaging in apraxia. , 1998, Archives of neurology.

[47]  Dirk Ostwald,et al.  Imaging tactile imagery: Changes in brain connectivity support perceptual grounding of mental images in primary sensory cortices , 2014, NeuroImage.

[48]  S. Bricogne,et al.  Neural Correlates of Topographic Mental Exploration: The Impact of Route versus Survey Perspective Learning , 2000, NeuroImage.

[49]  Julien Doyon,et al.  Functional neuroanatomical networks associated with expertise in motor imagery , 2008, NeuroImage.

[50]  A. Schleicher,et al.  Two different areas within the primary motor cortex of man , 1996, Nature.

[51]  A. Meltzoff,et al.  Empathy examined through the neural mechanisms involved in imagining how I feel versus how you feel pain , 2006, Neuropsychologia.

[52]  Matthew H. Davis,et al.  Hierarchical Organization of Auditory and Motor Representations in Speech Perception: Evidence from Searchlight Similarity Analysis , 2015, Cerebral cortex.

[53]  R. Malach,et al.  Negative BOLD Differentiates Visual Imagery and Perception , 2005, Neuron.

[54]  Béatrice de Gelder,et al.  Tease or threat? Judging social interactions from bodily expressions , 2010, NeuroImage.

[55]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[56]  M. Denis,et al.  Functional Anatomy of Spatial Mental Imagery Generated from Verbal Instructions , 1996, The Journal of Neuroscience.

[57]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[58]  Jaeseung Jeong,et al.  Auditory Imagery Modulates Frequency-specific Areas in the Human Auditory Cortex , 2013, Journal of Cognitive Neuroscience.

[59]  S. Kosslyn,et al.  Functional Anatomy of High-Resolution Visual Mental Imagery , 2000, Journal of Cognitive Neuroscience.

[60]  Anna C. Nobre,et al.  Imagery for shapes activates position-invariant representations in human visual cortex , 2011, NeuroImage.

[61]  Seung-Schik Yoo,et al.  Neural substrates of tactile imagery: a functional MRI study , 2003, Neuroreport.

[62]  M. Jeannerod,et al.  Possible involvement of primary motor cortex in mentally simulated movement: a functional magnetic resonance imaging study. , 1996, Neuroreport.

[63]  Alexander T Sack,et al.  Brain networks underlying mental imagery of auditory and visual information , 2013, The European journal of neuroscience.

[64]  S. Francis,et al.  Within-Digit Functional Parcellation of Brodmann Areas of the Human Primary Somatosensory Cortex Using Functional Magnetic Resonance Imaging at 7 Tesla , 2012, The Journal of Neuroscience.

[65]  Karsten Mueller,et al.  The functional architecture of S1 during touch observation described with 7 T fMRI , 2013, Brain Structure and Function.

[66]  Laurie Anne Pearlman,et al.  Psychological trauma and the adult survivor: Theory, therapy, and transformation. , 2015 .

[67]  R. Zatorre,et al.  Behavioral and neural correlates of perceived and imagined musical timbre , 2004, Neuropsychologia.

[68]  M. Greenlee,et al.  Cortical activation evoked by visual mental imagery as measured by fMRI , 2000, Neuroreport.

[69]  Jack L. Gallant,et al.  A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes , 2015, NeuroImage.

[70]  Karl J. Friston,et al.  Predictive coding, precision and synchrony , 2012, Cognitive neuroscience.

[71]  Richard J. Caselli,et al.  The nature of tactile agnosia: A case study , 1994, Neuropsychologia.

[72]  Rudolf Stark,et al.  Motor imagery of hand actions: Decoding the content of motor imagery from brain activity in frontal and parietal motor areas , 2015, Human brain mapping.

[73]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[74]  K. Amunts,et al.  Brodmann's Areas 17 and 18 Brought into Stereotaxic Space—Where and How Variable? , 2000, NeuroImage.

[75]  K. Zilles,et al.  Human Somatosensory Area 2: Observer-Independent Cytoarchitectonic Mapping, Interindividual Variability, and Population Map , 2001, NeuroImage.

[76]  Lars Muckli,et al.  Decoding Sound and Imagery Content in Early Visual Cortex , 2014, Current Biology.

[77]  M. Jeannerod,et al.  The cognitive neuroscience of mental imagery , 1995, Neuropsychologia.

[78]  Simon B. Eickhoff,et al.  Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps , 2006, NeuroImage.

[79]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[80]  K. Zilles,et al.  Areas 3a, 3b, and 1 of Human Primary Somatosensory Cortex 2. Spatial Normalization to Standard Anatomical Space , 2000, NeuroImage.

[81]  B. Cohen,et al.  Tactile, olfactory, and gustatory hallucinations in psychotic disorders: a descriptive study. , 2009, Annals of the Academy of Medicine, Singapore.

[82]  Guy A. Orban,et al.  The overlap of the EBA and the MT/V5 cluster , 2013, NeuroImage.

[83]  Joseph B. Sala,et al.  Dissociable functional cortical topographies for working memory maintenance of voice identity and location. , 2004, Cerebral cortex.

[84]  K. Amunts,et al.  Human V5/MT+: comparison of functional and cytoarchitectonic data , 2005, Anatomy and Embryology.

[85]  Riitta Hari,et al.  Observing touch activates human primary somatosensory cortex , 2010, The European journal of neuroscience.

[86]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[87]  Xiaoping Hu,et al.  Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception , 2007, Neuropsychologia.

[88]  Peter S. Jones,et al.  Mapping the involvement of BA 4a and 4p during Motor Imagery , 2008, NeuroImage.

[89]  A. Schleicher,et al.  Cytoarchitectonic analysis of the human extrastriate cortex in the region of V5/MT+: a probabilistic, stereotaxic map of area hOc5. , 2006, Cerebral cortex.

[90]  R. Zatorre,et al.  When that tune runs through your head: a PET investigation of auditory imagery for familiar melodies. , 1999, Cerebral cortex.

[91]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[92]  M. Erb,et al.  Dynamic Brain Activation during Processing of Emotional Intonation: Influence of Acoustic Parameters, Emotional Valence, and Sex , 2002, NeuroImage.

[93]  Emily S. Cross,et al.  Disentangling neural processes of egocentric and allocentric mental spatial transformations using whole-body photos of self and other , 2015, NeuroImage.

[94]  R Saxe,et al.  People thinking about thinking people The role of the temporo-parietal junction in “theory of mind” , 2003, NeuroImage.

[95]  C. Urgesi,et al.  Distinct contributions of extrastriate body area and temporoparietal junction in perceiving one’s own and others’ body , 2014, Cognitive, Affective, & Behavioral Neuroscience.

[96]  T. Baumgartner,et al.  Chronometric features of processing unpleasant stimuli: a functional MRI-based transcranial magnetic stimulation study , 2008, Neuroreport.

[97]  P. Good Permutation, Parametric, and Bootstrap Tests of Hypotheses , 2005 .