Spatial pattern of BOLD fMRI activation reveals cross-modal information in auditory cortex.

Recent findings suggest that neural representations in early auditory cortex reflect not only the physical properties of a stimulus, but also high-level, top-down, and even cross-modal information. However, the nature of cross-modal information in auditory cortex remains poorly understood. Here, we used pattern analyses of fMRI data to ask whether early auditory cortex contains information about the visual environment. Our data show that 1) early auditory cortex contained information about a visual stimulus when there was no bottom-up auditory signal, and that 2) no influence of visual stimulation was observed in auditory cortex when visual stimuli did not provide a context relevant to audition. Our findings attest to the capacity of auditory cortex to reflect high-level, top-down, and cross-modal information and indicate that the spatial patterns of activation in auditory cortex reflect contextual/implied auditory information but not visual information per se.

[1]  S. Hillyard,et al.  Electrical Signs of Selective Attention in the Human Brain , 1973, Science.

[2]  R. Näätänen The role of attention in auditory information processing as revealed by event-related potentials and other brain measures of cognitive function , 1990, Behavioral and Brain Sciences.

[3]  S. Hillyard,et al.  Modulation of early auditory processing during selective listening to rapidly presented tones. , 1991, Electroencephalography and clinical neurophysiology.

[4]  F. Bloom,et al.  Modulation of early sensory processing in human auditory cortex during auditory selective attention. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[5]  B R Rosen,et al.  Modulation of auditory and visual cortex by selective attention is modality-dependent. , 1996, Neuroreport.

[6]  E. Bullmore,et al.  Activation of auditory cortex during silent lipreading. , 1997, Science.

[7]  B. Renault,et al.  Functional Anatomy of Human Auditory Attention Studied with PET , 1997, NeuroImage.

[8]  J. Rauschecker,et al.  Attention‐related modulation of activity in primary and secondary auditory cortex , 1997, Neuroreport.

[9]  T. Cizadlo,et al.  Auditory and visual attention assessed with PET , 1997, Human brain mapping.

[10]  J. M. Hupé,et al.  Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons , 1998, Nature.

[11]  M. Tervaniemi,et al.  Selective tuning of the left and right auditory cortices during spatially directed attention. , 1999, Brain research. Cognitive brain research.

[12]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[13]  Lutz Jäncke,et al.  Attention modulates activity in the primary and the secondary auditory cortex: a functional magnetic resonance imaging study in human subjects , 1999, Neuroscience Letters.

[14]  D. Heeger,et al.  Activity in primary visual cortex predicts performance in a visual detection task , 2000, Nature Neuroscience.

[15]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[16]  Seung-Schik Yoo,et al.  Human brain mapping of auditory imagery: event-related functional MRI study , 2001, Neuroreport.

[17]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[18]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[19]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[20]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[21]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[22]  Régine Kolinsky,et al.  Attention-Dependent Changes of Activation and Connectivity in Dichotic Listening , 2002, NeuroImage.

[23]  C. Lammertmann,et al.  Magnetoencephalographic Correlates of Audiotactile Interaction , 2002, NeuroImage.

[24]  Paul Schrater,et al.  Shape perception reduces activity in human primary visual cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Riitta Hari,et al.  Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction , 2003, NeuroImage.

[26]  R. Campbell,et al.  Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech , 2003, Journal of Cognitive Neuroscience.

[27]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.

[28]  J. Schwartz,et al.  Seeing to hear better: evidence for early audio-visual interactions in speech identification , 2004, Cognition.

[29]  R. Zatorre,et al.  Behavioral and neural correlates of perceived and imagined musical timbre , 2004, Neuropsychologia.

[30]  A. Fort,et al.  Bimodal speech: early suppressive visual effects in human auditory cortex , 2004, The European journal of neuroscience.

[31]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[32]  Lynne E. Bernstein,et al.  Auditory speech detection in noise enhanced by lipreading , 2004, Speech Commun..

[33]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[34]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[35]  John J. Foxe,et al.  The case for feedforward multisensory convergence during early cortical processing , 2005, Neuroreport.

[36]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[37]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[38]  Jacob Jolij,et al.  Figure–ground segregation requires two distinct periods of activity in V1: a transcranial magnetic stimulation study , 2005, Neuroreport.

[39]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[40]  H. Scheich,et al.  Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys , 2005, The Journal of Neuroscience.

[41]  Robert J. Zatorre,et al.  Mental Concerts: Musical Imagery and Auditory Cortex , 2005, Neuron.

[42]  Hans-Jochen Heinze,et al.  Scanning silence: Mental imagery of complex sounds , 2005, NeuroImage.

[43]  M. Sams,et al.  Primary auditory cortex activation by visual speech: an fMRI study at 3 T , 2005, Neuroreport.

[44]  Gideon Paul Caplovitz,et al.  Bistable illusory rebound motion: Event-related functional magnetic resonance imaging of perceptual states and switches , 2006, NeuroImage.

[45]  Klaus Scheffler,et al.  Differential patterns of multisensory interactions in core and belt areas of human auditory cortex , 2006, NeuroImage.

[46]  D. Kersten,et al.  The representation of perceived angular size in human primary visual cortex , 2006, Nature Neuroscience.

[47]  N. Logothetis,et al.  Functional Imaging Reveals Numerous Fields in the Monkey Auditory Cortex , 2006, PLoS biology.

[48]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[49]  Jean-Philippe Thiran,et al.  Multisensory interactions within human primary cortices revealed by BOLD dynamics. , 2007, Cerebral cortex.

[50]  D. Kersten,et al.  Responses to Lightness Variations in Early Human Visual Cortex , 2007, Current Biology.

[51]  Christoph Kayser,et al.  Behavioral/systems/cognitive Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2022 .

[52]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[53]  Fang Fang,et al.  Perceptual grouping and inverse fMRI activity patterns in human visual cortex. , 2008, Journal of vision.

[54]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[55]  Scott O Murray,et al.  The effects of spatial attention in early human visual cortex are stimulus independent. , 2008, Journal of vision.

[56]  M. Sahani,et al.  Implicit knowledge of visual uncertainty guides decisions with asymmetric outcomes. , 2008, Journal of vision.

[57]  John T Serences,et al.  Value-Based Modulations in Human Visual Cortex , 2008, Neuron.

[58]  M. Paradiso,et al.  V1 response timing and surface filling-in. , 2008, Journal of neurophysiology.

[59]  O. Bertrand,et al.  Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans , 2008, The Journal of Neuroscience.

[60]  Edgar A. DeYoe,et al.  I know where you are secretly attending! The topography of human visual attention revealed with fMRI , 2009, Vision Research.

[61]  P. Tse,et al.  Microsaccade Rate Varies with Subjective Visibility during Motion-Induced Blindness , 2009, PloS one.

[62]  David Whitney,et al.  Attention Narrows Position Tuning of Population Responses in V1 , 2009, Current Biology.

[63]  Christoph Kayser,et al.  Multisensory interactions in primate auditory cortex: fMRI and electrophysiology , 2009, Hearing Research.

[64]  Stefano Panzeri,et al.  Visual Enhancement of the Information Representation in Auditory Cortex , 2010, Current Biology.

[65]  Po-Jang Hsieh,et al.  “Brain‐reading” of perceived colors reveals a feature mixing mechanism underlying perceptual filling‐in in cortical area V1 , 2010, Human brain mapping.

[66]  P. Tse,et al.  BOLD Signal in Both Ipsilateral and Contralateral Retinotopic Cortex Modulates with Perceptual Fading , 2010, PloS one.

[67]  Hanna Damasio,et al.  Predicting visual stimuli on the basis of activity in auditory cortices , 2010, Nature Neuroscience.

[68]  N. Kanwisher,et al.  Recognition alters the spatial pattern of FMRI activation in early retinotopic cortex. , 2010, Journal of neurophysiology.