Multisensory interactions within human primary cortices revealed by BOLD dynamics.

Whether signals from different sensory modalities converge and interact within primary cortices in humans is unresolved, despite emerging evidence in animals. This is partially because of debates concerning the appropriate analyses of functional magnetic resonance imaging (fMRI) data in response to multisensory phenomena. Using event-related fMRI, we observed that simple auditory stimuli (noise bursts) activated primary visual cortices and that simple visual stimuli (checkerboards) activated primary auditory cortices, indicative of multisensory convergence. Moreover, analyses of blood oxygen level-dependent response dynamics revealed facilitation of hemodynamic response peak latencies and slopes for multisensory auditory-visual stimuli versus either unisensory condition, indicative of multisensory interactions within primary sensory cortices. Neural processing at the lowest cortical levels can be modulated by interactions between the senses. Temporal information in fMRI data can reveal these modulations and overcome analytic and interpretational challenges of more traditional procedures. In addition to providing an essential translational link with animal models, these results suggest that longstanding notions of cortical organization need to be revised to include multisensory interactions as an inherent component of functional brain organization.

[1]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[2]  M. Wallace,et al.  A revised view of sensory cortical parcellation , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[3]  P. Barone,et al.  Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey , 2005, The European journal of neuroscience.

[4]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[5]  M. Honda,et al.  Behavioral / Systems / Cognitive Functionally Segregated Neural Substrates for Arbitrary Audiovisual Paired-Association Learning , 2005 .

[6]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  H. Scheich,et al.  Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys , 2005, The Journal of Neuroscience.

[8]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.

[9]  R. Turner,et al.  Detecting Latency Differences in Event-Related BOLD Responses: Application to Words versus Nonwords and Initial versus Repeated Face Presentations , 2002, NeuroImage.

[10]  T. Hackett,et al.  Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[11]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[12]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[13]  Christoph M. Michel,et al.  Visuo-motor pathways in humans revealed by event-related fMRI , 2006, Experimental Brain Research.

[14]  D. Raab DIVISION OF PSYCHOLOGY: STATISTICAL FACILITATION OF SIMPLE REACTION TIMES* , 1962 .

[15]  John J. Foxe,et al.  Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. , 2002, Journal of neurophysiology.

[16]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[17]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[18]  Henry Kennedy,et al.  Long-distance feedback projections to area V1: Implications for multisensory integration, spatial awareness, and visual consciousness , 2004, Cognitive, affective & behavioral neuroscience.

[19]  Nikos K Logothetis,et al.  Laminar specificity in monkey V1 using high-resolution SE-fMRI. , 2006, Magnetic resonance imaging.

[20]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[21]  D. Raab Statistical facilitation of simple reaction times. , 1962, Transactions of the New York Academy of Sciences.

[22]  Micah M. Murray,et al.  How single-trial electrical neuroimaging contributes to multisensory research , 2005, Experimental Brain Research.

[23]  R. Goebel,et al.  Tracking the Mind's Image in the Brain I Time-Resolved fMRI during Visuospatial Mental Imagery , 2002, Neuron.

[24]  Paul J. Laurienti,et al.  On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies , 2005, Experimental Brain Research.

[25]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[26]  N. Logothetis The Underpinnings of the BOLD Functional Magnetic Resonance Imaging Signal , 2003, The Journal of Neuroscience.

[27]  Karl J. Friston,et al.  Event‐related f MRI , 1997, Human brain mapping.

[28]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[29]  C. Schroeder,et al.  A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. , 1998, Cerebral cortex.

[30]  P. Bandettini,et al.  Understanding neural system dynamics through task modulation and measurement of functional MRI amplitude, latency, and width , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[31]  P. Morosan,et al.  Probabilistic Mapping and Volume Measurement of Human Primary Auditory Cortex , 2001, NeuroImage.

[32]  Y. Yen,et al.  Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli , 2002, Journal of Cognitive Neuroscience.

[33]  R Kawashima,et al.  Positron-emission tomography studies of cross-modality inhibition in selective attentional tasks: closing the "mind's eye". , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[34]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[35]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[36]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[37]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[38]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[39]  Skyler Dean,et al.  Mind's Eye , 1973 .

[40]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[41]  M. Sams,et al.  Primary auditory cortex activation by visual speech: an fMRI study at 3 T , 2005, Neuroreport.

[42]  K. Amunts,et al.  Brodmann's Areas 17 and 18 Brought into Stereotaxic Space—Where and How Variable? , 2000, NeuroImage.

[43]  Leslie G. Ungerleider,et al.  The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[44]  E. Schröger,et al.  Speeded responses to audiovisual signal changes result from bimodal integration. , 1998, Psychophysiology.