Subcortical, modality-specific pathways contribute to multisensory processing in humans.

Oftentimes, we perceive our environment by integrating information across multiple senses. Recent studies suggest that such integration occurs at much earlier processing stages than once thought possible, including in thalamic nuclei and putatively unisensory cortical brain regions. Here, we used diffusion tensor imaging (DTI) and an audiovisual integration task to test the hypothesis that anatomical connections between sensory-related subcortical structures and sensory cortical areas govern multisensory processing in humans. Twenty-five subjects (mean age 22 years, 22 females) participated in the study. In line with our hypothesis, we show that estimated strength of white-matter connections between the first relay station in the auditory processing stream (the cochlear nucleus), the auditory thalamus, and primary auditory cortex predicted one's ability to combine auditory and visual information in a visual search task. This finding supports a growing body of work that indicates that subcortical sensory pathways do not only feed forward unisensory information to the cortex, and suggests that anatomical brain connectivity contributes to multisensory processing ability in humans.

[1]  K. Fujii,et al.  Visualization for the analysis of fluid motion , 2005, J. Vis..

[2]  Michael X. Cohen,et al.  Hippocampal-Prefrontal Connectivity Predicts Midfrontal Oscillations and Long-Term Memory Performance , 2011, Current Biology.

[3]  Timothy Edward John Behrens,et al.  Reliable identification of the auditory thalamus using multi-modal structural analyses , 2006, NeuroImage.

[4]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[5]  L. Benevento,et al.  The ascending projections of the superior colliculus in the rhesus monkey (Macaca mulatta) , 1975, The Journal of comparative neurology.

[6]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[7]  Rainer Goebel,et al.  fMR-adaptation indicates selectivity to audiovisual content congruency in distributed clusters in human superior temporal cortex , 2010, BMC Neuroscience.

[8]  D. Tomasi,et al.  Sex differences in sensory gating of the thalamus during auditory interference of visual attention tasks , 2008, Neuroscience.

[9]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[10]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[11]  J. Driver,et al.  Sound-Induced Enhancement of Low-Intensity Vision: Multisensory Influences on Human Sensory-Specific Cortices and Thalamic Bodies Relate to Perceptual Enhancement of Visual Detection Sensitivity , 2010, The Journal of Neuroscience.

[12]  Mark W. Woolrich,et al.  Probabilistic diffusion tractography with multiple fibre orientations: What can we gain? , 2007, NeuroImage.

[13]  E. M. Rouiller,et al.  Multisensory anatomical pathways , 2009, Hearing Research.

[14]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[15]  Christoph Kayser,et al.  Do early sensory cortices integrate cross-modal information? , 2007, Brain Structure and Function.

[16]  E. Macaluso.  Multisensory Processing in Sensory-Specific Cortical Areas , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[17]  S. Raghavachari,et al.  A game of pool, a game of tag: AMPA trafficking in the post-synaptic density , 2010, BMC Neuroscience.

[18]  Heidi Johansen-Berg,et al.  Using diffusion imaging to study human connectional anatomy. , 2009, Annual review of neuroscience.

[19]  K. R. Ridderinkhof,et al.  Corticostriatal Connectivity Underlies Individual Differences in the Balance between Habitual and Goal-Directed Action Control , 2012, The Journal of Neuroscience.

[20]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[21]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[22]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[23]  G. W. Hoesen,et al.  Retinal innervation of the inferior colliculus in rat and monkey , 1982, Brain Research.

[24]  Jan Theeuwes,et al.  Early multisensory interactions affect the competition among multiple visual objects , 2011, NeuroImage.

[25]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[26]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[27]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[28]  R. Linke.  Differential projection patterns of superior and inferior collicular neurons onto posterior paralaminar nuclei of the thalamus surrounding the medial geniculate body in the rat , 1999, The European journal of neuroscience.

[29]  Bernd Weber,et al.  Amygdala tractography predicts functional connectivity and learning during feedback-guided decision-making , 2008, NeuroImage.

[30]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[31]  E. Van der Burg,et al.  Audiovisual events capture attention: evidence from temporal order judgments. , 2008, Journal of vision.

[32]  Mikko Sams,et al.  Seeing speech affects acoustic information processing in the human brainstem , 2005, Experimental Brain Research.

[33]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[34]  Michael X. Cohen,et al.  Connectivity-based segregation of the human striatum predicts personality characteristics , 2009, Nature Neuroscience.

[35]  T. Ono,et al.  Auditory thalamus integrates visual inputs into behavioral gains , 2005, Nature Neuroscience.

[36]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[37]  J. Pernier,et al.  Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. , 2002, Cerebral cortex.

[38]  Manuel S Malmierca,et al.  Direct Projections from Cochlear Nuclear Complex to Auditory Thalamus in the Rat , 2002, The Journal of Neuroscience.

[39]  S. Debener,et al.  Cross-Modal Phase Reset Predicts Auditory Task Performance in Humans , 2011, The Journal of Neuroscience.

[40]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[41]  M. Malmierca,et al.  Evidence for a direct, short latency projection from the dorsal cochlear nucleus to the auditory thalamus in the guinea pig , 2006, The European journal of neuroscience.

[42]  S. Mori,et al.  Principles of Diffusion Tensor Imaging and Its Applications to Basic Neuroscience Research , 2006, Neuron.

[43]  René J. Huster,et al.  Auditory Event-Related Response in Visual Cortex Modulates Subsequent Visual Responses in Humans , 2011, The Journal of Neuroscience.

[44]  Jean-Philippe Thiran,et al.  Multisensory interactions within human primary cortices revealed by BOLD dynamics. , 2007, Cerebral cortex.

[45]  Emiliano Macaluso,et al.  Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites , 2009, The European journal of neuroscience.

[46]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[47]  L. Busse,et al.  The spread of attention across modalities and space in a multisensory object. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[48]  G Mann,et al.  ON THE THALAMUS * , 1905, British medical journal.

[49]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[50]  A. Kleinschmidt,et al.  Cross-Modal Processing in Early Visual and Auditory Cortices depends on Expected Statistical Relationship of Multisensory Information , 2006, The Journal of Neuroscience.

[51]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[52]  N. Suga,et al.  Criticisms of 'Specific long-term memory traces in primary auditory cortex' , 2004, Nature Reviews Neuroscience.

[53]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[54]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[55]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[56]  R. Goebel,et al.  Histological validation of DW-MRI tractography in human postmortem tissue. , 2013, Cerebral cortex.