Multisensory Functional Magnetic Resonance Imaging

Since its invention almost two decades ago functional magnetic resonance imaging (fMRI) has become the prime research methodology in human neuroscience. Its capabilities continue to evolve based on combined improvements of scanner hardware, experimental designs, and data analysis tools. Within the rapidly growing field of multisensory research the use of noninvasive neuroimaging techniques in general and fMRI in particular is also of increasing relevance. For several years, discussion in the multisensory fMRI community has mainly focused on principles of and statistical criteria for multisensory integration. The recent availability of more sophisticated experimental designs and increasingly sensitive (multivariate) analysis tools allows multisensory researchers to (noninvasively) differentiate between regional and neuronal convergence and to reveal the connectional basis of human multisensory integration.

[1]  M. Naumer,et al.  Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration , 2008, Brain Research.

[2]  Xiaoping Hu,et al.  Effective connectivity during haptic perception: A study using Granger causality analysis of functional magnetic resonance imaging data , 2008, NeuroImage.

[3]  Steven L. Small,et al.  Abstract Coding of Audiovisual Speech: Beyond Sensory Representation , 2007, Neuron.

[4]  Bruce D. McCandliss,et al.  Left lateralized white matter microstructure accounts for individual differences in reading ability and disability , 2006, Neuropsychologia.

[5]  N. Kriegeskorte,et al.  Revealing representational content with pattern-information fMRI--an introductory guide. , 2009, Social cognitive and affective neuroscience.

[6]  J. Kaiser,et al.  Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas , 2007, The Journal of Neuroscience.

[7]  Karl J. Friston,et al.  Dynamic causal modeling , 2010, Scholarpedia.

[8]  J. Kaiser,et al.  Visuohaptic convergence in a corticocerebellar network , 2010, The European journal of neuroscience.

[9]  M. Greicius,et al.  A cross‐modal system linking primary auditory and visual cortices: Evidence from intrinsic fMRI connectivity analysis , 2008, Human brain mapping.

[10]  Tony Ro,et al.  Neural Substrates of Sound–Touch Synesthesia after a Thalamic Lesion , 2008, The Journal of Neuroscience.

[11]  R. Goebel,et al.  Multisensory functional magnetic resonance imaging: a future perspective , 2009, Experimental Brain Research.

[12]  Alexander Borst,et al.  How does Nature Program Neuron Types? , 2008, Front. Neurosci..

[13]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[14]  Ryan A. Stevenson,et al.  Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition , 2009, NeuroImage.

[15]  Andreas Kleinschmidt,et al.  Interaction of Face and Voice Areas during Speaker Recognition , 2005, Journal of Cognitive Neuroscience.

[16]  Céline Cappe,et al.  The Thalamocortical Projection Systems in Primate: An Anatomical Support for Multisensory and Sensorimotor Interplay , 2009, Cerebral cortex.

[17]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[18]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[19]  Karl J. Friston,et al.  Evaluation of different measures of functional connectivity using a neural mass model , 2004, NeuroImage.

[20]  Rainer Goebel,et al.  Mapping directed influence over the brain using Granger causality and fMRI , 2005, NeuroImage.

[21]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[22]  W. Singer,et al.  Retinotopic effects during spatial audio-visual integration , 2007, Neuropsychologia.

[23]  Jochen Kaiser,et al.  Audiovisual Functional Magnetic Resonance Imaging Adaptation Reveals Multisensory Integration Effects in Object-Related Sensory Cortices , 2010, The Journal of Neuroscience.

[24]  N. Logothetis.  What we can do and what we cannot do with fMRI , 2008, Nature.

[25]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[26]  Rainer Goebel,et al.  Reply to Friston and David: After comments on: The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution , 2011, NeuroImage.

[27]  C. Price,et al.  The role of the posterior superior temporal sulcus in audiovisual processing. , 2008, Cerebral cortex.

[28]  M. Naumer,et al.  Touching sounds: thalamocortical plasticity and the neural basis of multisensory integration. , 2009, Journal of neurophysiology.

[29]  U. Noppeney,et al.  Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. , 2010, Cerebral cortex.

[30]  Karl J. Friston,et al.  Psychophysiological and Modulatory Interactions in Neuroimaging , 1997, NeuroImage.

[31]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[32]  Paul J. Laurienti,et al.  On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies , 2005, Experimental Brain Research.

[33]  Christian Keysers,et al.  Testing Simulation Theory with Cross-Modal Multivariate Classification of fMRI Data , 2008, PloS one.

[34]  S. Iversen,et al.  Detection of Audio-Visual Integration Sites in Humans by Application of Electrophysiological Criteria to the BOLD Effect , 2001, NeuroImage.

[35]  Thomas W James,et al.  Enhanced effectiveness in visuo‐haptic object‐selective brain regions with increasing stimulus salience , 2009, Human brain mapping.

[36]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[37]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[38]  H. Scholte,et al.  Increased structural connectivity in grapheme-color synesthesia , 2007, Nature Neuroscience.

[39]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[40]  B. Stein,et al.  The Merging of the Senses , 1993 .

[41]  T. James,et al.  An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI , 2009, Experimental Brain Research.

[42]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[43]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[44]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[45]  Ravi S. Menon,et al.  Haptic study of three-dimensional objects activates extrastriate visual areas , 2002, Neuropsychologia.

[46]  Michael S Beauchamp,et al.  Distributed Representation of Single Touches in Somatosensory and Visual Cortex , 2009, NeuroImage.

[47]  Sven Haller,et al.  Mapping continuous neuronal activation without an ON–OFF paradigm: initial results of BOLD ceiling fMRI , 2006, The European journal of neuroscience.

[48]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[49]  D. V. van Essen,et al.  Functional and structural mapping of human cerebral cortex: solutions are in the surfaces. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[50]  A. Giraud,et al.  Implicit Multisensory Associations Influence Voice Recognition , 2006, PLoS biology.

[51]  Rainer Goebel,et al.  Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: From single‐subject to cortically aligned group general linear model analysis and self‐organizing group independent component analysis , 2006, Human brain mapping.

[52]  Karl J. Friston,et al.  The effect of prior visual information on recognition of speech and sounds. , 2008, Cerebral cortex.

[53]  Ryan A. Stevenson,et al.  Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects , 2007, Experimental Brain Research.

[54]  Rainer Goebel,et al.  The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution , 2011, NeuroImage.

[55]  Rainer Goebel,et al.  Interaction of speech and script in human auditory cortex: Insights from neuro-imaging and effective connectivity , 2009, Hearing Research.

[56]  Michael S. Beauchamp,et al.  Statistical criteria in fMRI studies of multisensory integration , 2005, Neuroinformatics.

[57]  Emiliano Ricciardi,et al.  Beyond sensory images: Object-based representation in the human ventral pathway. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[58]  K. Grill-Spector,et al.  fMR-adaptation: a tool for studying the functional properties of human cortical neurons. , 2001, Acta psychologica.

[59]  Lars Muckli,et al.  Functional Magnetic Resonance Adaptation in Visual Neuroscience , 2008, Reviews in the neurosciences.