Capture of Auditory Motion by Vision Is Represented by an Activation Shift from Auditory to Visual Motion Cortex

The brain is capable of integrating motion information arising from visual and auditory input. Such integration between sensory modalities can aid one another and helps to stabilize the motion percept. However, if motion information differs between sensory modalities, it can also result in an illusory auditory motion percept. This phenomenon is referred to as the cross-modal dynamic capture (CDC) illusion. We used functional magnetic resonance imaging to investigate whether early visual and auditory motion areas are involved in the generation of this illusion. Among the trials containing conflicting audiovisual motion, we compared the trials in which CDC occurred to those in which it did not and used a region of interest approach to see whether the auditory motion complex (AMC) and the visual motion area hMT/V5+ were affected by this illusion. Our results show that CDC reduces activation in bilateral auditory motion areas while increasing activity in the bilateral hMT/V5+. Interestingly, our data show that the CDC illusion is preceded by an enhanced activation that is most dominantly present in the ventral intraparietal sulcus. Moreover, we assessed the effect of motion coherency, which was found to enhance activation in bilateral hMT/V5+ as well as in an area adjacent to the right AMC. Together, our results show that audiovisual integration occurs in early motion areas. Furthermore, it seems that the cognitive state of subjects before stimulus onset plays an important role in the generation of multisensory illusions.

[1]  M. Torrens.  Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[2]  Leonard K. Kaczmarek,et al.  Protein Kinase Modulation of a Neuronal Cation Channel Requires Protein–Protein Interactions Mediated by an Src homology 3 Domain , 2002, The Journal of Neuroscience.

[3]  G F Meyer,et al.  The integration of auditory and visual motion signals at threshold , 2003, Perception & psychophysics.

[4]  G. Fink,et al.  REVIEW: The functional organization of the intraparietal sulcus in humans and monkeys , 2005, Journal of anatomy.

[5]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[6]  G. Glover.  Deconvolution of Impulse Response in Event-Related BOLD fMRI1 , 1999, NeuroImage.

[7]  Charles Spence,et al.  Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: A signal detection study , 2007, Cognition.

[8]  Bill Gardner,et al.  HRTF Measurements of a KEMAR Dummy-Head Microphone , 1994 .

[9]  Alan Kingstone,et al.  Integrating motion information across sensory modalities: the role of top-down factors. , 2006, Progress in brain research.

[10]  Alan Kingstone,et al.  The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. , 2002, Brain research. Cognitive brain research.

[11]  S. Mohand-Said,et al.  Neurodegenerative and Neuroprotective Effects of Tumor Necrosis Factor (TNF) in Retinal Ischemia: Opposite Roles of TNF Receptor 1 and TNF Receptor 2 , 2002, The Journal of Neuroscience.

[12]  Mark W Greenlee,et al.  Neural correlates of coherent audiovisual motion perception. , 2007, Cerebral cortex.

[13]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[14]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[15]  A. Kleinschmidt,et al.  Cross-Modal Processing in Early Visual and Auditory Cortices depends on Expected Statistical Relationship of Multisensory Information , 2006, The Journal of Neuroscience.

[16]  Gereon R. Fink,et al.  Space Coding in Primate Posterior Parietal Cortex , 2001, NeuroImage.

[17]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[18]  Mikhail A. Semenov,et al.  Climate variability and crop yields in Europe , 1999, Nature.

[19]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[20]  S. Hillyard,et al.  Neural Basis of the Ventriloquist Illusion , 2007, Current Biology.

[21]  Rainer Goebel,et al.  Apparent Motion: Event-Related Functional Magnetic Resonance Imaging of Perceptual Switches and States , 2002, The Journal of Neuroscience.

[22]  Sophie M. Wuerger,et al.  Low-level integration of auditory and visual motion signals requires spatial co-localisation , 2005, Experimental Brain Research.

[23]  Hans-Jochen Heinze,et al.  A movement-sensitive area in auditory cortex , 1999, Nature.

[24]  Alan Kingstone,et al.  Assessing automaticity in the audiovisual integration of motion. , 2005, Acta psychologica.

[25]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[26]  M Zaitsev,et al.  Point spread function mapping with parallel imaging techniques and high acceleration factors: Fast, robust, and flexible method for echo‐planar imaging distortion correction , 2004, Magnetic resonance in medicine.