Neural correlates of coherent audiovisual motion perception.

Real-life moving objects are often detected by multisensory cues. We investigated the cortical activity associated with coherent visual motion perception in the presence of a stationary or moving auditory noise source using functional magnetic resonance imaging. Twelve subjects judged episodes of 5-s random-dot motion containing either no (0%) or abundant (16%) coherent direction information. Auditory noise was presented with the displayed visual motion that was moving in phase, was moving out-of-phase, or was stationary. Subjects judged whether visual coherent motion was present, and if so, whether the auditory noise source was moving in phase, was moving out-of-phase, or was not moving. Performance was greatest for a moving sound source that was in phase with the visual coherent dot motion compared with when it was in antiphase. A random-effects analysis revealed that auditory motion activated extended regions in both cerebral hemispheres in the superior temporal gyrus (STG), with a right-hemispheric preponderance. Combined audiovisual motion led to activation clusters in the STG, the supramarginal gyrus, the superior parietal lobule, and the cerebellum. The size of the activated regions was substantially larger than that evoked by either visual or auditory motion alone. The congruent audiovisual motion evoked the most extensive activation pattern, exhibiting several exclusively activated subregions.

[1]  O B Paulson,et al.  The activation pattern in normal humans during suppression, imagination and performance of saccadic eye movements. , 1997, Acta physiologica Scandinavica.

[2]  M. Ahissar,et al.  Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. , 1992, Journal of neurophysiology.

[3]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[4]  Richard S. J. Frackowiak,et al.  Right parietal cortex is involved in the perception of sound movement in humans , 1998, Nature Neuroscience.

[5]  Alan Kingstone,et al.  Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities. , 2004, Journal of experimental psychology. Human perception and performance.

[6]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[7]  P. Cavanagh,et al.  Cortical fMRI activation produced by attentive tracking of moving targets. , 1998, Journal of neurophysiology.

[8]  Shinsuke Shimojo,et al.  Sound-induced illusory flash perception: role of gamma band responses , 2002, Neuroreport.

[9]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[10]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[11]  Jack L. Lancaster,et al.  The Talairach Daemon a database server for talairach atlas labels , 1997 .

[12]  Leslie G. Ungerleider,et al.  Object and spatial visual working memory activate separate neural systems in human cortex. , 1996, Cerebral cortex.

[13]  Sophie M. Wuerger,et al.  Low-level integration of auditory and visual motion signals requires spatial co-localisation , 2005, Experimental Brain Research.

[14]  Alan C. Evans,et al.  Event-Related fMRI of the Auditory Cortex , 1998, NeuroImage.

[15]  M. Corbetta,et al.  Top-down modulation of early sensory cortex. , 1997 .

[16]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[17]  E. DeYoe,et al.  Graded effects of spatial and featural attention on human area MT and associated motion processing areas. , 1997, Journal of neurophysiology.

[18]  Karl J. Friston,et al.  The physiological basis of attentional modulation in extrastriate visual areas , 1999, Nature Neuroscience.

[19]  M W von Grünau,et al.  Attentional selection of motion states. , 1998, Spatial vision.

[20]  A. Chaudhuri.  Modulation of the motion aftereffect by selective attention , 1990, Nature.

[21]  E. Macaluso,et al.  A Common Cortical Substrate Activated by Horizontal and Vertical Sound Movement in the Human Brain , 2002, Current Biology.

[22]  M. W. Greenlee,et al.  MR-Eyetracker: a new method for eye movement recording in functional magnetic resonance imaging , 1999, Experimental Brain Research.

[23]  Brigitte Röder,et al.  Attending to visual or auditory motion affects perception within and across modalities: an event‐related potential study , 2005, The European journal of neuroscience.

[24]  G. Rizzolatti,et al.  Parietal cortex: from sight to action , 1997, Current Opinion in Neurobiology.

[25]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[26]  B. Stein,et al.  The Merging of the Senses , 1993 .

[27]  H. Spitzer,et al.  Increased attention enhances both behavioral and neuronal performance. , 1988, Science.

[28]  Brigitte Röder,et al.  A new method for detecting interactions between the senses in event-related potentials , 2006, Brain Research.

[29]  D. Gitelman,et al.  Neuroanatomic Overlap of Working Memory and Spatial Attention Networks: A Functional MRI Comparison within Subjects , 1999, NeuroImage.

[30]  Hans-Jochen Heinze,et al.  A movement-sensitive area in auditory cortex , 1999, Nature.

[31]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[32]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[33]  Alan Kingstone,et al.  The ventriloquist in motion: illusory capture of dynamic information across sensory modalities. , 2002, Brain research. Cognitive brain research.

[34]  J L Lancaster,et al.  Automated Talairach Atlas labels for functional brain mapping , 2000, Human brain mapping.

[35]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[36]  Leslie G. Ungerleider,et al.  The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  G. Rizzolatti,et al.  Spatial attention and eye movements , 2004, Experimental Brain Research.

[38]  D. V. van Essen,et al.  Windows on the brain: the emerging role of atlases and databases in neuroscience , 2002, Current Opinion in Neurobiology.

[39]  J. Rauschecker,et al.  Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans , 1999, Nature Neuroscience.

[40]  P. Goldman-Rakic,et al.  Functional magnetic resonance imaging of human prefrontal cortex activation during a spatial working memory task. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Andrew R Mitz,et al.  Somatotopy of monkey premotor cortex examined with microstimulation , 1995, Neuroscience Research.

[42]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[43]  D. Boussaoud,et al.  Spatial attention and memory versus motor preparation: premotor cortex involvement as revealed by fMRI. , 2002, Journal of neurophysiology.

[44]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[45]  A. Treisman,et al.  Voluntary Attention Modulates fMRI Activity in Human MT–MST , 1997, Neuron.

[46]  Karl J. Friston,et al.  Spatial registration and normalization of images , 1995 .

[47]  J. P. H. Reulen,et al.  Precise recording of eye movement: the IRIS technique Part 1 , 2006, Medical and Biological Engineering and Computing.

[48]  S. Shimojo,et al.  Sound alters visual evoked potentials in humans , 2001, Neuroreport.

[49]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[50]  R. Patterson,et al.  The Processing of Temporal Pitch and Melody Information in Auditory Cortex , 2002, Neuron.

[51]  Karl J. Friston,et al.  The functional anatomy of attention to visual motion. A functional MRI study. , 1998, Brain : a journal of neurology.

[52]  Waka Fujisaki,et al.  Temporal frequency characteristics of synchrony–asynchrony discrimination of audio-visual signals , 2005, Experimental Brain Research.

[53]  R. Bowtell,et al.  “sparse” temporal sampling in auditory fMRI , 1999, Human brain mapping.

[54]  R. Blake,et al.  Neural strength of visual attention gauged by motion adaptation , 1999, Nature Neuroscience.

[55]  R Sekuler,et al.  Mental set alters visibility of moving targets , 1977, Science.

[56]  Vittorio Gallese,et al.  Somatotopic organization of the lateral part of area F2 (dorsal premotor cortex) of the macaque monkey. , 2003, Journal of neurophysiology.

[57]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[58]  B. Röder,et al.  Attention to motion enhances processing of both visual and auditory stimuli: an event-related potential study. , 2004, Brain research. Cognitive brain research.

[59]  Andrew P. Holmes,et al.  CHAPTER 65 – Nonparametric Analysis of Statistic Images from Functional Mapping Experiments , 1996 .

[60]  Edward E. Smith,et al.  Spatial working memory in humans as revealed by PET , 1993, Nature.

[61]  J. A. Frost,et al.  Conceptual Processing during the Conscious Resting State: A Functional MRI Study , 1999, Journal of Cognitive Neuroscience.

[62]  N. Logothetis,et al.  Integration of Touch and Sound in Auditory Cortex , 2005, Neuron.

[63]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[64]  Juhani Hyvärinen,et al.  Distribution of visual and somatic functions in the parietal associative area 7 of the monkey , 1979, Brain Research.

[65]  Paul J. Laurienti,et al.  An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets , 2003, NeuroImage.

[66]  M. Goldberg,et al.  Visual, presaccadic, and cognitive activation of single neurons in monkey lateral intraparietal area. , 1996, Journal of neurophysiology.

[67]  Andrew P. Holmes,et al.  Statistical issues in functional brain mapping. , 1994 .

[68]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[69]  B. Mazoyer,et al.  PET study of the human foveal fixation system , 1999, Human brain mapping.

[70]  L. Benevento,et al.  Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey , 1977, Experimental Neurology.

[71]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[72]  T. Wickens.  Elementary Signal Detection Theory , 2001 .

[73]  Karl J. Friston,et al.  A direct quantitative relationship between the functional properties of human and macaque V5 , 2000, Nature Neuroscience.