Is there a role of visual cortex in spatial hearing?

The integration of auditory and visual spatial information is an important prerequisite for accurate orientation in the environment. However, while visual spatial information is based on retinal coordinates, the auditory system receives information on sound location in relation to the head. Thus, any deviation of the eyes from a central position results in a divergence between the retinal visual and the head‐centred auditory coordinates. It has been suggested that this divergence is compensated for by a neural coordinate transformation, using a signal of eye‐in‐head position. Using functional magnetic resonance imaging, we investigated which cortical areas of the human brain participate in such auditory–visual coordinate transformations. Sounds were produced with different interaural level differences, leading to left, right or central intracranial percepts, while subjects directed their gaze to visual targets presented to the left, to the right or straight ahead. When gaze was to the left or right, we found the primary visual cortex (V1/V2) activated in both hemispheres. The occipital activation did not occur with sound lateralization per se, but was found exclusively in combination with eccentric eye positions. This result suggests a relation of neural processing in the visual cortex and the transformation of auditory spatial coordinates responsible for maintaining the perceptual alignment of audition and vision with changes in gaze direction.

[1]  A. Mills.  Lateralization of High‐Frequency Tones , 1960 .

[2]  FRANK MORRELL,et al.  Visual System's View of Acoustic Space , 1972, Nature.

[3]  C. R. Michael,et al.  Integration of auditory information in the cat's visual cortex. , 1973, Vision research.

[4]  L. Benevento,et al.  Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey , 1977, Experimental Neurology.

[5]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[6]  David L. Sparks,et al.  Auditory receptive fields in primate superior colliculus shift with changes in eye position , 1984, Nature.

[7]  E. Vaadia,et al.  Unit study of monkey frontal cortex: active localization of auditory and of visual stimuli. , 1986, Journal of neurophysiology.

[8]  D L Sparks,et al.  Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. , 1987, Journal of neurophysiology.

[9]  David L. Sparks,et al.  Sensori-motor integration in the primate superior colliculus , 1991 .

[10]  J. Malpeli,et al.  Responses of neurons in primary visual cortex are modulated by eye position. , 1993, Journal of neurophysiology.

[11]  R. Andersen,et al.  Spatially tuned auditory responses in area LIP of macaques performing delayed memory saccades to acoustic targets. , 1996, Journal of neurophysiology.

[12]  R. Andersen,et al.  Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. , 1996, Journal of neurophysiology.

[13]  M. Wallace,et al.  Representation and integration of multiple sensory inputs in primate superior colliculus. , 1996, Journal of neurophysiology.

[14]  Jörg Lewald,et al.  Eye-position effects in directional hearing , 1997, Behavioural Brain Research.

[15]  A. Opstal,et al.  Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus , 1998, Brain Research Bulletin.

[16]  R. Turner,et al.  Event-Related fMRI: Characterizing Differential Responses , 1998, NeuroImage.

[17]  C Tempelmann,et al.  Electrodynamic headphones and woofers for application in magnetic resonance imaging scanners. , 1998, Medical physics.

[18]  W H Ehrenstein,et al.  Auditory-visual spatial integration: a new psychophysical approach using laser pointing to acoustic targets. , 1998, The Journal of the Acoustical Society of America.

[19]  Jörg Lewald,et al.  The effect of gaze eccentricity on perceived sound direction and its relation to visual localization , 1998, Hearing Research.

[20]  Scott T. Grafton,et al.  Involvement of visual cortex in tactile discrimination of orientation , 1999, Nature.

[21]  Leslie G. Ungerleider,et al.  Increased Activity in Human Visual Cortex during Directed Attention in the Absence of Visual Stimulation , 1999, Neuron.

[22]  R. Andersen,et al.  Responses to auditory stimuli in macaque lateral intraparietal area. II. Behavioral modulation. , 1999, Journal of neurophysiology.

[23]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[24]  M. W. Greenlee,et al.  MR-Eyetracker: a new method for eye movement recording in functional magnetic resonance imaging , 1999, Experimental Brain Research.

[25]  S. Celebrini,et al.  Gaze direction controls response gain in primary visual-cortex neurons , 1999, Nature.

[26]  J. Rauschecker,et al.  Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans , 1999, Nature Neuroscience.

[27]  R. Andersen,et al.  Responses to auditory stimuli in macaque lateral intraparietal area. I. Effects of training. , 1999, Journal of neurophysiology.

[28]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[29]  R. Andersen,et al.  Reaches to Sounds Encoded in an Eye-Centered Reference Frame , 2000, Neuron.

[30]  J L Lancaster,et al.  Automated Talairach Atlas labels for functional brain mapping , 2000, Human brain mapping.

[31]  G. Mangun,et al.  The neural mechanisms of top-down attentional control , 2000, Nature Neuroscience.

[32]  T. Sawaguchi,et al.  Parallel visuospatial and audiospatial working memory processes in the monkey dorsolateral prefrontal cortex , 2000, Nature Neuroscience.

[33]  J. Rauschecker,et al.  A Positron Emission Tomographic Study of Auditory Localization in the Congenitally Blind , 2000, The Journal of Neuroscience.

[34]  S. Inati,et al.  Eye Position Influences Auditory Responses in Primate Inferior Colliculus , 2001, Neuron.

[35]  M A Meredith,et al.  The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. , 2001, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[36]  Richard B Buxton,et al.  Putting spatial attention on the map: timing and localization of stimulus selection processes in striate and extrastriate visual areas , 2001, Vision Research.

[37]  Beng Hong. Goh,et al.  Effects of training. , 2001 .

[38]  J. Thiran,et al.  Distinct Pathways Involved in Sound Recognition and Localization: A Human fMRI Study , 2000, NeuroImage.

[39]  J Driver,et al.  Crossmodal Spatial Influences of Touch on Extrastriate Visual Areas Take Current Gaze Direction into Account , 2002, Neuron.

[40]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[41]  S. Hillyard,et al.  Delayed Striate Cortical Activation during Spatial Attention , 2002, Neuron.

[42]  Y. Yen,et al.  Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli , 2002, Journal of Cognitive Neuroscience.

[43]  John M. Allman,et al.  The Effect of Gaze Angle and Fixation Distance on the Responses of Neurons in V1, V2, and V4 , 2002, Neuron.

[44]  J. Groh,et al.  Eye Position Affects Activity in Primary Auditory Cortex of Primates , 2003, Current Biology.

[45]  W. Ehrenstein,et al.  The effect of eye position on auditory lateralization , 1996, Experimental Brain Research.

[46]  John A. Baro,et al.  Effects of eye position on saccadic eye movements and on the neuronal responses to auditory and visual stimuli in cat superior colliculus , 2004, Experimental Brain Research.

[47]  I. Meister,et al.  Involvement of the Superior Temporal Cortex and the Occipital Cortex in Spatial Hearing: Evidence from Repetitive Transcranial Magnetic Stimulation , 2004, Journal of Cognitive Neuroscience.

[48]  D. Kurylo,et al.  Effects of eye position on auditory localization and neural representation of space in superior colliculus of cats , 2004, Experimental Brain Research.