Hearing visual motion in depth

Auditory spatial perception is strongly affected by visual cues. For example, if auditory and visual stimuli are presented synchronously but from different positions, the auditory event is mislocated towards the locus of the visual stimulus—the ventriloquism effect. This ‘visual capture’ also occurs in motion perception in which a static auditory stimulus appears to move with the visual moving object. We investigated how the human perceptual system coordinates complementary inputs from auditory and visual senses. Here we show that an auditory aftereffect occurs from adaptation to visual motion in depth. After a few minutes of viewing a square moving in depth, a steady sound was perceived as changing loudness in the opposite direction. Adaptation to a combination of auditory and visual stimuli changing in a compatible direction increased the aftereffect and the effect of visual adaptation almost disappeared when the directions were opposite. On the other hand, listening to a sound changing in intensity did not affect the visual changing-size aftereffect. The results provide psychophysical evidence that, for processing of motion in depth, the auditory system responds to both auditory changing intensity and visual motion in depth.

[1]  T. Cornsweet,et al.  The staircrase-method in psychophysics. , 1962, The American journal of psychology.

[2]  W R Thurlow,et al.  Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. , 1973, Perceptual and motor skills.

[3]  D. Regan,et al.  Illusory motion in depth: Aftereffect of adaptation to changing size , 1978, Vision Research.

[4]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[5]  S. Anstis,et al.  Auditory adaptation to gradual rise or fall in intensity of a tone , 1982, Perception & psychophysics.

[6]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[7]  S. Mateeff,et al.  Dynamic Visual Capture: Apparent Auditory Motion Induced by a Moving Visual Target , 1985, Perception.

[8]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[9]  D. Grantham.  Motion aftereffects with horizontally moving sound sources in the free field , 1989, Perception & psychophysics.

[10]  N. V. Swindale,et al.  Spectral motion produces an auditory after-effect , 1993, Nature.

[11]  N J Wade,et al.  A Selective History of the Study of Visual Motion Aftereffects , 1994, Perception.

[12]  Andrew T. Smith,et al.  Visual detection of motion , 1994 .

[13]  W H Ehrenstein,et al.  A Cross-Modal Aftereffect: Auditory Displacement following Adaptation to Visual Motion , 1996, Perceptual and motor skills.

[14]  E. Bullmore,et al.  Activation of auditory cortex during silent lipreading. , 1997, Science.

[15]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[16]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[17]  William A. Yost,et al.  Spatial hearing: The psychophysics of human sound localization, revised edition , 1998 .

[18]  A. Reinhardt-Rutland.  Increasing-Loudness Aftereffect following Decreasing-Intensity Adaptation: Spectral Dependence in Interotic and Monotic Testing , 1998, Perception.

[19]  Frans A. J. Verstraten,et al.  The Motion Aftereffect:A Modern Perspective , 1998 .

[20]  Mark Persoff.  UK , 1999, EC Tax Review.

[21]  N. Kitajima,et al.  Dynamic Capture of Sound Motion by Light Stimuli Moving in Three-Dimensional Space , 1999, Perceptual and motor skills.

[22]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[23]  M. Cynader,et al.  The auditory motion aftereffect: its tuning and specificity in the spatial and frequency domains. , 2000, Perception & psychophysics.

[24]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[25]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[26]  M. Cynader,et al.  The auditory motionaftereffect: Its tuning and specificity in the spatial and frequency domains , 2000 .

[27]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[28]  Aftereffect of adaptation to uni-directional frequency change : Evidence for selective processing mechanism , 2001 .