When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception

Two identical visual targets moving across each other can be perceived either to bounce off or to stream through each other. A brief sound at the moment the targets coincide biases perception toward bouncing. We found that this bounce-inducing effect was attenuated when other identical sounds (auditory flankers) were presented 300 ms before and after the simultaneous sound. The attenuation occurred only when the simultaneous sound and auditory flankers had similar acoustic characteristics and the simultaneous sound was not salient. These results suggest that there is an aspect of auditory-grouping (saliency-assigning) processes that is context-sensitive and can be utilized by the visual system for solving ambiguity. Further-more, control experiments revealed that such auditory context did not affect the perceptual qualities of the simultaneous sound. Because the attenuation effect is not manifest in the perception of acoustic characteristics of individual sound elements, we conclude that it is a genuine cross-modal effect.

[1]  Arthur Dainley-Neeles,et al.  The Approach , 1925 .

[2]  W. Metzger.  Beobachtungen über phänomenale Identität , 1934 .

[3]  Jozef J. Zwislocki,et al.  The Decay of Sensation and the Remainder of Adaptation after Short Pure-Tone Impulses on the Ear , 1947 .

[4]  G. A. Miller,et al.  The Trill Threshold , 1950 .

[5]  R. C. Oldfield.  THE PERCEPTION OF CAUSALITY , 1963 .

[6]  D. C. Higgins.  Human Spatial Orientation , 1967, The Yale Journal of Biology and Medicine.

[7]  W. J. Dowling,et al.  Rhythmic Fission and Perceptual Organization , 1968 .

[8]  D W Massaro,et al.  Retroactive interference in short-term recognition memory for pitch. , 1970, Journal of experimental psychology.

[9]  A. Bregman,et al.  Primary auditory stream segregation and perception of order in rapid sequences of tones. , 1971, Journal of experimental psychology.

[10]  D. Massaro.  Backward recognition masking. , 1975, The Journal of the Acoustical Society of America.

[11]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[12]  D. Massaro,et al.  Recognition masking of auditory lateralization and pitch judgments. , 1976, The Journal of the Acoustical Society of America.

[13]  J R Pomerantz,et al.  Models of illusory pausing and sticking. , 1982, Journal of experimental psychology. Human perception and performance.

[14]  S. S. Stevens,et al.  Hearing, Its Psychology And Physiology , 1983 .

[15]  H. Barlow.  Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[16]  Stephen McAdams,et al.  The Auditory Image: A Metaphor for Musical and Psychological Research on Auditory Organization , 1984 .

[17]  Yoshitaka Nakajima,et al.  Auditory Scene Analysis: The Perceptual Organization of Sound Albert S. Bregman , 1992 .

[18]  K Nakayama,et al.  Experiencing and perceiving visual surfaces. , 1992, Science.

[19]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[20]  B I Bertenthal,et al.  Directional Bias in the Perception of Translating Patterns , 1993, Perception.

[21]  William W. Gaver.  How Do We Hear in the World?: Explorations in Ecological Acoustics , 1993 .

[22]  William T. Freeman,et al.  The generic viewpoint assumption in a framework for visual perception , 1994, Nature.

[23]  S. Sumi.  Bounce effect seen in opposite motions , 1995 .

[24]  M. Tervaniemi,et al.  Preattentive periodicity detection in auditory patterns as governed by time and intensity information. , 1996, Brain research. Cognitive brain research.

[25]  David C. Knill,et al.  Introduction: a Bayesian formulation of visual perception , 1996 .

[26]  Alan L. Yuille,et al.  Perception as Bayesian Inference: Introduction , 1996 .

[27]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[28]  S Shimojo,et al.  Attentional Modulation in Perception of Visual Motion Events , 1998, Perception.

[29]  Michael J Brammer,et al.  Crossmodal identification , 1998, Trends in Cognitive Sciences.

[30]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[31]  R. Sekuler,et al.  Collisions between Moving Visual Targets: What Controls Alternative Ways of Seeing an Ambiguous Display? , 1999, Perception.

[32]  S Shimojo,et al.  Postcoincidence trajectory duration affects motion event perception , 2001, Perception & psychophysics.

[33]  Time for analysis , 2002, Nature.