Sound increases the saliency of visual events

We show that concurrent auditory stimuli can enhance the visual system's ability to detect brief visual events. Participants indicated which of two visual stimuli was briefly blinked off. A spatially non-aligned auditory cue - simultaneous with the blink - significantly enhanced subjects' detection ability, while a visual cue decreased detection ability relative to a no-cue condition. Control experiments indicate that the auditory-driven enhancement was not attributable to a warning effect. Also, the enhancement did not depend on an exact temporal alignment of cue-target onsets or offsets. In combination, our results provide evidence that the sound-induced enhancement is not due to a sharpening of visual temporal responses or apparent prolongation of the visual event. Rather, this enhancement seems to reflect an increase in phenomenal visual saliency.

[1]  Francesco Pavani,et al.  Acoustical Vision of Neglected Stimuli: Interaction among Spatially Converging Audiovisual Inputs in Neglect Patients , 2002, Journal of Cognitive Neuroscience.

[2]  Robert B. Welch,et al.  Contributions of audition and vision to temporal rate perception , 1986, Perception & psychophysics.

[3]  G. Recanzone.  Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[4]  D. Burr,et al.  Perceived timing of first- and second-order changes in vision and hearing , 2005, Experimental Brain Research.

[5]  Christopher T. Lovelace,et al.  An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. , 2003, Brain research. Cognitive brain research.

[6]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[7]  B. Stein,et al.  The Merging of the Senses , 1993 .

[8]  J. Vroomen,et al.  Temporal ventriloquism: sound modulates the flash-lag effect. , 2004, Journal of experimental psychology. Human perception and performance.

[9]  Lawrence E Marks,et al.  Cross-modal enhancement of perceived brightness: Sensory interaction versus response bias , 2003, Perception & psychophysics.

[10]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[11]  N. Bolognini,et al.  “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs , 2004, Experimental Brain Research.

[12]  L. Busse,et al.  The spread of attention across modalities and space in a multisensory object. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[13]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[14]  Stephen Lakatos,et al.  Cross-modal interactions in auditory and visual discrimination. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[15]  H Spekreijse,et al.  Auditory—Visual Interactions and the Correspondence between Perceived Auditory Space and Perceived Visual Space , 1977, Perception.

[16]  S. Hillyard,et al.  Involuntary orienting to sound improves visual perception , 2000, Nature.

[17]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[18]  R. Fendrich,et al.  Closer in time when farther in space--spatial factors in audiovisual temporal integration. , 2005, Brain research. Cognitive brain research.

[19]  B. Stein,et al.  Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis , 1996, Journal of Cognitive Neuroscience.

[20]  S. Nishida,et al.  Time perception: Brain time or event time? , 2001, Current Biology.

[21]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[23]  S. Shimojo,et al.  Sound-aided recovery from and persistence against visual filling-in , 2004, Vision Research.

[24]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[25]  J. Enns,et al.  What’s new in visual masking? , 2000, Trends in Cognitive Sciences.

[26]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[27]  John J. Foxe,et al.  Multisensory Convergence in Early Cortical Processing. , 2004 .

[28]  Gillian Rhodes,et al.  Cross-modal effects on visual and auditory object perception , 1984, Perception & psychophysics.