Auditory grouping occurs prior to intersensory pairing: evidence from temporal ventriloquism

The authors examined how principles of auditory grouping relate to intersensory pairing. Two sounds that normally enhance sensitivity on a visual temporal order judgement task (i.e. temporal ventriloquism) were embedded in a sequence of flanker sounds which either had the same or different frequency (Exp. 1), rhythm (Exp. 2), or location (Exp. 3). In all experiments, we found that temporal ventriloquism only occurred when the two capture sounds differed from the flankers, demonstrating that grouping of the sounds in the auditory stream took priority over intersensory pairing. By combining principles of auditory grouping with intersensory pairing, we demonstrate that capture sounds were, counter-intuitively, more effective when their locations differed from that of the lights rather than when they came from the same position as the lights.

[1]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[2]  Robert B. Welch,et al.  Contributions of audition and vision to temporal rate perception , 1986, Perception & psychophysics.

[3]  M. Radeau Auditory-visual spatial interaction and modularity , 1994, Current psychology of cognition = Cahiers de psychologie cognitive : CPC.

[4]  R. Lynn,et al.  A secular decline in the strength of Spearman’s g in Japan , 1994 .

[5]  Charles Spence,et al.  Exploring the role of visual perceptual grouping on the audiovisual integration of motion , 2004, Neuroreport.

[6]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[7]  P. Bertelson Chapter 14 Ventriloquism: A case of crossmodal perceptual grouping , 1999 .

[8]  R. Welch Chapter 15 Meaning, attention, and the “unity assumption” in the intersensory bias of spatial and temporal perceptions , 1999 .

[9]  Béatrice de Gelder,et al.  Exploring the relation between mcgurk interference and ventriloquism , 1994, ICSLP.

[10]  J. Gaddum Probit Analysis , 1948, Nature.

[11]  Cécile Colin,et al.  Rules of intersensory integration in spatial scene analysis and speechreading , 2001 .

[12]  Christoph M. Michel,et al.  Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging , 2004, NeuroImage.

[13]  Sidney S. Simon,et al.  Merging of the Senses , 2008, Front. Neurosci..

[14]  T. Bachmann,et al.  Cognitive contributions to the perception of spatial and temporal events (Advances in Psychology, Vol. 129) , 1999 .

[15]  S. Shimojo,et al.  When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception , 2001, Psychological science.

[16]  Charles Spence,et al.  Bouncing or streaming? Exploring the influence of auditory cues on the interpretation of ambiguous visual motion , 2004, Experimental Brain Research.

[17]  G. Aschersleben,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[18]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[19]  J. Vroomen,et al.  Temporal ventriloquism: sound modulates the flash-lag effect. , 2004, Journal of experimental psychology. Human perception and performance.

[20]  M. Kubovy,et al.  Auditory and visual objects , 2001, Cognition.

[21]  B. Stein,et al.  Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis , 1996, Journal of Cognitive Neuroscience.

[22]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[23]  Paul Bertelson,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[24]  Jean Vroomen,et al.  The spatial constraint in intersensory pairing: no role in temporal ventriloquism. , 2006, Journal of experimental psychology. Human perception and performance.

[25]  J. Vroomen,et al.  An event-related potential investigation of the time-course of temporal ventriloquism , 2005, Neuroreport.

[26]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[27]  F. Bedford Constraints on learning new mappings between perceptual dimensions , 1989 .

[28]  H. Heffner,et al.  Visual factors in sound localization in mammals , 1992, The Journal of comparative neurology.