Depth Cues and Perceived Audiovisual Synchrony of Biological Motion

Background Due to their different propagation times, visual and auditory signals from external events arrive at the human sensory receptors with a disparate delay. This delay consistently varies with distance, but, despite such variability, most events are perceived as synchronic. There is, however, contradictory data and claims regarding the existence of compensatory mechanisms for distance in simultaneity judgments. Principal Findings In this paper we have used familiar audiovisual events – a visual walker and footstep sounds – and manipulated the number of depth cues. In a simultaneity judgment task we presented a large range of stimulus onset asynchronies corresponding to distances of up to 35 meters. We found an effect of distance over the simultaneity estimates, with greater distances requiring larger stimulus onset asynchronies, and vision always leading. This effect was stronger when both visual and auditory cues were present but was interestingly not found when depth cues were impoverished. Significance These findings reveal that there should be an internal mechanism to compensate for audiovisual delays, which critically depends on the depth information available.

[1]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[2]  B. Khurana,et al.  Space and Time in Perception and Action: Space–time and awareness , 2010 .

[3]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[4]  V. Virsu,et al.  Simultaneity learning in vision, audition, tactile sense and their cross-modal combinations , 2008, Experimental Brain Research.

[5]  J. Lewald,et al.  Auditory-visual temporal integration as a function of distance: no compensation for sound-transmission time in human perception , 2004, Neuroscience Letters.

[6]  M. Marcell,et al.  Confrontation Naming of Environmental Sounds , 2000, Journal of clinical and experimental neuropsychology.

[7]  L. Harris,et al.  Simultaneity Constancy , 2004, Perception.

[8]  F. Pollick,et al.  A Psychophysical Investigation of Differences between Synchrony and Temporal Order Judgments , 2013, PloS one.

[9]  P. Lennie The physiological basis of variations in visual latency , 1981, Vision Research.

[10]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[11]  Yôiti Suzuki,et al.  Implicit estimation of sound-arrival time , 2003, Nature.

[12]  David Whitaker,et al.  Adaptation minimizes distance-related audiovisual delays. , 2007, Journal of vision.

[13]  G. R. Engel,et al.  Visual-Auditory Distance Constancy , 1971, Nature.

[14]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[15]  D. Lewkowicz Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[16]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[17]  Jean Vroomen,et al.  The role of spatial disparity and hemifields in audio-visual temporal order judgments , 2005, Experimental Brain Research.

[18]  J. Vroomen,et al.  Perception of intersensory synchrony: A tutorial review , 2010, Attention, perception & psychophysics.

[19]  David Alais,et al.  Perceptual synchrony of audiovisual streams for natural and artificial motion sequences. , 2006, Journal of vision.

[20]  Katsumi Aoki,et al.  Recent development of flow visualization , 2004, J. Vis..

[21]  Simon Carlile,et al.  Synchronizing to real events: subjective audiovisual alignment scales with perceived auditory depth and speed of sound. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Derek H. Arnold,et al.  Timing sight and sound , 2005, Vision Research.

[23]  John H. R. Maunsell,et al.  Visual response latencies in striate cortex of the macaque monkey. , 1992, Journal of neurophysiology.

[24]  James V. Stone,et al.  When is now? Perception of simultaneity , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[25]  Nao Ninomiya,et al.  The 10th anniversary of journal of visualization , 2007, J. Vis..

[26]  Tammo Houtgast,et al.  Auditory distance perception in rooms , 1999, Nature.

[27]  F. Pollick,et al.  Expertise with multisensory events eliminates the effect of biological motion rotation on audiovisual synchrony perception. , 2010, Journal of vision.