Perceptual synchrony of audiovisual streams for natural and artificial motion sequences.

We investigated the conditions necessary for perceptual simultaneity of visual and auditory stimuli under natural conditions: video sequences of conga drumming at various rhythms. Under most conditions, the auditory stream needs to be delayed for sight and sound to be perceived simultaneously. The size of delay for maximum perceived simultaneity varied inversely with drumming tempo, from about 100 ms at 1 Hz to 30 ms at 4 Hz. Random drumming motion produced similar results, with higher random tempos requiring less delay. Video sequences of disk stimuli moving along a motion profile matched to the drummer produced near-identical results. When the disks oscillated at constant speed rather than following "biological" speed variations, the delays necessary for perceptual synchrony were systematically less. The results are discussed in terms of real-world constraints for perceptual synchrony and possible neural mechanisms.

[1]  G. Recanzone.  Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[2]  Waka Fujisaki,et al.  Temporal frequency characteristics of synchrony–asynchrony discrimination of audio-visual signals , 2005, Experimental Brain Research.

[3]  T SHIPLEY,et al.  Auditory Flutter-Driving of Visual Flicker , 1964, Science.

[4]  Derek H. Arnold,et al.  A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation , 2003, Vision Research.

[5]  P. Viviani,et al.  Colour, form, and movement are not perceived simultaneously , 2001, Vision Research.

[6]  F. K. Berrien,et al.  Errors in perceiving the temporal order of auditory and visual stimuli. , 1942 .

[7]  P. Lennie.  The physiological basis of variations in visual latency , 1981, Vision Research.

[8]  Derek H. Arnold,et al.  Asynchronous processing in vision Color leads motion , 2001, Current Biology.

[9]  Q. Summerfield,et al.  Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. , 1985, The Journal of the Acoustical Society of America.

[10]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[11]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[12]  J. Movshon,et al.  Adaptive Temporal Integration of Motion in Direction-Selective Neurons in Macaque Visual Cortex , 2004, The Journal of Neuroscience.

[13]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  S. Zeki.  A vision of the brain , 1993 .

[15]  S. Zeki,et al.  A direct demonstration of perceptual asynchrony in vision , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[16]  T. Albright.  Direction and orientation selectivity of neurons in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[17]  DH Hubel,et al.  Psychophysical evidence for separate channels for the perception of form, color, movement, and depth , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  P. Viviani,et al.  Perceptual asynchronies for biological and non-biological visual events , 2004, Vision Research.

[19]  R. W. Rodieck.  The First Steps in Seeing , 1998 .

[20]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[21]  I. Hirsh,et al.  Perceived order in different sense modalities. , 1961, Journal of experimental psychology.

[22]  P. Jaśkowski,et al.  Simple Reaction Time and Perception of Temporal Order: Dissociations and Hypotheses , 1996, Perceptual and motor skills.

[23]  D. Baylor,et al.  Spectral sensitivity of human cone photoreceptors , 1987, Nature.

[24]  D. Burr,et al.  Vision: Modular analysis – or not? , 1999, Current Biology.

[25]  D. Lewkowicz.  Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[26]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[27]  G Johansson,et al.  Spatio-temporal differentiation and integration in visual motion perception , 1976, Psychological research.

[28]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[29]  J. Rutschmann,et al.  Perception of Temporal Order of Stimuli Differing in Sense Mode and Simple Reaction Time , 1964, Perceptual and motor skills.

[30]  S. Nishida,et al.  Marker Correspondence, Not Processing Latency, Determines Temporal Binding of Visual Attributes , 2002, Current Biology.

[31]  G. Johansson.  Studies on Visual Perception of Locomotion , 1977, Perception.

[32]  H. Wässle,et al.  Response latency of brisk‐sustained (X) and brisk‐transient (Y) cells in the cat retina , 1982, The Journal of physiology.

[33]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[34]  A. Hudspeth,et al.  Response latency of vertebrate hair cells. , 1979, Biophysical journal.

[35]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[36]  P. Lennie.  Single Units and Visual Cortical Organization , 1998, Perception.

[37]  E N Pugh,et al.  A quantitative account of the activation steps involved in phototransduction in amphibian photoreceptors. , 1992, The Journal of physiology.

[38]  Alice J. Hamlin,et al.  On the Least Observable Interval between Stimuli Addressed to Disparate Senses and to Different Organs of the Same Sense , 1895 .