Adaptation to audiovisual asynchrony modulates the speeded detection of sound

The brain adapts to asynchronous audiovisual signals by reducing the subjective temporal lag between them. However, it is currently unclear which sensory signal (visual or auditory) shifts toward the other. According to the idea that the auditory system codes temporal information more precisely than the visual system, one should expect to find some temporal shift of vision toward audition (as in the temporal ventriloquism effect) as a result of adaptation to asynchronous audiovisual signals. Given that visual information gives a more exact estimate of the time of occurrence of distal events than auditory information (due to the fact that the time of arrival of visual information regarding an external event is always closer to the time at which this event occurred), the opposite result could also be expected. Here, we demonstrate that participants' speeded reaction times (RTs) to auditory (but, critically, not visual) stimuli are altered following adaptation to asynchronous audiovisual stimuli. After receiving “baseline” exposure to synchrony, participants were exposed either to auditory-lagging asynchrony (VA group) or to auditory-leading asynchrony (AV group). The results revealed that RTs to sounds became progressively faster (in the VA group) or slower (in the AV group) as participants' exposure to asynchrony increased, thus providing empirical evidence that speeded responses to sounds are influenced by exposure to audiovisual asynchrony.

[1]  R. Ratcliff Group reaction time distributions and an analysis of distribution statistics. , 1979, Psychological bulletin.

[2]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[3]  Q. Summerfield,et al.  Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. , 1985, The Journal of the Acoustical Society of America.

[4]  D. Lewkowicz Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[5]  D. Lewkowicz Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[6]  P. Jaśkowski,et al.  Simple Reaction Time and Perception of Temporal Order: Dissociations and Hypotheses , 1996, Perceptual and motor skills.

[7]  E. Pöppel,et al.  A hierarchical model of temporal perception , 1997, Trends in Cognitive Sciences.

[8]  G. Rizzolatti,et al.  Action for perception: a motor-visual attentional effect. , 1999, Journal of experimental psychology. Human perception and performance.

[9]  G. Rizzolatti,et al.  Action for perception: a motor-visual attentional effect. , 1999 .

[10]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[11]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[12]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[13]  G. Aschersleben,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[14]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[15]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[16]  J. Vroomen,et al.  Temporal ventriloquism: sound modulates the flash-lag effect. , 2004, Journal of experimental psychology. Human perception and performance.

[17]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J. Vroomen,et al.  An event-related potential investigation of the time-course of temporal ventriloquism , 2005, Neuroreport.

[19]  Charles Spence,et al.  Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. , 2005, Brain research. Cognitive brain research.

[20]  P. Montague,et al.  Motor-Sensory Recalibration Leads to an Illusory Reversal of Action and Sensation , 2006, Neuron.

[21]  David Whitaker,et al.  Adaptation minimizes distance-related audiovisual delays. , 2007, Journal of vision.

[22]  Charles Spence,et al.  Temporal recalibration during asynchronous audiovisual speech perception , 2007, Experimental Brain Research.

[23]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[24]  Jean Vroomen,et al.  Neural Correlates of Multisensory Integration of Ecologically Valid Audiovisual Events , 2007, Journal of Cognitive Neuroscience.

[25]  Charles Spence,et al.  Adaptation to audiotactile asynchrony , 2007, Neuroscience Letters.

[26]  L. Harris,et al.  The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity , 2008, Experimental Brain Research.

[27]  Patrick Haggard,et al.  Agency, subjective time, and other minds. , 2007, Journal of experimental psychology. Human perception and performance.

[28]  A. Gorea,et al.  Temporal order judgment and simple reaction times: evidence for a common processing system. , 2007, Journal of vision.

[29]  C. Spence,et al.  Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments , 2008, Experimental Brain Research.

[30]  J. Todd,et al.  The effects of viewing angle, camera angle, and sign of surface curvature on the perception of three-dimensional shape from texture. , 2007, Journal of vision.

[31]  D. Whitaker,et al.  Recalibration of perceived time across sensory modalities , 2008, Experimental Brain Research.

[32]  Laurence R. Harris,et al.  The relative timing of active and passive touch , 2008, Brain Research.

[33]  Jean Vroomen,et al.  Sounds change four-dot masking. , 2009, Acta psychologica.