Temporal calibration between the visual, auditory and tactile senses: A psychophysical approach

Temporal miscalibration between different feedback devices in multimodal virtual environments may decrease observers’ sense of immersion and presence. However, subjective perception of synchrony may vary from physical synchrony. Using a psychophysical paradigm we assessed the perception of synchrony for three different modality pairs (audiovisual, audiotactile, visuotactile). For each pair we found that one of the two modalities has to lead considerably in order for the two to be perceived as synchronous. Further, observers were most sensitive to asynchrony in the visuotactile modality combination. Taking these findings into account we offer general suggestions for the temporal calibration of multimodal virtual environments.

[1]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[2]  P. Gribble,et al.  Temporal constraints on the McGurk effect , 1996, Perception & psychophysics.

[3]  J. Lewald,et al.  Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. , 2003, Brain research. Cognitive brain research.

[4]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[5]  A. King,et al.  Multisensory Integration: Strategies for Synchronization , 2005, Current Biology.

[6]  C. Spence,et al.  Audio-visual simultaneity judgments , 2005, Perception & psychophysics.