Space and Time in Perception and Action: Mechanisms of simultaneity constancy

There is a delay before sensory information arising from a given event reaches the central nervous system. This delay may be different for information carried by different senses. It will also vary depending on how far the event is from the observer and stimulus properties such as intensity. However, it seems that at least some of these processing time differences can be compensated for by a mechanism that resynchronizes asynchronous signals and enables us to perceive simultaneity correctly. This chapter explores how effectively simultaneity constancy can be achieved, both intramodally within the visual and tactile systems and cross-modally between combinations of auditory, visual, and tactile stimuli. We propose and provide support for a three-stage model of simultaneity constancy in which (1) signals within temporal and spatial windows are identified as corresponding to a single event, (2) a crude resynchronization is applied based on simple rules corresponding to the average processing speed differences between the individual sensory systems, and (3) fine-tuning adjustments are applied based on previous experience with particular combinations of stimuli.

[1]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[2]  L. Harris,et al.  Simultaneity constancy: detecting events with touch and vision , 2005, Experimental Brain Research.

[3]  C. Spence,et al.  Confusing the mind by crossing the hands. , 2002, Brain research. Cognitive brain research.

[4]  Ernst Pöppel,et al.  Sensory integration within temporally neutral systems states: A hypothesis , 1990, Naturwissenschaften.

[5]  D. Dennett,et al.  The Nature of Consciousness , 2006 .

[6]  C. Pulfrich.  Die Stereoskopie im Dienste der isochromen und heterochromen Photometrie , 2005, Naturwissenschaften.

[7]  R. Nickalls.  The Influence of Target Angular Velocity on Visual Latency Difference Determined using the Rotating Pulfrich Effect , 1996, Vision Research.

[8]  Paul Bertelson,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[9]  N Lesevre,et al.  CHRONOTOPOGRAPHICAL ANALYSIS OF THE HUMAN EVOKED POTENTIAL IN RELATION TO THE VISUAL FIELD (DATA FROM NORMAL INDIVIDUALS AND HEMIANOPIC PATIENTS) , 1980, Annals of the New York Academy of Sciences.

[10]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[11]  Derek H. Arnold,et al.  Timing sight and sound , 2005, Vision Research.

[12]  Charles Spence,et al.  Adaptation to audiotactile asynchrony , 2007, Neuroscience Letters.

[13]  J. Lewald,et al.  Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. , 2003, Brain research. Cognitive brain research.

[14]  Simon Carlile,et al.  Synchronizing to real events: subjective audiovisual alignment scales with perceived auditory depth and speed of sound. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[15]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[16]  S. Anstis,et al.  Visual delay as a function of luminance. , 1969, The American journal of psychology.

[17]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[18]  G. Celesia.  Organization of auditory cortical areas in man. , 1976, Brain : a journal of neurology.

[19]  J. Craig,et al.  Temporal order and tactile patterns , 1990, Perception & psychophysics.

[20]  C. Spence,et al.  Audiovisual prior entry , 2003, Neuroscience Letters.

[21]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[22]  R. Baddeley,et al.  Multisensory temporal order judgments: When two locations are better than one , 2003, Perception & psychophysics.

[23]  R. Gregory,et al.  Distortion of Visual Space as Inappropriate Constancy Scaling , 1963, Nature.

[24]  Y. Sugita,et al.  Audiovisual perception: Implicit estimation of sound-arrival time , 2003, Nature.

[25]  L. Harris,et al.  The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity , 2008, Experimental Brain Research.

[26]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[27]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[28]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[29]  C. Spence,et al.  Audiovisual synchrony perception for speech and music assessed using a temporal order judgment task , 2006, Neuroscience Letters.

[30]  R. Verleger,et al.  Attentional Bias toward Low-Intensity Stimuli: An Explanation for the Intensity Dissociation between Reaction Time and Temporal Order Judgment? , 2000, Consciousness and Cognition.

[31]  J. Lewald,et al.  Auditory-visual temporal integration as a function of distance: no compensation for sound-transmission time in human perception , 2004, Neuroscience Letters.

[32]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[33]  C. Spence,et al.  Tactile selective attention and body posture: Assessing the multisensory contributions of vision and proprioception , 2004, Perception & psychophysics.

[34]  D. Bavelier,et al.  Components of visual prior entry , 2003, Cognitive Psychology.

[35]  L. Allan.  The relationship between judgments ofsuccessiveness and judgments of order , 1975 .

[36]  G. Aschersleben,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[37]  P. Chauvel,et al.  Localization of the primary auditory area in man. , 1991, Brain : a journal of neurology.

[38]  C. Spence,et al.  Audio-visual simultaneity judgments , 2005, Perception & psychophysics.

[39]  B. Libet.  Mind Time: The Temporal Factor in Consciousness , 2004 .

[40]  G. R. Engel,et al.  Visual-Auditory Distance Constancy , 1971, Nature.

[41]  G. Aschersleben.  Task-dependent timing of perceptual events , 1999 .

[42]  Charles Spence,et al.  Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. , 2005, Brain research. Cognitive brain research.

[43]  G. VonBékésy,et al.  INTERACTION OF PAIRED SENSORY STIMULI AND CONDUCTION IN PERIPHERAL NERVES. , 1963, Journal of applied physiology.

[44]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[45]  S. Kitazawa,et al.  Bayesian calibration of simultaneity in tactile temporal order judgment , 2006, Nature Neuroscience.

[46]  D. Burke,et al.  Conduction velocities of muscle and cutaneous afferents in the upper and lower limbs of human subjects. , 1989, Brain : a journal of neurology.

[47]  Laurence R. Harris,et al.  Auditory–visual temporal integration measured by shifts in perceived temporal location , 2007, Neuroscience Letters.

[48]  R. L. Knoll,et al.  The Perception of Temporal Order: Fundamental Issues and a General Model , 1973 .

[49]  Vincent Walsh,et al.  Perceptual Constancy: why things look as they do , 1998 .

[50]  C. Spence,et al.  Audiotactile temporal order judgments. , 2005, Acta psychologica.

[51]  L. Harris,et al.  Simultaneity Constancy , 2004, Perception.