Audiovisual Temporal Recalibration: Space-Based versus Context-Based

Recalibration of perceived simultaneity has been widely accepted to minimise delay between multisensory signals owing to different physical and neural conduct times. With concurrent exposure, temporal recalibration is either contextually or spatially based. Context-based recalibration was recently described in detail, but evidence for space-based recalibration is scarce. In addition, the competition between these two reference frames is unclear. Here, we examined participants who watched two distinct blob-and-tone couples that laterally alternated with one asynchronous and the other synchronous and then judged their perceived simultaneity and sequence when they swapped positions and varied in timing. For low-level stimuli with abundant auditory location cues space-based aftereffects were significantly more apparent (8.3%) than context-based aftereffects (4.2%), but without such auditory cues space-based aftereffects were less apparent (4.4%) and were numerically smaller than context-based aftereffects (6.0%). These results suggested that stimulus level and auditory location cues were both determinants of the recalibration frame. Through such joint judgments and the simple reaction time task, our results further revealed that criteria from perceived simultaneity to successiveness profoundly shifted without accompanying perceptual latency changes across adaptations, hence implying that criteria shifts, rather than perceptual latency changes, accounted for space-based and context-based temporal recalibration.

[1]  R. Andersen,et al.  Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. , 1996, Journal of neurophysiology.

[2]  T. Sato,et al.  Perceiving the direction of walking , 2008 .

[3]  R. Ulrich Threshold models of temporal-order judgments evaluated by a ternary response task , 1987, Perception & psychophysics.

[4]  J. Y. Goulermas,et al.  Multivoxel fMRI analysis of color tuning in human primary visual cortex. , 2009, Journal of vision.

[5]  L. Harris,et al.  The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity , 2008, Experimental Brain Research.

[6]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[7]  Mark Wexler,et al.  The nonlinear structure of motion perception during smooth eye movements. , 2009, Journal of vision.

[8]  J. Vroomen,et al.  Perception of intersensory synchrony: A tutorial review , 2010, Attention, perception & psychophysics.

[9]  Joseph A Maldjian,et al.  Cross‐modal sensory processing in the anterior cingulate and medial prefrontal cortices , 2003, Human brain mapping.

[10]  B. Stein,et al.  Spatial determinants of multisensory integration in cat superior colliculus neurons. , 1996, Journal of neurophysiology.

[11]  R. Verleger,et al.  Attentional Bias toward Low-Intensity Stimuli: An Explanation for the Intensity Dissociation between Reaction Time and Temporal Order Judgment? , 2000, Consciousness and Cognition.

[12]  M T Wallace,et al.  Comparisons of cross-modality integration in midbrain and cortex. , 1996, Progress in brain research.

[13]  B. Stein,et al.  Adult Plasticity in Multisensory Neurons: Short-Term Experience-Dependent Changes in the Superior Colliculus , 2009, The Journal of Neuroscience.

[14]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[15]  David Whitaker,et al.  Adaptation minimizes distance-related audiovisual delays. , 2007, Journal of vision.

[16]  C. Spence,et al.  Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli , 2007, Perception & psychophysics.

[17]  Bruno B Averbeck,et al.  Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex , 2006, The Journal of Neuroscience.

[18]  Derek H. Arnold,et al.  Twice Upon a Time , 2011, Psychological science.

[19]  L. Allan The relationship between judgments of successiveness , 1975 .

[20]  Y. Xiang Audiovisual Temporal Recalibration: The Combined and Separate Effect of Adapted Space and Object , 2012 .

[21]  Robert Sekuler,et al.  Learning to imitate novel motion sequences. , 2007, Journal of vision.

[22]  Derek H. Arnold,et al.  Timing sight and sound , 2005, Vision Research.

[23]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[24]  R. Andersen,et al.  Reaches to Sounds Encoded in an Eye-Centered Reference Frame , 2000, Neuron.

[25]  Mark T. Wallace,et al.  The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus , 2001, Experimental Brain Research.

[26]  Frank W. Ohl,et al.  Effects of Prolonged Exposure to Audiovisual Stimuli with Fixed Stimulus Onset Asynchrony on Interaction Dynamics between Primary Auditory and Primary Visual Cortex , 2012 .

[27]  Laurie M Wilcox,et al.  A reevaluation of the tolerance to vertical misalignment in stereopsis. , 2009, Journal of vision.

[28]  Derek H. Arnold,et al.  Spatial grouping resolves ambiguity to drive temporal recalibration. , 2011, Journal of experimental psychology. Human perception and performance.

[29]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[30]  Jean Vroomen,et al.  No effect of auditory–visual spatial disparity on temporal recalibration , 2007, Experimental Brain Research.

[31]  Shin'ya Nishida,et al.  The spatial tuning of adaptation-based time compression. , 2009, Journal of vision.

[32]  David Whitaker,et al.  Audiovisual time perception is spatially specific , 2012, Experimental Brain Research.

[33]  C. Spence,et al.  Adaptation to audiovisual asynchrony modulates the speeded detection of sound , 2009, Proceedings of the National Academy of Sciences.

[34]  David L. Sparks,et al.  Auditory receptive fields in primate superior colliculus shift with changes in eye position , 1984, Nature.

[35]  Mark T. Wallace,et al.  The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus , 2002, Experimental Brain Research.

[36]  S. Nishida,et al.  Marker Correspondence, Not Processing Latency, Determines Temporal Binding of Visual Attributes , 2002, Current Biology.

[37]  L. Allan The relationship between judgments ofsuccessiveness and judgments of order , 1975 .

[38]  Derek H. Arnold,et al.  Shifts of criteria or neural timing? The assumptions underlying timing perception studies , 2011, Consciousness and Cognition.

[39]  Massimiliano Di Luca,et al.  Recalibration of multisensory simultaneity: cross-modal transfer coincides with a change in perceptual latency. , 2009, Journal of vision.

[40]  Eye-Centered , Head-Centered , and Intermed iate Cod Remembered Sound Locations in Area LIP , 2002 .

[41]  Gerald Westheimer,et al.  Grouping of contextual elements that affect vernier thresholds. , 2007, Journal of vision.

[42]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[43]  Neil W. Roach,et al.  Asynchrony adaptation reveals neural population code for audio-visual timing , 2010, Proceedings of the Royal Society B: Biological Sciences.