Crossmodal duration perception involves perceptual grouping, temporal ventriloquism, and variable internal clock rates

Here, we investigate how audiovisual context affects perceived event duration with experiments in which observers reported which of two stimuli they perceived as longer. Target events were visual and/or auditory and could be accompanied by nontargets in the other modality. Our results demonstrate that the temporal information conveyed by irrelevant sounds is automatically used when the brain estimates visual durations but that irrelevant visual information does not affect perceived auditory duration (Experiment 1). We further show that auditory influences on subjective visual durations occur only when the temporal characteristics of the stimuli promote perceptual grouping (Experiments 1 and 2). Placed in the context of scalar expectancy theory of time perception, our third and fourth experiments have the implication that audiovisual context can lead both to changes in the rate of an internal clock and to temporal ventriloquism-like effects on perceived on- and offsets. Finally, intramodal grouping of auditory stimuli diminished any crossmodal effects, suggesting a strong preference for intramodal over crossmodal perceptual grouping (Experiment 5).

[1]  R. Turner,et al.  Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain , 2000, Current Biology.

[2]  J. Devin McAuley,et al.  Duration Discrimination in Crossmodal Sequences , 2009, Perception.

[3]  C. Spence,et al.  Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli. , 2008, Acta psychologica.

[4]  F A Wichmann,et al.  Ning for Helpful Comments and Suggestions. This Paper Benefited Con- Siderably from Conscientious Peer Review, and We Thank Our Reviewers the Psychometric Function: I. Fitting, Sampling, and Goodness of Fit , 2001 .

[5]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[6]  A. Kingstone,et al.  Auditory capture of vision: examining temporal ventriloquism. , 2003, Brain research. Cognitive brain research.

[7]  Yôiti Suzuki,et al.  Implicit estimation of sound-arrival time , 2003, Nature.

[8]  Laurence R. Harris,et al.  Auditory–visual temporal integration measured by shifts in perceived temporal location , 2007, Neuroscience Letters.

[9]  Frans A. J. Verstraten,et al.  Dynamic Perceptual Changes in Audiovisual Simultaneity , 2007, PloS one.

[10]  D. Eagleman Human time perception and its illusions , 2008, Current Opinion in Neurobiology.

[11]  G. Recanzone Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[12]  D. Burr,et al.  Combining visual and auditory information. , 2006, Progress in brain research.

[13]  J. Wearden,et al.  Why “Sounds Are Judged Longer than Lights”: Application of a Model of the Internal Clock in Humans , 1998, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[14]  M Concetta Morrone,et al.  Saccadic eye movements cause compression of time as well as space , 2005, Nature Neuroscience.

[15]  C. Spence,et al.  Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli , 2007, Perception & psychophysics.

[16]  Vani Pariyadath,et al.  Brief subjective durations contract with repetition. , 2008, Journal of vision.

[17]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[18]  Sheng He,et al.  Larger stimuli are judged to last longer. , 2007, Journal of vision.

[19]  Eli Brenner,et al.  Temporal information can influence spatial localization. , 2009, Journal of neurophysiology.

[20]  A. Angrilli,et al.  The influence of affective factors on time perception , 1997, Perception & psychophysics.

[21]  J. T. Walker,et al.  Auditory-visual conflicts in the perceived duration of lights, tones and gaps. , 1981, Journal of experimental psychology. Human perception and performance.

[22]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[23]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[24]  C. Spence,et al.  Multisensory perception: Beyond modularity and convergence , 2000, Current Biology.

[25]  Shinsuke Shimojo,et al.  Distortions of Subjective Time Perception Within and Across Senses , 2008, PloS one.

[26]  J H Wearden,et al.  Speeding up an internal clock in humans? Effects of click trains on subjective duration. , 1996, Journal of experimental psychology. Animal behavior processes.

[27]  G. Aschersleben,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[28]  Ilana B. Witten,et al.  Why Seeing Is Believing: Merging Auditory and Visual Worlds , 2005, Neuron.

[29]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[30]  Charles Spence,et al.  When does visual perceptual grouping affect multisensory integration? , 2004, Cognitive, affective & behavioral neuroscience.

[31]  John C. Rothwell,et al.  Illusory perceptions of space and time preserve cross-saccadic perceptual continuity , 2001, Nature.

[32]  A. V. van den Berg,et al.  Occlusion-related lateral connections stabilize kinetic depth stimuli through perceptual coupling. , 2009, Journal of vision.

[33]  S. Getzmann,et al.  Audiovisual influences on the perception of visual apparent motion: exploring the effect of a single sound. , 2008, Acta psychologica.

[34]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[35]  A. Watson,et al.  Quest: A Bayesian adaptive psychometric method , 1983, Perception & psychophysics.

[36]  Charles Spence,et al.  Intramodal perceptual grouping modulates multisensory integration: evidence from the crossmodal dynamic capture task , 2005, Neuroscience Letters.

[37]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[38]  C. Spence,et al.  Cross-modal links in spatial attention. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[39]  M. Wallace,et al.  Unifying multisensory signals across time and space , 2004, Experimental Brain Research.

[40]  Madeleine Schlag-Rey,et al.  Voluntary action expands perceived duration of its sensory consequence , 2003, Experimental Brain Research.

[41]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[42]  D Rose,et al.  Duration Illusions in a Train of Visual Stimuli , 1995, Perception.

[43]  Charles Spence,et al.  The modulation of crossmodal integration by unimodal perceptual grouping: a visuotactile apparent motion study , 2006, Experimental Brain Research.

[44]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[45]  Stephan Getzmann,et al.  The Effect of Brief Auditory Stimuli on Visual Apparent Motion , 2007, Perception.

[46]  Masataka Watanabe,et al.  Visual onset expands subjective time , 2006, Perception & psychophysics.

[47]  Bruno H Repp,et al.  Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. , 2002, Journal of experimental psychology. Human perception and performance.

[48]  S. Shimojo,et al.  Visual illusion induced by sound. , 2002, Brain research. Cognitive brain research.

[49]  J. Gibbon,et al.  Interval-timing deficits in individuals at high risk for schizophrenia , 2005, Brain and Cognition.

[50]  Jean Vroomen,et al.  Auditory grouping occurs prior to intersensory pairing: evidence from temporal ventriloquism , 2007, Experimental Brain Research.

[51]  Norimichi Kitagawa,et al.  Audio-visual integration in temporal perception. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[52]  Su-Ling Yeh,et al.  Asymmetric cross-modal effects in time perception. , 2009, Acta psychologica.

[53]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[54]  Simon Grondin,et al.  Sensory modalities and temporal processing. , 2003 .

[55]  Jon Driver,et al.  Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound , 2008, Current Biology.

[56]  J. Gibbon,et al.  Differential effects of auditory and visual signals on clock speed and temporal memory. , 2000, Journal of experimental psychology. Human perception and performance.

[57]  Shinsuke Shimojo,et al.  Perceptual organization of moving stimuli modulates the flash-lag effect. , 2001, Journal of experimental psychology. Human perception and performance.

[58]  Jean Vroomen,et al.  No effect of auditory–visual spatial disparity on temporal recalibration , 2007, Experimental Brain Research.

[59]  Robert Rousseau,et al.  Stop—reaction time and the internal clock , 1996, Perception & psychophysics.

[60]  J. Devin McAuley,et al.  Time judgments in global temporal contexts , 2005, Perception & psychophysics.

[61]  D. Burr,et al.  Auditory dominance over vision in the perception of interval duration , 2009, Experimental Brain Research.

[62]  R M Church,et al.  Scalar Timing in Memory , 1984, Annals of the New York Academy of Sciences.

[63]  A. Kingstone,et al.  Flexible and abstract resolutions to crossmodal conflicts , 2004, Brain and Cognition.

[64]  Ryota Kanai,et al.  Time dilation in dynamic visual display. , 2006, Journal of vision.

[65]  D. Buonomano,et al.  The neural basis of temporal processing. , 2004, Annual review of neuroscience.

[66]  M. Boltz,et al.  Duration judgments of naturalistic events in the auditory and visual modalities , 2005, Perception & psychophysics.