Attention regulates the plasticity of multisensory timing

Evidence suggests than human time perception is likely to reflect an ensemble of recent temporal experience. For example, prolonged exposure to consistent temporal patterns can adaptively realign the perception of event order, both within and between sensory modalities (e.g. Fujisaki et al., 2004 Nat. Neurosci., 7, 773–778). In addition, the observation that ‘a watched pot never boils’ serves to illustrate the fact that dynamic shifts in our attentional state can also produce marked distortions in our temporal estimates. In the current study we provide evidence for a hitherto unknown link between adaptation, temporal perception and our attentional state. We show that our ability to use recent sensory history as a perceptual baseline for ongoing temporal judgments is subject to striking top‐down modulation via shifts in the observer’s selective attention. Specifically, attending to the temporal structure of asynchronous auditory and visual adapting stimuli generates a substantial increase in the temporal recalibration induced by these stimuli. We propose a conceptual framework accounting for our findings whereby attention modulates the perceived salience of temporal patterns. This heightened salience allows the formation of audiovisual perceptual ‘objects’, defined solely by their temporal structure. Repeated exposure to these objects induces high‐level pattern adaptation effects, akin to those found in visual and auditory domains (e.g. Leopold & Bondar (2005) Fitting the Mind to the World: Adaptation and Aftereffects in High‐Level Vision. Oxford University Press, Oxford, 189–211; Schweinberger et al. (2008) Curr. Biol., 18, 684–688).

[1]  W. Bevan,et al.  The perceived duration of auditory and visual intervals: cross-modal comparison and interaction. , 1961, American Journal of Psychology.

[2]  M. Treisman Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[3]  Ewart A. C. Thomas,et al.  Time perception and the filled-duration illusion , 1974 .

[4]  Ewart A. C. Thomas,et al.  Cognitive processing and time perception , 1975 .

[5]  M. Bozkurt,et al.  Functional anatomy. , 1980, Equine veterinary journal.

[6]  James T. Walker,et al.  Simple and contingent aftereffects of perceived duration in vision and audition , 1981, Perception & psychophysics.

[7]  J T Petersik The perceptual fate of letters in two kinds of apparent movement displays , 1984, Perception & psychophysics.

[8]  S. W. Brown,et al.  Time perception and attention: The effects of prospective versus retrospective paradigms and task demands on perceived duration , 1985, Perception & psychophysics.

[9]  A. Chaudhuri Modulation of the motion aftereffect by selective attention , 1990, Nature.

[10]  S. Anstis,et al.  Properties of the visual channels that underlie adaptation to gradual change of luminance , 1993, Vision Research.

[11]  Richard B. Ivry,et al.  Neural mechanisms of timing , 1997, Trends in Cognitive Sciences.

[12]  Scott W. Brown Attentional resources in timing: Interference effects in concurrent temporal and nontemporal working memory tasks , 1997, Perception & psychophysics.

[13]  R Blake,et al.  Spatial and temporal coherence in perceptual binding. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[14]  D. Zakay Attention allocation policy influences prospective timing , 1998 .

[15]  James V. Stone Object recognition using spatiotemporal signatures , 1998, Vision Research.

[16]  Marius Usher,et al.  Visual synchrony affects binding and segmentation in perception , 1998, Nature.

[17]  R Blake,et al.  Visual form created solely from temporal structure. , 1999, Science.

[18]  Otto H. MacLin,et al.  Figural aftereffects in the perception of faces , 1999, Psychonomic bulletin & review.

[19]  Frans A. J. Verstraten,et al.  Independent Aftereffects of Attention and Motion , 2000, Neuron.

[20]  Michael J. Spivey,et al.  Selective visual attention modulates the direct tilt aftereffect , 2000, Perception & psychophysics.

[21]  A. O'Toole,et al.  Prototype-referenced shape encoding revealed by high-level aftereffects , 2001, Nature Neuroscience.

[22]  D. Heeger,et al.  Neuronal Basis of the Motion Aftereffect Reconsidered , 2001, Neuron.

[23]  S. Shimojo,et al.  Sound alters visual evoked potentials in humans , 2001, Neuroreport.

[24]  John C. Rothwell,et al.  Illusory perceptions of space and time preserve cross-saccadic perceptual continuity , 2001, Nature.

[25]  S. Suzuki,et al.  Attention-dependent brief adaptation to contour orientation: a high-level aftereffect for convexity? , 2001, Vision Research.

[26]  M. Pinsk,et al.  Attention modulates responses in the human lateral geniculate nucleus , 2002, Nature Neuroscience.

[27]  R. Aslin,et al.  Statistical learning of higher-order temporal structure from visual shape sequences. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[28]  Marvin M Chun,et al.  Visual marking: selective attention to asynchronous temporal groups. , 2002, Journal of experimental psychology. Human perception and performance.

[29]  Adriane E Seiffert,et al.  Functional MRI studies of human visual motion perception: texture, luminance, attention and after-effects. , 2003, Cerebral cortex.

[30]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[31]  G. Recanzone Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[32]  Amy A. Rezec,et al.  Attention enhances adaptability: evidence from motion adaptation experiments , 2004, Vision Research.

[33]  F. Vidal,et al.  Functional Anatomy of the Attentional Modulation of Time Estimation , 2004, Science.

[34]  A. Cowey,et al.  Chronostasis without voluntary action , 2005, Experimental Brain Research.

[35]  Liang-Shih Fan,et al.  Electrical capacitance tomography imaging of gas-solid and gas-liquid-solid fluidized bed systems , 2004, J. Vis..

[36]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[37]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[38]  Leila Montaser-Kouhsari,et al.  Attentional modulation of adaptation to illusory lines. , 2004, Journal of vision.

[39]  M. Behrmann,et al.  Role of attention and perceptual grouping in visual statistical learning. , 2004, Psychological science.

[40]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[41]  D. Leopold,et al.  Adaptation to complex visual patterns in humans and monkeys , 2005 .

[42]  Randolph Blake,et al.  The role of temporal structure in human vision. , 2005, Behavioral and cognitive neuroscience reviews.

[43]  Christof Koch,et al.  Face Adaptation Depends on Seeing the Face , 2005, Neuron.

[44]  S. Hillyard,et al.  Neural basis of auditory-induced shifts in visual time-order perception , 2005, Nature Neuroscience.

[45]  B. Scholl,et al.  The Automaticity of Visual Statistical Learning Statistical Learning , 2005 .

[46]  M Concetta Morrone,et al.  Saccadic eye movements cause compression of time as well as space , 2005, Nature Neuroscience.

[47]  Charles Spence,et al.  Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. , 2005, Brain research. Cognitive brain research.

[48]  Randolph Blake,et al.  Mixed messengers, unified message: spatial grouping from temporal structure , 2005, Vision Research.

[49]  Frans A. J. Verstraten,et al.  Attention-based motion perception and motion adaptation: What does attention contribute? , 2005, Vision Research.

[50]  G. Rhodes,et al.  Fitting the Mind to the World: Adaptation and after-effects in high-level vision , 2005 .

[51]  Shin'ya Nishida,et al.  Visual search for a target changing in synchrony with an auditory signal , 2006, Proceedings of the Royal Society B: Biological Sciences.

[52]  Waka Fujisaki,et al.  Temporal frequency characteristics of synchrony–asynchrony discrimination of audio-visual signals , 2005, Experimental Brain Research.

[53]  Frans A. J. Verstraten,et al.  The Scope and Limits of Top-Down Attention in Unconscious Visual Processing , 2006, Current Biology.

[54]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[55]  P. Montague,et al.  Motor-Sensory Recalibration Leads to an Illusory Reversal of Action and Sensation , 2006, Neuron.

[56]  John Schlag,et al.  Transfer of learned perception of sensorimotor simultaneity , 2006, Experimental Brain Research.

[57]  Derek H. Arnold,et al.  Spatially Localized Distortions of Event Time , 2006, Current Biology.

[58]  S. Jackson,et al.  Recalibrating Time: When Did I Do that? , 2006, Current Biology.

[59]  M Concetta Morrone,et al.  Neural mechanisms for timing visual events are spatially selective in real-world coordinates , 2007, Nature Neuroscience.

[60]  David Whitaker,et al.  Adaptation minimizes distance-related audiovisual delays. , 2007, Journal of vision.

[61]  Charles Spence,et al.  Adaptation to audiotactile asynchrony , 2007, Neuroscience Letters.

[62]  Randolph Blake,et al.  Spatial grouping in human vision: Temporal structure trumps temporal synchrony , 2007, Vision Research.

[63]  L. Harris,et al.  The effect of exposure to asynchronous audio, visual, and tactile stimulus combinations on the perception of simultaneity , 2008, Experimental Brain Research.

[64]  Jean Vroomen,et al.  No effect of auditory–visual spatial disparity on temporal recalibration , 2007, Experimental Brain Research.

[65]  C. Spence,et al.  Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments , 2008, Experimental Brain Research.

[66]  Shin’ya Nishida,et al.  Feature-based processing of audio-visual synchrony perception revealed by random pulse trains , 2007, Vision Research.

[67]  B. Bahrami,et al.  Attentional Load Modulates Responses of Human Primary Visual Cortex to Invisible Stimuli , 2007, Current Biology.

[68]  Vani Pariyadath,et al.  Brief subjective durations contract with repetition. , 2008, Journal of vision.

[69]  Waka Fujisaki,et al.  Top-down feature-based selection of matching features for audio-visual synchrony discrimination , 2008, Neuroscience Letters.

[70]  Geraint Rees,et al.  Unconscious orientation processing depends on perceptual load. , 2008, Journal of vision.

[71]  D. Whitaker,et al.  Recalibration of perceived time across sensory modalities , 2008, Experimental Brain Research.

[72]  M. Nicolelis,et al.  Decoding of temporal intervals from cortical ensemble activity. , 2008, Journal of neurophysiology.

[73]  J. Vroomen,et al.  Temporal recalibration to tactile–visual asynchronous stimuli , 2008, Neuroscience Letters.

[74]  T. Sato,et al.  Perceiving the direction of walking , 2008 .

[75]  Hideki Kawahara,et al.  Auditory Adaptation in Voice Perception , 2008, Current Biology.

[76]  Frans A. J. Verstraten,et al.  Matching Auditory and Visual Signals: Is Sensory Modality Just Another Feature? , 2008, Perception.

[77]  Katsumi Watanabe,et al.  Realignment of temporal simultaneity between vision and touch , 2008, Neuroreport.

[78]  J. Tanji,et al.  Interval time coding by neurons in the presupplementary and supplementary motor areas , 2009, Nature Neuroscience.

[79]  John Christie,et al.  Temporal Order Judgments Activate Temporal Parietal Junction , 2009, The Journal of Neuroscience.

[80]  Aldo Genovesio,et al.  Feature- and Order-Based Timing Representations in the Frontal Cortex , 2009, Neuron.

[81]  Simon K Rushton,et al.  Adaptation to Sensory-Motor Temporal Misalignment: Instrumental or Perceptual Learning? , 2009, Quarterly journal of experimental psychology.

[82]  Guy Wallis,et al.  Learning Illumination-and Orientation-invariant Representations of Objects through Temporal Association General Methods Experiment Ii , 2022 .

[83]  C. Spence,et al.  Adaptation to audiovisual asynchrony modulates the speeded detection of sound , 2009, Proceedings of the National Academy of Sciences.