Visual and Audiovisual Effects of Isochronous Timing on Visual Perception and Brain Activity

Understanding how the brain extracts and combines temporal structure (rhythm) information from events presented to different senses remains unresolved. Many neuroimaging beat perception studies have focused on the auditory domain and show the presence of a highly regular beat (isochrony) in “auditory” stimulus streams enhances neural responses in a distributed brain network and affects perceptual performance. Here, we acquired functional magnetic resonance imaging (fMRI) measurements of brain activity while healthy human participants performed a visual task on isochronous versus randomly timed “visual” streams, with or without concurrent task-irrelevant sounds. We found that visual detection of higher intensity oddball targets was better for isochronous than randomly timed streams, extending previous auditory findings to vision. The impact of isochrony on visual target sensitivity correlated positively with fMRI signal changes not only in visual cortex but also in auditory sensory cortex during audiovisual presentations. Visual isochrony activated a similar timing-related brain network to that previously found primarily in auditory beat perception work. Finally, activity in multisensory left posterior superior temporal sulcus increased specifically during concurrent isochronous audiovisual presentations. These results indicate that regular isochronous timing can modulate visual processing and this can also involve multisensory audiovisual brain mechanisms.

[1]  Houeto Jean-Luc [Parkinson's disease]. , 2022, La Revue du praticien.

[2]  P Bertelson,et al.  The Time Course of Preparation* , 1967, The Quarterly journal of experimental psychology.

[3]  A. Nobre,et al.  Dissociating explicit timing from temporal expectation with fMRI , 2008, Current Opinion in Neurobiology.

[4]  Antonino Vallesi,et al.  Role of the prefrontal cortex in the foreperiod effect: TMS evidence for dual mechanisms in temporal preparation. , 2006, Cerebral cortex.

[5]  D. V. von Cramon,et al.  Functional organization of the lateral premotor cortex: fMRI reveals different regions activated by anticipation of object properties, location and speed. , 2001, Brain research. Cognitive brain research.

[6]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[7]  J. Devin McAuley,et al.  Neural bases of individual differences in beat perception , 2009, NeuroImage.

[8]  J. Devin McAuley,et al.  Duration Discrimination in Crossmodal Sequences , 2009, Perception.

[9]  B. Repp,et al.  Rhythmic movement is attracted more strongly to auditory than to visual rhythms , 2004, Psychological research.

[10]  Michael J. Todd,et al.  Vergence-mediated changes in the axis of eye rotation during the human vestibulo-ocular reflex can occur independent of eye position , 2003, Experimental Brain Research.

[11]  Franck Vidal,et al.  Orienting Attention in Time Activates Left Intraparietal Sulcus for Both Perceptual and Motor Task Goals , 2011, Journal of Cognitive Neuroscience.

[12]  G. Recanzone Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[13]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[14]  T. Griffiths,et al.  The planum temporale as a computational hub , 2002, Trends in Neurosciences.

[15]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[16]  J. Mabbott I.—OUR DIRECT EXPERIENCE OF TIME , 1951 .

[17]  A. Federspiel,et al.  Brain responses to auditory and visual stimulus offset: Shared representations of temporal edges , 2009, Human brain mapping.

[18]  A. T. Smith,et al.  The role of the human anterior insular cortex in time processing , 2010, Brain Structure and Function.

[19]  E. Bullmore,et al.  Society for Neuroscience Abstracts , 1997 .

[20]  D. Pandya,et al.  Intrinsic connections and architectonics of the superior temporal sulcus in the rhesus monkey , 1989, The Journal of comparative neurology.

[21]  Matthew B. Wall,et al.  Time perception: Manipulation of task difficulty dissociates clock functions from other cognitive demands , 2007, Neuropsychologia.

[22]  R. Carlyon,et al.  An Information Theoretic Characterisation of Auditory Encoding , 2007, PLoS biology.

[23]  Ross M. VanDerKlok,et al.  Multisensory perception of action in posterior temporal and parietal cortices , 2011, Neuropsychologia.

[24]  Franck Vidal,et al.  The supplementary motor area in motor and perceptual time processing: fMRI studies , 2006, Cognitive Processing.

[25]  F. Vidal,et al.  Functional Anatomy of the Attentional Modulation of Time Estimation , 2004, Science.

[26]  Lars Timmermann,et al.  Modulation of Human Time Processing by Subthalamic Deep Brain Stimulation , 2011, PloS one.

[27]  Hugo Merchant,et al.  Do we have a common mechanism for measuring time in the hundreds of millisecond range? Evidence from multiple-interval timing tasks. , 2008, Journal of neurophysiology.

[28]  B. Milliken,et al.  Endogenous temporal orienting of attention in detection and discrimination tasks , 2004, Perception & psychophysics.

[29]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[30]  C Miniussi,et al.  Orienting attention in time. , 2001, Frontiers in bioscience : a journal and virtual library.

[31]  Martin Wiener,et al.  Implicit timing activates the left inferior parietal cortex , 2010, Neuropsychologia.

[32]  Elyse Sussman,et al.  Auditory Target Detection Is Affected by Implicit Temporal and Spatial Expectations , 2011, Journal of Cognitive Neuroscience.

[33]  D. Harrington,et al.  The effects of stimulus modality and frequency of stimulus presentation on cross-modal distraction. , 2009, Cerebral cortex.

[34]  Geraint Rees,et al.  Encoding of Temporal Probabilities in the Human Brain , 2010, The Journal of Neuroscience.

[35]  P. J. Foley The foreperiod and simple reaction time. , 1959, Canadian journal of psychology.

[36]  Hans Forssberg,et al.  Listening to rhythms activates motor and premotor cortices , 2009, Cortex.

[37]  P. A. Lewis,et al.  A right hemispheric prefrontal system for cognitive time measurement , 2006, Behavioural Processes.

[38]  Jessica A. Grahn,et al.  Feeling the Beat: Premotor and Striatal Interactions in Musicians and Nonmusicians during Beat Perception , 2009, The Journal of Neuroscience.

[39]  Gereon R Fink,et al.  Left inferior parietal cortex integrates time and space during collision judgments , 2003, NeuroImage.

[40]  E Macaluso,et al.  Spatial and temporal factors during processing of audiovisual speech: a PET study , 2004, NeuroImage.

[41]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[42]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[43]  A. Nobre,et al.  Orienting attention in time: behavioural and neuroanatomical distinction between exogenous and endogenous shifts , 2000, Neuropsychologia.

[44]  F. Vidal,et al.  Activation of the supplementary motor area and of attentional networks during temporal processing , 2002, Experimental Brain Research.

[45]  Anna Christina Nobre,et al.  Orienting attention to instants in time , 2001, Neuropsychologia.

[46]  K. Zilles,et al.  Difficulty of perceptual spatiotemporal integration modulates the neural activity of left inferior parietal cortex , 2005, Neuroscience.

[47]  Alan C. Evans,et al.  An MRI-based stereotactic atlas from 250 young normal subjects , 1992 .

[48]  U. Noppeney,et al.  Audiovisual Synchrony Improves Motion Discrimination via Enhanced Connectivity between Early Visual and Auditory Areas , 2010, The Journal of Neuroscience.

[49]  Matthew Brett,et al.  Rhythm and Beat Perception in Motor Areas of the Brain , 2007, Journal of Cognitive Neuroscience.

[50]  J. Kaiser,et al.  Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas , 2007, The Journal of Neuroscience.

[51]  Warren H Meck,et al.  Cortico-striatal Representation of Time in Animals and Humans This Review Comes from a Themed Issue on Cognitive Neuroscience Edited Evidence from Patient Populations and Electrical Potentials Neuroimaging Evidence Using Fmri and Pet , 2022 .

[52]  J. Devin McAuley,et al.  FMRI investigation of cross-modal interactions in beat perception: Audition primes vision, but not vice versa , 2011, NeuroImage.

[53]  M. Brett,et al.  Impairment of beat-based rhythm discrimination in Parkinson's disease , 2009, Cortex.

[54]  P Bertelson,et al.  The Time-Course of Preparation with Regular and Irregular Foreperiods , 1968, The Quarterly journal of experimental psychology.

[55]  Anna C. Nobre,et al.  Synergistic Effect of Combined Temporal and Spatial Expectations on Visual Attention , 2005, The Journal of Neuroscience.

[56]  Menachem Jona,et al.  Temporal coding in rhythm tasks revealed by modality effects , 1991, Memory & cognition.

[57]  R. Miall,et al.  Brain activation patterns during measurement of sub- and supra-second intervals , 2003, Neuropsychologia.

[58]  S Lehéricy,et al.  Basal ganglia and supplementary motor area subtend duration perception: an fMRI study , 2003, NeuroImage.

[59]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[60]  Gregor Kjellström The evolution in the brain , 1999, Appl. Math. Comput..

[61]  M. Jones,et al.  Effects of auditory pattern structure on anticipatory and reactive attending , 2006, Cognitive Psychology.

[62]  Timothy D. Griffiths,et al.  Distinct Neural Substrates of Duration-Based and Beat-Based Auditory Timing , 2011, The Journal of Neuroscience.

[63]  M. Jones,et al.  Temporal Aspects of Stimulus-Driven Attending in Dynamic Arrays , 2002, Psychological science.

[64]  A. Friederici,et al.  Time Perception and Motor Timing: A Common Cortical and Subcortical Basis Revealed by fMRI , 2000, NeuroImage.

[65]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[66]  Jennifer T. Coull,et al.  Behavioural Dissociation between Exogenous and Endogenous Temporal Orienting of Attention , 2011, PloS one.

[67]  U. Noppeney,et al.  Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. , 2010, Cerebral cortex.

[68]  Benjamin Morillon,et al.  Three Stages and Four Neural Systems in Time Estimation , 2009, The Journal of Neuroscience.

[69]  J. Gibbon,et al.  Coupled Temporal Memories in Parkinson's Disease: A Dopamine-Related Dysfunction , 1998, Journal of Cognitive Neuroscience.

[70]  T. Rammsayer,et al.  Impaired temporal discrimination in Parkinson's disease: temporal processing of brief durations as an indicator of degeneration of dopaminergic neurons in the basal ganglia. , 1997, The International journal of neuroscience.

[71]  Ryan A. Stevenson,et al.  Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition , 2009, NeuroImage.

[72]  Terry M. Peters,et al.  3D statistical neuroanatomical models from 305 MRI volumes , 1993, 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference.

[73]  Deborah L. Harrington,et al.  Integrative Neuroscience Original Research Article Neural Underpinnings of Distortions in the Experience of Time across Senses , 2022 .

[74]  J. Lupiáñez,et al.  Attentional preparation based on temporal expectancy modulates processing at the perceptual level , 2005, Psychonomic bulletin & review.

[75]  Jon Driver,et al.  Audiovisual synchrony enhances BOLD responses in a brain network including multisensory STS while also enhancing target‐detection performance for both modalities , 2011, Human brain mapping.

[76]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[77]  W. Meck Neuroanatomical localization of an internal clock: A functional link between mesolimbic, nigrostriatal, and mesocortical dopaminergic systems , 2006, Brain Research.

[78]  Stephen M. Rao,et al.  The evolution of brain activation during temporal processing , 2001, Nature Neuroscience.

[79]  R. Knight,et al.  Cortical Networks Underlying Mechanisms of Time Perception , 1998, The Journal of Neuroscience.

[80]  W. Singer,et al.  Retinotopic effects during spatial audio-visual integration , 2007, Neuropsychologia.

[81]  Masaharu Kato,et al.  Auditory dominance in the error correction process: A synchronized tapping study , 2006, Brain Research.

[82]  S. Martens,et al.  Timing attention: Cuing target onset interval attenuates the attentional blink , 2005, Memory & cognition.

[83]  Mark G. Stokes,et al.  Functionally dissociating temporal and motor components of response preparation in left intraparietal sulcus , 2011, NeuroImage.

[84]  M. Jahanshahi,et al.  Time estimation and reproduction is abnormal in Parkinson's disease. , 1992, Brain : a journal of neurology.

[85]  J A Obeso,et al.  Temporal discrimination is abnormal in Parkinson's disease. , 1992, Brain : a journal of neurology.

[86]  Arthur M. Glenberg,et al.  Modality effects in the coding reproduction of rhythms , 1989 .

[87]  Christian Gaser,et al.  Processing of temporal information and the basal ganglia: new evidence from fMRI , 2003, Experimental Brain Research.

[88]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[89]  A. Nobre,et al.  Where and When to Pay Attention: The Neural Systems for Directing Attention to Spatial Locations and to Time Intervals as Revealed by Both PET and fMRI , 1998, The Journal of Neuroscience.

[90]  A. Glenberg,et al.  Modality effects in the coding and reproduction of rhythms. , 1989, Memory & cognition.

[91]  D. Vaitl,et al.  Utilizing the ventriloquism-effect to investigate audio-visual binding , 2007, Neuropsychologia.

[92]  David B. Pisoni,et al.  Neural processing of asynchronous audiovisual speech perception , 2010, NeuroImage.