Temporal-Order Judgment of Audiovisual Events Involves Network Activity Between Parietal and Prefrontal Cortices

Our perception of the temporal order of everyday external events depends on the integrated sensory information in the brain. Our understanding of the brain mechanism for temporal-order judgment (TOJ) of unisensory events, particularly in the visual domain, is advanced. In case of multisensory events, however, there are unanswered questions. Here, by using physically synchronous and asynchronous auditory-visual events in functional magnetic resonance imaging (fMRI) experiments, we identified the brain network that is associated with the perception of the temporal order of multisensory events. The activation in the right temporo-parietal junction was modulated by the perception of asynchronous audiovisual events. During this perception of temporal order, the right dorsolateral prefrontal cortex coordinated activity with the right temporo-parietal and the left inferior parietal cortices. These results suggest that the TOJ in the multisensory domain underlies a network activity between parietal and prefrontal cortices unlike the regional activity in the right temporo-parietal junction in the unisensory visual domain.

[1]  Mingzhou Ding,et al.  Estimating Granger causality from fourier and wavelet transforms of time series data. , 2007, Physical review letters.

[2]  E. Pöppel,et al.  A hierarchical model of temporal perception , 1997, Trends in Cognitive Sciences.

[3]  K. Zilles,et al.  Difficulty of perceptual spatiotemporal integration modulates the neural activity of left inferior parietal cortex , 2005, Neuroscience.

[4]  M. Shadlen,et al.  Representation of Time by Neurons in the Posterior Parietal Cortex of the Macaque , 2003, Neuron.

[5]  Gereon R Fink,et al.  Left inferior parietal cortex integrates time and space during collision judgments , 2003, NeuroImage.

[6]  K. Zilles,et al.  Neural mechanisms associated with attention to temporal synchrony versus spatial orientation: an fMRI study , 2003, NeuroImage.

[7]  R. Miall,et al.  Distinct systems for automatic and cognitively controlled time measurement: evidence from neuroimaging , 2003, Current Opinion in Neurobiology.

[8]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[9]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[10]  R. Miall,et al.  Remembering the time: a continuous clock , 2006, Trends in Cognitive Sciences.

[11]  Hidenao Fukuyama,et al.  Multisensory convergence at human temporo-parietal junction – epicortical recording of evoked responses , 2004, Clinical Neurophysiology.

[12]  S. Bressler,et al.  Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[14]  Viktor K. Jirsa,et al.  Noise during Rest Enables the Exploration of the Brain's Dynamic Repertoire , 2008, PLoS Comput. Biol..

[15]  E. Macaluso,et al.  Multisensory spatial interactions: a window onto functional integration in the human brain , 2005, Trends in Neurosciences.

[16]  M. Husain,et al.  Control of Visuotemporal Attention by Inferior Parietal and Superior Temporal Cortex , 2002, Current Biology.

[17]  Scott T. Grafton,et al.  Motor task difficulty and brain activity: investigation of goal-directed reciprocal aiming using positron emission tomography. , 1997, Journal of neurophysiology.

[18]  Egon Wanke,et al.  Mapping brains without coordinates , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[19]  P. Cavanagh,et al.  Opinion TRENDS in Cognitive Sciences Vol.11 No.5 The ‘when ’ pathway of the right parietal lobe , 2022 .

[20]  P. Cavanagh,et al.  Bilateral deficits of transient visual attention in right parietal patients. , 2003, Brain : a journal of neurology.

[21]  Yasuyoshi Watanabe,et al.  Cortical Networks Recruited for Time Perception: A Monkey Positron Emission Tomography (PET) Study , 2001, NeuroImage.

[22]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[23]  R. Knight,et al.  Cortical Networks Underlying Mechanisms of Time Perception , 1998, The Journal of Neuroscience.

[24]  Janice J. Snyder,et al.  Spatial-temporal anisometries following right parietal damage , 2004, Neuropsychologia.

[25]  Joy Hirsch,et al.  Functional Specialization within the Medial Frontal Gyrus for Perceptual Go/No-Go Decisions Based on What, When, and Where Related Information: An fMRI Study , 2005, Journal of Cognitive Neuroscience.

[26]  Ki-Hyun Kim,et al.  The role of the right posterior parietal cortex in temporal order judgment , 2009, Brain and Cognition.

[27]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[28]  G. Baylis,et al.  Visual extinction with double simultaneous stimulation: what is simultaneous? , 2002, Neuropsychologia.

[29]  James K. Kroger,et al.  Cross-modal and cross-temporal association in neurons of frontal cortex , 2000, Nature.

[30]  Chris Rorden,et al.  Temporal Order Processing of Syllables in the Left Parietal Lobe , 2009, The Journal of Neuroscience.

[31]  N. C. Silver,et al.  Averaging Correlation Coefficients: Should Fishers z Transformation Be Used? , 1987 .

[32]  D. Eagleman.  Human time perception and its illusions , 2008, Current Opinion in Neurobiology.

[33]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[34]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[35]  D. Buonomano,et al.  The neural basis of temporal processing. , 2004, Annual review of neuroscience.

[36]  R. Sparing,et al.  Hemiextinction induced by transcranial magnetic stimulation over the right temporo-parietal junction , 2006, Neuroscience.

[37]  Lucas Spierer,et al.  Plastic brain mechanisms for attaining auditory temporal order judgment proficiency , 2010, NeuroImage.

[38]  S. Kitazawa,et al.  Neural correlates of tactile temporal-order judgment in humans: an fMRI study. , 2013, Cerebral cortex.

[39]  Nicholas J. Cox,et al.  Speaking Stata: Correlation with Confidence, or Fisher's z revisited , 2008 .

[40]  R. Blair,et al.  An alternative method for significance testing of waveform difference potentials. , 1993, Psychophysiology.

[41]  Lucas Spierer,et al.  Interhemispheric coupling between the posterior sylvian regions impacts successful auditory temporal order judgment , 2010, Neuropsychologia.

[42]  S. Iversen,et al.  Detection of Audio-Visual Integration Sites in Humans by Application of Electrophysiological Criteria to the BOLD Effect , 2001, NeuroImage.

[43]  S. Yantis,et al.  Transient neural activity in human parietal cortex during spatial attention shifts , 2002, Nature Neuroscience.

[44]  Patrik Vuilleumier,et al.  Effects of emotional prosody on auditory extinction for voices in patients with spatial neglect , 2008, Neuropsychologia.

[45]  G. Calvert,et al.  Multisensory integration: methodological approaches and emerging principles in the human brain , 2004, Journal of Physiology-Paris.

[46]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[47]  C. Caltagirone,et al.  Selective deficit of time perception in a patient with right prefrontal cortex lesion , 2002, Neurology.

[48]  G. Rizzolatti,et al.  The Cortical Motor System , 2001, Neuron.

[49]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[50]  Katya Rubia,et al.  The neural correlates of cognitive time management: a review. , 2004, Acta neurobiologiae experimentalis.

[51]  Marc Wittmann,et al.  Effects of brain-lesion size and location on temporal-order judgment , 2004, Neuroreport.

[52]  Viktor K. Jirsa,et al.  Multisensory integration for timing engages different brain networks , 2007, NeuroImage.

[53]  John Christie,et al.  Temporal Order Judgments Activate Temporal Parietal Junction , 2009, The Journal of Neuroscience.

[54]  R. Klatzky,et al.  Cognitive representations of hand posture in ideomotor apraxia , 2003, Neuropsychologia.

[55]  J. Ashburner,et al.  Nonlinear spatial normalization using basis functions , 1999, Human brain mapping.

[56]  John C. Rothwell,et al.  The right dorsolateral prefrontal cortex is essential in time reproduction: an investigation with repetitive transcranial magnetic stimulation , 2004, Experimental Brain Research.

[57]  Mingzhou Ding,et al.  Analyzing information flow in brain networks with nonparametric Granger causality , 2008, NeuroImage.

[58]  M. Young,et al.  Computational analysis of functional connectivity between areas of primate cerebral cortex. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[59]  J. Mattingley,et al.  Visual extinction and prior entry: Impaired perception of temporal order with intact motion perception after unilateral parietal damage , 1997, Neuropsychologia.