Neural mechanisms for integrating consecutive and interleaved natural events

To understand temporally extended events, the human brain needs to accumulate information continuously across time. Interruptions that require switching of attention to other event sequences disrupt this process. To reveal neural mechanisms supporting integration of event information, we measured brain activity with functional magnetic resonance imaging (fMRI) from 18 participants while they viewed 6.5‐minute excerpts from three movies (i) consecutively and (ii) as interleaved segments of approximately 50‐s in duration. We measured inter‐subject reliability of brain activity by calculating inter‐subject correlations (ISC) of fMRI signals and analyzed activation timecourses with a general linear model (GLM). Interleaving decreased the ISC in posterior temporal lobes, medial prefrontal cortex, superior precuneus, medial occipital cortex, and cerebellum. In the GLM analyses, posterior temporal lobes were activated more consistently by instances of speech when the movies were viewed consecutively than as interleaved segments. By contrast, low‐level auditory and visual stimulus features and editing boundaries caused similar activity patterns in both conditions. In the medial occipital cortex, decreases in ISC were seen in short bursts throughout the movie clips. By contrast, the other areas showed longer‐lasting differences in ISC during isolated scenes depicting socially‐relevant and suspenseful content, such as deception or inter‐subject conflict. The areas in the posterior temporal lobes also showed sustained activity during continuous actions and were deactivated when actions ended at scene boundaries. Our results suggest that the posterior temporal and dorsomedial prefrontal cortices, as well as the cerebellum and dorsal precuneus, support integration of events into coherent event sequences. Hum Brain Mapp 38:3360–3376, 2017. © 2017 Wiley Periodicals, Inc.

[1]  Jeffrey M. Zacks,et al.  Prediction Error Associated with the Perceptual Segmentation of Naturalistic Events , 2011, Journal of Cognitive Neuroscience.

[2]  A. Calder,et al.  Neural mechanisms of social attention , 2009, Trends in Cognitive Sciences.

[3]  Frank E. Pollick,et al.  Differences in fMRI intersubject correlation while viewing unedited and edited videos of dance performance , 2015, Cortex.

[4]  Mikko Sams,et al.  Naturalistic fMRI Mapping Reveals Superior Temporal Sulcus as the Hub for the Distributed Brain Network for Social Perception , 2012, Front. Hum. Neurosci..

[5]  Torgeir Moberget,et al.  Annals of the New York Academy of Sciences Cerebellar Contributions to Motor Control and Language Comprehension: Searching for Common Computational Principles , 2022 .

[6]  Jussi Tohka,et al.  A versatile software package for inter-subject correlation based analyses of fMRI , 2014, Front. Neuroinform..

[7]  Mikko Sams,et al.  Emotional speech synchronizes brains across listeners and engages large-scale dynamic brain networks , 2014, NeuroImage.

[8]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[9]  Mikko Sams,et al.  Synchronous brain activity across individuals underlies shared psychological perspectives , 2014, NeuroImage.

[10]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[11]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[12]  L. Davachi,et al.  Enhanced Intersubject Correlations during Movie Viewing Correlate with Successful Episodic Encoding , 2008, Neuron.

[13]  Jouko Lampinen,et al.  Stimulus-Related Independent Component and Voxel-Wise Analysis of Human Brain Activity during Free Viewing of a Feature Film , 2012, PloS one.

[14]  R. Hari,et al.  Emotions promote social interaction by synchronizing brain activity across individuals , 2012, Proceedings of the National Academy of Sciences.

[15]  David C. Van Essen,et al.  Application of Information Technology: An Integrated Software Suite for Surface-based Analyses of Cerebral Cortex , 2001, J. Am. Medical Informatics Assoc..

[16]  Jeffrey M. Zacks,et al.  The Brain's Cutting-Room Floor: Segmentation of Narrative Cinema , 2010, Front. Hum. Neurosci..

[17]  Mikko Sams,et al.  Fronto-parietal network supports context-dependent speech comprehension , 2014, Neuropsychologia.

[18]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[19]  Joseph P. Magliano,et al.  The Impact of Continuity Editing in Narrative Film on Event Segmentation , 2011, Cogn. Sci..

[20]  Klaus Scheffler,et al.  Temporal integration of sequential auditory events: silent period in sound pattern activates human planum temporale , 2003, NeuroImage.

[21]  Mark W. Woolrich,et al.  Bayesian analysis of neuroimaging data in FSL , 2009, NeuroImage.

[22]  R. Malach,et al.  Intersubject Synchronization of Cortical Activity During Natural Vision , 2004, Science.

[23]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[24]  P. Vuilleumier,et al.  How brains beware: neural mechanisms of emotional attention , 2005, Trends in Cognitive Sciences.

[25]  S. Koelsch,et al.  Toward a general psychological model of tension and suspense , 2015, Front. Psychol..

[26]  Frank Van Overwalle,et al.  Social cognition and the cerebellum: A meta-analysis of over 350 fMRI studies , 2014, NeuroImage.

[27]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[28]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[29]  Jeffrey M. Zacks,et al.  Segmentation in the perception and memory of events , 2008, Trends in Cognitive Sciences.

[30]  Alexander G. Huth,et al.  Attention During Natural Vision Warps Semantic Representation Across the Human Brain , 2013, Nature Neuroscience.

[31]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[32]  T. Paus Location and function of the human frontal eye-field: A selective review , 1996, Neuropsychologia.

[33]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[34]  Scott L. Fairhall,et al.  Temporal Integration Windows for Naturalistic Visual Sequences , 2014, PloS one.

[35]  Daniel P. Kennedy,et al.  A specific hypoactivation of right temporo-parietal junction/posterior superior temporal sulcus in response to socially awkward situations in autism. , 2015, Social cognitive and affective neuroscience.

[36]  Kwang-Il Goh,et al.  Burstiness and memory in complex systems , 2006 .

[37]  Jeffrey M. Zacks,et al.  Action perception predicts action performance , 2013, Neuropsychologia.

[38]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[39]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.