Whole-Brain fMRI Functional Connectivity Signatures Predict Sustained Emotional Experience in Naturalistic Contexts

Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional magnetic resonance imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we use 10-minute movie clips as ecologically valid emotion-evoking procedures in n=52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (400 atlas based parcels). Employing machine-learning based decoding and cross validation procedures allowed to develop predictive FC profiles that can accurately distinguish sustained happiness and sadness and that generalize across movies and subjects. Both functional brain network-based and subnetwork-based emotion prediction results suggest that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results show that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibit a strong contribution to emotion prediction. To further estimate the cumulative effect of naturalistic long-term movie-based video-evoking emotions, we divide the 10-min episode into three stages: early stimulation (1 ~ 200 s), middle stimulation (201 ~ 400 s), and late stimulation (401 ~ 600 s) and examine the emotion prediction performance at different stimulation stages. We found that the late stimulation has a stronger predictive ability (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrates that sustained sadness and happiness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidate the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.

[1]  Jonathan W. Kanen,et al.  Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication , 2023, bioRxiv.

[2]  F. Zhou,et al.  Angiotensin II regulates the neural expression of subjective fear in humans - precision pharmaco-neuroimaging approach , 2022, bioRxiv.

[3]  Philip A. Kragel,et al.  Common and stimulus-type-specific brain representations of negative affect , 2022, Nature Neuroscience.

[4]  E. Rolls,et al.  The human orbitofrontal cortex, vmPFC, and anterior cingulate cortex effective connectome: emotion, memory, and action. , 2022, Cerebral cortex.

[5]  Preston P. Thakral,et al.  Representing the Good and Bad: fMRI signatures during the encoding of multisensory positive, negative, and neutral events , 2022, Cortex.

[6]  M. Sams,et al.  Classification of emotion categories based on functional connectivity patterns of the human brain , 2021, NeuroImage.

[7]  U. Hasson,et al.  Naturalistic imaging: The use of ecologically valid conditions to study brain function , 2021, NeuroImage.

[8]  Pedro A. M. Mediano,et al.  A synergistic core for human brain evolution and cognition , 2020, bioRxiv.

[9]  T. Wager,et al.  A distributed fMRI-based signature for the subjective experience of fear , 2021, Nature Communications.

[10]  D. Brang,et al.  Naturalistic Stimuli: A Paradigm for Multi-Scale Functional Characterization of the Human Brain. , 2021, Current opinion in biomedical engineering.

[11]  N. Sousa,et al.  Habitual coffee drinkers display a distinct pattern of brain functional connectivity , 2021, Molecular Psychiatry.

[12]  K. Kendrick,et al.  Segregating domain-general from emotional context-specific inhibitory control systems - ventral striatum and orbitofrontal cortex serve as emotion-cognition integration hubs , 2020, NeuroImage.

[13]  M. Sams,et al.  Movies and narratives as naturalistic stimuli in neuroimaging , 2020, NeuroImage.

[14]  Emmanuel A Stamatakis,et al.  Combining network topology and information theory to construct representative brain networks , 2020, Network Neuroscience.

[15]  Affect Dynamics , 2021 .

[16]  Luke J. Chang,et al.  Movie viewing elicits rich and reliable brain state dynamics , 2020, Nature Communications.

[17]  K. Kendrick,et al.  Empathic pain evoked by sensory and emotional-communicative cues share common and process-specific neural representations , 2020, eLife.

[18]  R. Ptak,et al.  Discrete Patterns of Cross-Hemispheric Functional Connectivity Underlie Impairments of Spatial Cognition after Stroke , 2020, The Journal of Neuroscience.

[19]  James A. Roberts,et al.  Manipulating the structure of natural scenes using wavelets to study the functional architecture of perceptual hierarchies in the brain , 2020, NeuroImage.

[20]  K. Kendrick,et al.  Oxytocin Differentially Modulates Amygdala Responses during Top‐Down and Bottom‐Up Aversive Anticipation , 2020, Advanced science.

[21]  Timo T. Heikkilä,et al.  Decoding music-evoked emotions in the auditory and motor cortex , 2020, bioRxiv.

[22]  K. Kendrick,et al.  Intrinsic connectivity of the prefrontal cortex and striato-limbic system respectively differentiate Major Depressive from Generalized Anxiety Disorder , 2020, bioRxiv.

[23]  J. Hajnal,et al.  Emerging functional connectivity differences in newborn infants vulnerable to autism spectrum disorders , 2020, Translational Psychiatry.

[24]  M. Milham,et al.  Towards clinical applications of movie fMRI , 2020, NeuroImage.

[25]  Christine E. Weber,et al.  An fMRI Study of Affective Congruence across Visual and Auditory Modalities , 2020, Journal of Cognitive Neuroscience.

[26]  Ale Smidts,et al.  Decoding dynamic affective responses to naturalistic videos with shared neural patterns , 2020, NeuroImage.

[27]  S. Sturgeon The Bayesian Model , 2020 .

[28]  Daniel S. Margulies,et al.  Topographic organization of the human subcortex unveiled with functional connectivity gradients , 2020, Nature Neuroscience.

[29]  Erez Simony,et al.  Analysis of stimulus-induced brain dynamics during naturalistic paradigms , 2019, NeuroImage.

[30]  X. Di,et al.  Intersubject consistent dynamic connectivity during natural vision revealed by functional MRI , 2019, NeuroImage.

[31]  J. Poline,et al.  Nature abhors a paywall: How open science can realize the potential of naturalistic stimuli , 2019, NeuroImage.

[32]  Alan S. Cowen,et al.  The Neural Representation of Visually Evoked Emotion Is High-Dimensional, Categorical, and Distributed across Transmodal Brain Regions , 2019, bioRxiv.

[33]  Ajay B. Satpute,et al.  The Default Mode Network’s Role in Discrete Emotion , 2019, Trends in Cognitive Sciences.

[34]  Amy M. Belfi,et al.  The default-mode network represents aesthetic appeal that generalizes across visual domains , 2019, Proceedings of the National Academy of Sciences.

[35]  Richard F. Betzel,et al.  Temporal fluctuations in the brain's modular architecture during movie-watching , 2019, NeuroImage.

[36]  Michael Breakspear,et al.  Naturalistic Stimuli in Neuroscience: Critically Acclaimed , 2019, Trends in Cognitive Sciences.

[37]  Ling Zou,et al.  The Cortical Network of Emotion Regulation: Insights From Advanced EEG-fMRI Integration Analysis , 2019, IEEE Transactions on Medical Imaging.

[38]  Catherine Perrodin,et al.  Are We Ready for Real-world Neuroscience? , 2019, Journal of Cognitive Neuroscience.

[39]  Tor D. Wager,et al.  Emotion schemas are embedded in the human visual system , 2018, Science Advances.

[40]  Emiliano Ricciardi,et al.  Emotionotopy in the human right temporo-parietal cortex , 2019, Nature Communications.

[41]  Matthieu Gilson,et al.  Distinct modes of functional connectivity induced by movie-watching , 2019, NeuroImage.

[42]  Marianne C. Reddan,et al.  Attenuating Neural Threat Expression with Imagination , 2018, Neuron.

[43]  M. Breakspear,et al.  Effective connectivity of the anterior hippocampus predicts recollection confidence during natural memory retrieval , 2018, Nature Communications.

[44]  C. Kilts,et al.  Common Functional Brain States Encode both Perceived Emotion and the Psychophysiological Response to Affective Stimuli , 2018, Scientific Reports.

[45]  Simon B Eickhoff,et al.  Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional MRI results , 2018, Network Neuroscience.

[46]  K. Kendrick,et al.  Shifted balance of dorsal versus ventral striatal communication with frontal reward and regulatory regions in cannabis‐dependent males , 2018, Human brain mapping.

[47]  Maurizio Corbetta,et al.  A New Modular Brain Organization of the BOLD Signal during Natural Vision , 2018, Cerebral cortex.

[48]  Vince D. Calhoun,et al.  Classification and Prediction of Brain Disorders Using Functional Connectivity: Promising but Challenging , 2018, Front. Neurosci..

[49]  Nathan M. Petro,et al.  Individual differences in valence bias: fMRI evidence of the initial negativity hypothesis , 2018, Social cognitive and affective neuroscience.

[50]  M. Sams,et al.  Distributed affective space represents multiple emotion categories across the human brain , 2018, Social cognitive and affective neuroscience.

[51]  Anthony R. McIntosh,et al.  Unique Mapping of Structural and Functional Connectivity on Cognition , 2018, The Journal of Neuroscience.

[52]  Luiz Pessoa,et al.  Understanding emotion with brain networks , 2018, Current Opinion in Behavioral Sciences.

[53]  Matthieu Gilson,et al.  Effective connectivity inferred from fMRI transition dynamics during movie viewing points to a balanced reconfiguration of cortical interactions , 2017, NeuroImage.

[54]  B. Weber,et al.  Altered orbitofrontal activity and dorsal striatal connectivity during emotion processing in dependent marijuana users after 28 days of abstinence , 2018, Psychopharmacology.

[55]  Yuanqing Li,et al.  An MVPA method based on sparse representation for pattern localization in fMRI data analysis , 2017, Neurocomputing.

[56]  David K. Menon,et al.  Default mode contributions to automated information processing , 2017, Proceedings of the National Academy of Sciences.

[57]  Alan S. Cowen,et al.  Self-report captures 27 distinct categories of emotion bridged by continuous gradients , 2017, Proceedings of the National Academy of Sciences.

[58]  Evan M. Gordon,et al.  Local-Global Parcellation of the Human Cerebral Cortex From Intrinsic Functional Connectivity MRI , 2017, bioRxiv.

[59]  L. Pessoa A Network Model of the Emotional Brain , 2017, Trends in Cognitive Sciences.

[60]  Peter Kuppens,et al.  The neural basis of emotions varies over time: different regions go with onset- and offset-bound processes underlying emotion intensity , 2017, Social cognitive and affective neuroscience.

[61]  Xintao Hu,et al.  Test-retest reliability of functional connectivity networks during naturalistic fMRI paradigms , 2016, bioRxiv.

[62]  R. Cameron Craddock,et al.  Individual differences in functional connectivity during naturalistic viewing conditions , 2016, NeuroImage.

[63]  Alzheimer's Disease Neuroimaging Initiative,et al.  Altered functional brain networks in amnestic mild cognitive impairment: a resting-state fMRI study , 2016, Brain Imaging and Behavior.

[64]  Jessica R. Cohen,et al.  The Segregation and Integration of Distinct Brain Networks and Their Relationship to Cognition , 2016, The Journal of Neuroscience.

[65]  Yufeng Zang,et al.  DPABI: Data Processing & Analysis for (Resting-State) Brain Imaging , 2016, Neuroinformatics.

[66]  R. Hari,et al.  Discrete Neural Signatures of Basic Emotions. , 2016, Cerebral cortex.

[67]  Philip A. Kragel,et al.  Multivariate neural biomarkers of emotional states are categorically distinct. , 2015, Social cognitive and affective neuroscience.

[68]  N. Jaworska,et al.  A review of fMRI studies during visual emotive processing in major depressive disorder , 2015, The world journal of biological psychiatry : the official journal of the World Federation of Societies of Biological Psychiatry.

[69]  J. P. Hamilton,et al.  Depressive Rumination, the Default-Mode Network, and the Dark Matter of Clinical Neuroscience , 2015, Biological Psychiatry.

[70]  Karim Jerbi,et al.  Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy , 2015, Journal of Neuroscience Methods.

[71]  M. Raichle The brain's default mode network. , 2015, Annual review of neuroscience.

[72]  Thomas E. Nichols,et al.  A Bayesian Model of Category-Specific Emotional Brain Responses , 2015, PLoS Comput. Biol..

[73]  Mikko Sams,et al.  Emotional speech synchronizes brains across listeners and engages large-scale dynamic brain networks , 2014, NeuroImage.

[74]  Moors Pieter,et al.  Test-retest reliability. , 2014 .

[75]  Rebecca Saxe,et al.  Contributions of episodic retrieval and mentalizing to autobiographical thought: Evidence from functional neuroimaging, resting-state connectivity, and fMRI meta-analyses , 2014, NeuroImage.

[76]  Dewen Hu,et al.  Unsupervised classification of major depression using functional connectivity MRI , 2014, Human brain mapping.

[77]  Kevin S. LaBar,et al.  Advancing Emotion Theory with Multivariate Pattern Classification , 2014, Emotion review : journal of the International Society for Research on Emotion.

[78]  G. Glover,et al.  Causal interactions between fronto-parietal central executive and default-mode networks in humans , 2013, Proceedings of the National Academy of Sciences.

[79]  Viviana Betti,et al.  Natural Scenes Viewing Alters the Dynamics of Functional Connectivity in the Human Brain , 2013, Neuron.

[80]  M. Maroun,et al.  Medial Prefrontal Cortex , 2013, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[81]  M. Just,et al.  Identifying Emotions on the Basis of Neural Activation , 2013, PloS one.

[82]  Florin Dolcos,et al.  Neural Correlates of Opposing Effects of Emotional Distraction on Working Memory and Episodic Memory: An Event-Related fMRI Investigation , 2013, Front. Psychol..

[83]  Antonio Rangel,et al.  Stimulus Value Signals in Ventromedial PFC Reflect the Integration of Attribute Value Signals Computed in Fusiform Gyrus and Posterior Superior Temporal Gyrus , 2013, The Journal of Neuroscience.

[84]  Steffen Katzner,et al.  Visual cortical networks: of mice and men , 2013, Current Opinion in Neurobiology.

[85]  A. Damasio,et al.  The nature of feelings: evolutionary and neurobiological origins , 2013, Nature Reviews Neuroscience.

[86]  Kristen A. Lindquist,et al.  A functional architecture of the human brain: emerging insights from the science of emotion , 2012, Trends in Cognitive Sciences.

[87]  Christian E. Waugh,et al.  Timing: A missing key ingredient in typical fMRI studies of emotion , 2012, Behavioral and Brain Sciences.

[88]  Kristen A. Lindquist,et al.  The brain basis of emotion: A meta-analytic review , 2012, Behavioral and Brain Sciences.

[89]  D. Hu,et al.  Identifying major depression using whole-brain functional connectivity: a multivariate pattern analysis. , 2012, Brain : a journal of neurology.

[90]  Feng Liu,et al.  Alterations of the amplitude of low-frequency fluctuations in treatment-resistant and treatment-response depression: A resting-state fMRI study , 2012, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[91]  J. Ford,et al.  Default mode network activity and connectivity in psychopathology. , 2012, Annual review of clinical psychology.

[92]  Jing Wang,et al.  Decoding the neural representation of affective states , 2012, NeuroImage.

[93]  Ilya M. Veer,et al.  Beyond acute social stress: Increased functional connectivity between amygdala and cortical midline structures , 2011, NeuroImage.

[94]  S. Hamann,et al.  Neuroimaging Support for Discrete Neural Correlates of Basic Emotions: A Voxel-based Meta-analysis , 2010, Journal of Cognitive Neuroscience.

[95]  J. Yiend The effects of emotion on attention: A review of attentional processing of emotional information , 2010 .

[96]  劉 健勤,et al.  複雑系に見たDefault Mode Network , 2009, ICS 2009.

[97]  F. Van Overwalle Social cognition and the brain: A meta‐analysis , 2009, Human brain mapping.

[98]  R. Nathan Spreng,et al.  The Common Neural Basis of Autobiographical Memory, Prospection, Navigation, Theory of Mind, and the Default Mode: A Quantitative Meta-analysis , 2009, Journal of Cognitive Neuroscience.

[99]  J. Andrews-Hanna,et al.  The brain's default network: Anatomy, function, and consequence of disruption , 2009 .

[100]  A. Craig,et al.  How do you feel — now? The anterior insula and human awareness , 2009, Nature Reviews Neuroscience.

[101]  Lisa Feldman Barrett,et al.  Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies , 2008, NeuroImage.

[102]  E. Kensinger,et al.  Emotional valence influences the neural correlates associated with remembering and knowing , 2008, Cognitive, affective & behavioral neuroscience.

[103]  D. Schacter,et al.  The cognitive neuroscience of constructive memory: remembering the past and imagining the future , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[104]  O. Pollatos,et al.  Neural systems connecting interoceptive awareness and feelings , 2007, Human brain mapping.

[105]  Janaina Mourão Miranda,et al.  Contributions of stimulus valence and arousal to visual activation during emotional perception , 2003, NeuroImage.

[106]  K. Luan Phan,et al.  Functional Neuroanatomy of Emotion: A Meta-Analysis of Emotion Activation Studies in PET and fMRI , 2002, NeuroImage.

[107]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[108]  G. Shulman,et al.  Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function , 2001, Proceedings of the National Academy of Sciences of the United States of America.