Exploring EEG microstates for affective computing: decoding valence and arousal experiences during video watching*

Investigating the electroencephalography (EEG) correlates of human emotional experiences has attracted increasing interest in the field of affective computing. Substantial progress has been made during the past decades, mainly by using EEG features extracted from localized brain activities. The present study explored a brain network-based feature defined by EEG microstates for a possible representation of emotional experiences. A publicly available and widely used benchmarking EEG dataset called DEAP was used, in which 32 participants watched 40 one-minute music videos with their 32channel EEG recorded. Four quasi-stable prototypical microstates were obtained, and their temporal parameters were extracted as features. In random forest regression, the microstate features showed better performances for decoding valence (model fitting mean squared error (MSE) = 3.85±0.28 and 4.07 ± 0.30, respectively, p = 0.022) and comparable performances for decoding arousal (MSE = 3.30±0.30 and 3.41 ±0.31, respectively, p = 0.169), as compared to conventional spectral power features. As microstate features describe neural activities from a global spatiotemporal dynamical perspective, our findings demonstrate a possible new mechanism for understanding human emotion and provide a promising type of EEG feature for affective computing.

[1]  Ying Zeng,et al.  EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations , 2019, IEEE Transactions on Biomedical Engineering.

[2]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[3]  M. Paulus,et al.  EEG Microstates Temporal Dynamics Differentiate Individuals with Mood and Anxiety Disorders From Healthy Subjects , 2019, Front. Hum. Neurosci..

[4]  Thomas Dierks,et al.  Discovering EEG resting state alterations of semantic dementia , 2016, Clinical Neurophysiology.

[5]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[6]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[7]  Emiliano Santarnecchi,et al.  EEG Microstate Correlates of Fluid Intelligence and Response to Cognitive Training , 2017, Brain Topography.

[8]  John J. B. Allen,et al.  Frontal EEG alpha asymmetry and emotion: From neural underpinnings and methodological considerations to psychopathology and social cognition. , 2018, Psychophysiology.

[9]  Dimitri Van De Ville,et al.  BOLD correlates of EEG topography reveal rapid resting-state network dynamics , 2010, NeuroImage.

[10]  D Lehmann,et al.  EEG alpha map series: brain micro-states by space-oriented adaptive segmentation. , 1987, Electroencephalography and clinical neurophysiology.

[11]  Thomas Koenig,et al.  EEG microstates as a tool for studying the temporal dynamics of whole-brain neuronal networks: A review , 2017, NeuroImage.

[12]  Yu Zhang,et al.  An electroencephalographic signature predicts antidepressant response in major depression , 2019, Nature Biotechnology.

[13]  Dimitri Van De Ville,et al.  Electroencephalographic Resting-State Networks: Source Localization of Microstates , 2017, Brain Connect..

[14]  Yue Ding,et al.  Inter-Brain EEG Feature Extraction and Analysis for Continuous Implicit Emotion Tagging During Video Watching , 2021, IEEE Transactions on Affective Computing.

[15]  Dan Zhang,et al.  Novel semi-dry electrodes for brain–computer interface applications , 2016, Journal of neural engineering.

[16]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[17]  S. Hsieh,et al.  Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns , 2014, PloS one.

[18]  L. Pessoa A Network Model of the Emotional Brain , 2017, Trends in Cognitive Sciences.

[19]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[20]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[21]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[22]  Viola Borchardt,et al.  Cognitive Processing Involves Dynamic Reorganization of the Whole-Brain Network's Functional Community Structure , 2016, The Journal of Neuroscience.

[23]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[24]  Dietrich Lehmann,et al.  The functional significance of EEG microstates—Associations with modalities of thinking , 2016, NeuroImage.

[25]  R. Buckner,et al.  The brain’s default network: updated anatomy, physiology and evolving insights , 2019, Nature Reviews Neuroscience.

[26]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[27]  Christoph M. Michel,et al.  Electrical neuroimaging reveals early generator modulation to emotional words , 2004, NeuroImage.

[28]  Fei Wang,et al.  EEG Correlates of Ten Positive Emotions , 2017, Front. Hum. Neurosci..

[29]  Patricia Milz Keypy – An Open Source Library For EEG Microstate Analysis , 2016, European Psychiatry.