MSFR-GCN: A Multi-Scale Feature Reconstruction Graph Convolutional Network for EEG Emotion and Cognition Recognition

Graph Convolutional Network (GCN) excels at EEG recognition by capturing brain connections, but previous studies neglect the important EEG feature itself. In this study, we propose MSFR-GCN, a multi-scale feature reconstruction GCN for recognizing emotion and cognition tasks. Specifically, MSFR-GCN includes the MSFR and feature-pool characteristically, with the MSFR consisting of two sub-modules, multi-scale Squeeze-and-Excitation (MSSE) and multi-scale sample re-weighting (MSSR). MSSE assigns weights to channels and frequency bands based on their separate statistical information, while MSSR assigns sample weights based on combined channel and frequency information. The feature-pool, which pools across the feature dimension, is applied after GCN to retain EEG channel information. The MSFR-GCN achieves excellent results in emotion recognition when first tested on two public datasets, SEED and SEED-IV. Than the MSFR-GCN is tested on our self-collected Emotion and Cognition EEG dataset (ECED) for both emotion and cognition classification tasks. The results show MSFR-GCN’s good performance in emotion and cognition classification tasks and reveal the implicit relationship between the two, which may provide aid in the rehabilitation of people with cognitive impairments from an emotional perspective.

[1]  T. Zhang,et al.  Hierarchical Dynamic Graph Convolutional Network With Interpretability for EEG-Based Emotion Recognition. , 2022, IEEE transactions on neural networks and learning systems.

[2]  Wenming Zheng,et al.  GMSS: Graph-Based Multi-Task Self-Supervised Learning for EEG Emotion Recognition , 2022, IEEE Transactions on Affective Computing.

[3]  Yang Li,et al.  A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition , 2021, IEEE Transactions on Affective Computing.

[4]  Jing Wang,et al.  GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification , 2020, IJCAI.

[5]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[6]  Yuan Zong,et al.  Instance-Adaptive Graph for EEG Emotion Recognition , 2020, AAAI.

[7]  C. L. Philip Chen,et al.  GCB-Net: Graph Convolutional Broad Network and Its Application in Emotion Recognition , 2019, IEEE Transactions on Affective Computing.

[8]  Chunyan Miao,et al.  EEG-Based Emotion Recognition Using Regularized Graph Neural Networks , 2019, IEEE Transactions on Affective Computing.

[9]  Soraia M. Alarcão,et al.  Emotions Recognition Using EEG Signals: A Survey , 2019, IEEE Transactions on Affective Computing.

[10]  Andrzej Cichocki,et al.  EmotionMeter: A Multimodal Framework for Recognizing Human Emotions , 2019, IEEE Transactions on Cybernetics.

[11]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Lina Yao,et al.  Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface , 2017, AAAI.

[13]  Tong Zhang,et al.  Spatial–Temporal Recurrent Neural Network for Emotion Recognition , 2017, IEEE Transactions on Cybernetics.

[14]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[15]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[16]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[17]  Bao-Liang Lu,et al.  Personalizing EEG-Based Affective Models with Transfer Learning , 2016, IJCAI.

[18]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[19]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[20]  Carol Brayne,et al.  Montreal Cognitive Assessment for the diagnosis of Alzheimer's disease and other dementias. , 2015, The Cochrane database of systematic reviews.

[21]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[22]  L. Pessoa,et al.  The neurobiology of emotion–cognition interactions: fundamental questions and strategies for future research , 2015, Front. Hum. Neurosci..

[23]  X. Bornas,et al.  Spontaneous EEG activity and spontaneous emotion regulation. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[24]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[25]  Gregor Leicht,et al.  Emotion regulation by cognitive reappraisal — The role of frontal theta oscillations , 2013, NeuroImage.

[26]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based emotion classification , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[27]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[28]  C. Grady The cognitive neuroscience of ageing , 2012, Nature Reviews Neuroscience.

[29]  Alexandru D. Iordan,et al.  Neural correlates of emotion–cognition interactions: A review of evidence from brain imaging investigations , 2011, Journal of cognitive psychology.

[30]  Rajeevan T. Narayanan,et al.  Patterns of Coupled Theta Activity in Amygdala-Hippocampal-Prefrontal Cortical Circuits during Fear Extinction , 2011, PloS one.

[31]  Bao-Liang Lu,et al.  EEG-based emotion recognition during watching movies , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[32]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[33]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[34]  Gennady G. Knyazev,et al.  Anxiety and oscillatory responses to emotional facial expressions , 2008, Brain Research.

[35]  Jennifer C. Britton,et al.  Neural correlates of social and nonsocial emotions: An fMRI study , 2006, NeuroImage.

[36]  R. Dolan,et al.  Emotion, Cognition, and Behavior , 2002, Science.

[37]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[38]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[39]  E. Halgren,et al.  Generators of the late cognitive potentials in auditory and visual oddball tasks. , 1998, Electroencephalography and clinical neurophysiology.

[40]  J. Morris,et al.  Clinical Dementia Rating: A Reliable and Valid Diagnostic and Staging Measure for Dementia of the Alzheimer Type , 1997, International Psychogeriatrics.

[41]  E. Teng,et al.  The Cognitive Abilities Screening Instrument (CASI): A Practical Test for Cross-Cultural Epidemiological Studies of Dementia , 1994, International Psychogeriatrics.

[42]  B. Hjorth EEG analysis based on time domain properties. , 1970, Electroencephalography and clinical neurophysiology.

[43]  Seetha Hari,et al.  Learning From Imbalanced Data , 2019, Advances in Computer and Electrical Engineering.

[44]  Olga Sourina,et al.  Real-Time Fractal-Based Valence Level Recognition from EEG , 2013, Trans. Comput. Sci..

[45]  Olga Sourina,et al.  Real-Time EEG-Based Emotion Recognition and Its Applications , 2011, Trans. Comput. Sci..