Subject-Independent EEG-based Emotion Recognition using Adversarial Learning

Electroencephalography (EEG) based emotion recognition studies have been conducted in recent years. Most prior researches are based on subject-dependent models since EEG signals have a large variation between individuals. In this paper, we propose a novel EEG-based emotion recognition approach that addresses the challenging issue of the subject-dependency. To solve the problem, we design a multi-task deep neural network, which consists of two objectives. The first one is to classify subject-independent emotional labels, and the second is to make the model cannot distinguish the subject labels. To achieve the latter purpose, we adversarially learn the proposed model, which has three components: 1) Emotion classification module, 2) Subject classification module, 3) Adversarial module. To make the model confuse the subject labels, we apply the randomization function to the subject classification module for adversarial learning. For the experiment, we evaluate the proposed method to classify EEG emotional labels with a leave-one-subject-out scheme on SEED dataset, which has recorded EEG from 15 participants and contains three emotional labels: positive, negative, and neutral. We compare the proposed method with a single-task deep neural network and multi-task model that classify emotional labels with subject labels. Our experimental results show that the proposed method achieves better results than the others with an average accuracy of 75.31%. Moreover, the standard deviation of our model was 7.33%, which is the lowest with the compared models.

[1]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[2]  M. Thulasidas,et al.  Robust classification of EEG signal for brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Yang Li,et al.  A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition , 2021, IEEE Transactions on Affective Computing.

[4]  J. X. Chen,et al.  A Hierarchical Bidirectional GRU Model With Attention for EEG-Based Emotion Classification , 2019, IEEE Access.

[5]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[6]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[7]  Lina Yao,et al.  Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-based Intention Recognition for Brain Computer Interface , 2017, AAAI.

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Ji-Hoon Jeong,et al.  Recurrent convolutional neural network model based on temporal and spatial feature for motor imagery classification , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[10]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[11]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based emotion classification , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[13]  Liang Dong,et al.  Emotion Recognition from Multiband EEG Signals Using CapsNet , 2019, Sensors.

[14]  Lina Yao,et al.  A Convolutional Recurrent Attention Model for Subject-Independent EEG Signal Analysis , 2019, IEEE Signal Processing Letters.

[15]  Kyandoghere Kyamakya,et al.  A Deep-Learning Model for Subject-Independent Human Emotion Recognition Using Electrodermal Activity Sensors , 2019, Sensors.

[16]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[17]  Florian Yger,et al.  Towards Adaptive Classification using Riemannian Geometry approaches in Brain-Computer Interfaces , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).

[18]  Jun-Wen Tan,et al.  Similarities and differences of emotions in human–machine and human–human interactions: what kind of emotions are relevant for future companion systems? , 2014, Ergonomics.

[19]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[20]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[21]  Masayuki Numao,et al.  Subject-Independent Emotion Recognition During Music Listening Based on EEG Using Deep Convolutional Neural Networks , 2019, 2019 IEEE 15th International Colloquium on Signal Processing & Its Applications (CSPA).

[22]  H. Guesgen,et al.  Motor Imagery Classification Based on Subject to Subject Transfer in Riemannian Manifold , 2019, 2019 7th International Winter Conference on Brain-Computer Interface (BCI).