A 3D Convolutional Neural Network for Emotion Recognition based on EEG Signals

As an important field of research in Human-Machine Interactions, emotion recognition based on the electroencephalography (EEG) signals has become common research. The traditional machine learning approaches use well-designed classifiers with hand-crafted features which may be limited to domain knowledge. Motivated by the outstanding performance of deep learning approaches in recognition tasks, we proposed a 3D convolutional neural network model to extract the spatial-temporal features automatically in the EEG signals. By the pre-processing method with baseline signals and the electrode topological structure relocated, the proposed model achieves a high accuracy rate of 96.61%, 96.43% in the Two class classification task (low/high arousal, low/high valence) and 93.53% in the Four class classification task (low arousal and low valence/high arousal and low valence/low arousal and high valence/high arousal and high valence) in the DEAP dataset, and 97.52%, 96.96% in the Two class classification task and 95.86% in the Four class classification task in the AMIGOS dataset.

[1]  Enas Abdulhay,et al.  Using Deep Convolutional Neural Network for Emotion Detection on a Physiological Signals Dataset (AMIGOS) , 2019, IEEE Access.

[2]  Nadia Mammone,et al.  A deep CNN approach to decode motor preparation of upper limbs from time-frequency maps of EEG signals at source level , 2020, Neural Networks.

[3]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[4]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[5]  Mufti Mahmud,et al.  Deep Learning in Mining Biological Data , 2020, Cognitive Computation.

[6]  Yike Guo,et al.  Multiple Feature Fusion for Automatic Emotion Recognition Using EEG Signals , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[8]  Ming Qiu,et al.  Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[9]  Ioannis Patras,et al.  AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups , 2017, IEEE Transactions on Affective Computing.

[10]  Kyungha Min,et al.  A Multi-Column CNN Model for Emotion Recognition from EEG Signals , 2019, Sensors.

[11]  Reda A. El-Khoribi,et al.  EEG-Based Emotion Recognition using 3D Convolutional Neural Networks , 2018 .

[12]  Yang Li,et al.  Boosted Convolutional Neural Networks for Motor Imagery EEG Decoding with Multiwavelet-based Time-Frequency Conditional Granger Causality Analysis , 2018, ArXiv.

[13]  Francesco Carlo Morabito,et al.  A Convolutional Neural Network approach for classification of dementia stages based on 2D-spectral representation of EEG recordings , 2019, Neurocomputing.

[14]  Shin-Dug Kim,et al.  Electroencephalography Based Fusion Two-Dimensional (2D)-Convolution Neural Networks (CNN) Model for Emotion Recognition System , 2018, Sensors.

[15]  Charalampos Bratsas,et al.  Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli , 2010, IEEE Transactions on Information Technology in Biomedicine.

[16]  Leo Galway,et al.  Feature Extraction for Emotion Recognition and Modelling Using Neurophysiological Data , 2016, 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS).

[17]  Soraia M. Alarcão,et al.  Emotions Recognition Using EEG Signals: A Survey , 2019, IEEE Transactions on Affective Computing.

[18]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[19]  Xiangmin Xu,et al.  EEG-based emotion classification using convolutional neural network , 2017, 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[20]  Ioannis Patras,et al.  A Multi-Task Cascaded Network for Prediction of Affect, Personality, Mood and Social Context Using EEG Signals , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[21]  Seyed Kamaledin Setarehdan,et al.  Emotion recognition through EEG phase space dynamics and Dempster-Shafer theory. , 2019, Medical hypotheses.

[22]  Ram Bilas Pachori,et al.  Cross-Subject Emotion Recognition Using Flexible Analytic Wavelet Transform From EEG Signals , 2019, IEEE Sensors Journal.

[23]  Chao Li,et al.  Exploring temporal representations by leveraging attention-based bidirectional LSTM-RNNs for multi-modal emotion recognition , 2020, Inf. Process. Manag..

[24]  Brendan McCane,et al.  EmotioNet: A 3-D Convolutional Neural Network for EEG-based Emotion Recognition , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[25]  Ali Motie Nasrabadi,et al.  A novel EEG-based approach to classify emotions through phase space dynamics , 2019, Signal Image Video Process..