Continuous Convolutional Neural Network with 3D Input for EEG-Based Emotion Recognition

Automatic emotion recognition based on EEG is an important issue in Brain-Computer Interface (BCI) applications. In this paper, baseline signals were taken into account to improve recognition accuracy. Multi-Layer Perceptron (MLP), Decision Tree (DT) and our proposed approach were adopted to verify the effectiveness of baseline signals on classification results. Besides, a 3D representation of EEG segment was proposed to combine features of signals from different frequency bands while preserving spatial information among channels. The continuous convolutional neural network takes the constructed 3D EEG cube as input and makes prediction. Extensive experiments on public DEAP dataset indicate that the proposed method is well suited for emotion recognition tasks after considering the baseline signals. Our comparative experiments also confirmed that higher frequency bands of EEG signals can better characterize emotional states, and that the combination of features of multiple bands can complement each other and further improve the recognition accuracy.

[1]  Xiang Li,et al.  Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[2]  Zhong Yin,et al.  Recognition of emotions using multimodal physiological signals and an ensemble deep learning model , 2017, Comput. Methods Programs Biomed..

[3]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[4]  Peter W. McOwan,et al.  A real-time automated system for the recognition of human facial expressions , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Xiaoli Li,et al.  Signal Processing in Neuroscience , 2016, Springer Singapore.

[6]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based emotion classification , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[7]  Wei Liu,et al.  Multimodal Emotion Recognition Using Deep Neural Networks , 2017, ICONIP.

[8]  E. Ba§ar,et al.  EEG-Brain dynamics: Relation between EEG and brain evoked potentials , 1982 .

[9]  Soraia M. Alarcão,et al.  Emotions Recognition Using EEG Signals: A Survey , 2019, IEEE Transactions on Affective Computing.

[10]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[11]  Zhaoxiang Zhang,et al.  Hierarchical Convolutional Neural Networks for EEG-Based Emotion Recognition , 2017, Cognitive Computation.

[12]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[13]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based vigilance estimation , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[15]  Youjun Li,et al.  Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks , 2017 .

[16]  Valery A. Petrushin,et al.  EMOTION IN SPEECH: RECOGNITION AND APPLICATION TO CALL CENTERS , 1999 .

[17]  M. Steriade Alertness, Quiet Sleep, Dreaming , 1991 .