ScalingNet: extracting features from raw EEG data for emotion recognition

Convolutional Neural Networks(CNNs) has achieved remarkable performance breakthrough in a variety of tasks. Recently, CNNs based methods that are fed with hand-extracted EEG features gradually produce a powerful performance on the EEG data based emotion recognition task. In this paper, we propose a novel convolutional layer allowing to adaptively extract effective data-driven spectrogram-like features from raw EEG signals, which we reference as scaling layer. Further, it leverages convolutional kernels scaled from one data-driven pattern to exposed a frequency-like dimension to address the shortcomings of prior methods requiring hand-extracted features or their approximations. The proposed neural network architecture based on the scaling layer, references as ScalingNet, has achieved the state-of-the-art result across the established DEAP benchmark dataset.

[1]  D. Song,et al.  EEG Based Emotion Identification Using Unsupervised Deep Feature Learning , 2015 .

[2]  K. R. Seeja,et al.  Subject independent emotion recognition from EEG using VMD and deep learning , 2019, J. King Saud Univ. Comput. Inf. Sci..

[3]  Tiago H. Falk,et al.  Deep learning-based electroencephalography analysis: a systematic review , 2019, Journal of neural engineering.

[4]  Zhenqi Li,et al.  SAE+LSTM: A New Framework for Emotion Recognition From Multi-Channel EEG , 2019, Front. Neurorobot..

[5]  Yu-Xuan Yang,et al.  A GPSO-optimized convolutional neural networks for EEG-based emotion recognition , 2020, Neurocomputing.

[6]  Yanli Li,et al.  A Channel-Fused Dense Convolutional Network for EEG-Based Emotion Recognition , 2020, IEEE Transactions on Cognitive and Developmental Systems.

[7]  Shaun Canavan,et al.  Emotion Recognition Using Fused Physiological Signals , 2019, 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).

[8]  Qinmu Peng,et al.  Emotion Classification Using EEG Brain Signals and the Broad Learning System , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[9]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999 .

[10]  Wei Liu,et al.  Emotion Recognition Using Multimodal Deep Learning , 2016, ICONIP.

[11]  Amit Konar,et al.  Emotion Recognition: A Pattern Analysis Approach , 2015 .

[12]  Tiago H. Falk,et al.  Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization , 2016, Neurocomputing.

[13]  Ying Zeng,et al.  EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations , 2019, IEEE Transactions on Biomedical Engineering.

[14]  Youjun Li,et al.  Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks , 2017 .

[15]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[16]  Chao Li,et al.  Exploring temporal representations by leveraging attention-based bidirectional LSTM-RNNs for multi-modal emotion recognition , 2020, Inf. Process. Manag..

[17]  Liang Dong,et al.  Emotion Recognition from Multiband EEG Signals Using CapsNet , 2019, Sensors.

[18]  Sungho Jo,et al.  Deep Physiological Affect Network for the Recognition of Human Emotions , 2020, IEEE Transactions on Affective Computing.

[19]  Andrzej Cichocki,et al.  EmotionMeter: A Multimodal Framework for Recognizing Human Emotions , 2019, IEEE Transactions on Cybernetics.

[20]  Ioannis Patras,et al.  AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups , 2017, IEEE Transactions on Affective Computing.

[21]  Brendan McCane,et al.  EmotioNet: A 3-D Convolutional Neural Network for EEG-based Emotion Recognition , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[22]  R. Adolphs,et al.  Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala , 1994, Nature.

[23]  J. X. Chen,et al.  A Hierarchical Bidirectional GRU Model With Attention for EEG-Based Emotion Classification , 2019, IEEE Access.

[24]  Kyungha Min,et al.  A Multi-Column CNN Model for Emotion Recognition from EEG Signals , 2019, Sensors.

[25]  Mei Wang,et al.  Review of the emotional feature extraction and classification using EEG signals , 2021 .

[26]  N. Sadato,et al.  Neural Interaction of the Amygdala with the Prefrontal and Temporal Cortices in the Processing of Facial Expressions as Revealed by fMRI , 2001, Journal of Cognitive Neuroscience.