Interpretable Emotion Recognition Using EEG Signals

Electroencephalogram (EEG) signal-based emotion recognition has attracted wide interests in recent years and has been broadly adopted in medical, affective computing, and other relevant fields. However, the majority of the research reported in this field tends to focus on the accuracy of classification whilst neglecting the interpretability of emotion progression. In this paper, we propose a new interpretable emotion recognition approach with the activation mechanism by using machine learning and EEG signals. This paper innovatively proposes the emotional activation curve to demonstrate the activation process of emotions. The algorithm first extracts features from EEG signals and classifies emotions using machine learning techniques, in which different parts of a trial are used to train the proposed model and assess its impact on emotion recognition results. Second, novel activation curves of emotions are constructed based on the classification results, and two emotion coefficients, i.e., the correlation coefficients and entropy coefficients. The activation curve can not only classify emotions but also reveals to a certain extent the emotional activation mechanism. Finally, a weight coefficient is obtained from the two coefficients to improve the accuracy of emotion recognition. To validate the proposed method, experiments have been carried out on the DEAP and SEED dataset. The results support the point that emotions are progressively activated throughout the experiment, and the weighting coefficients based on the correlation coefficient and the entropy coefficient can effectively improve the EEG-based emotion recognition accuracy.

[1]  J. Russell A circumplex model of affect. , 1980 .

[2]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[3]  José Manuel Pastor,et al.  Application of Entropy-Based Metrics to Identify Emotional Distress from Electroencephalographic Recordings , 2016, Entropy.

[4]  Idoia Cearreta,et al.  ASSISTIVE TECHNOLOGY AND AFFECTIVE MEDIATION , 2006 .

[5]  K. Vohs,et al.  Case Western Reserve University , 1990 .

[6]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[7]  Chi Zhang,et al.  Investigating Patterns for Self-Induced Emotion Recognition from EEG Signals , 2018, Sensors.

[8]  Bao-Liang Lu,et al.  Augmentation for Emotion Recognition Using a Conditional , 2018 .

[9]  Gernot R. Müller-Putz,et al.  Domain Adaptation Techniques for EEG-Based Emotion Recognition: A Comparative Study on Two Public Datasets , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[10]  Abeer Al-Nafjan,et al.  Recognition of Affective States via Electroencephalogram Analysis and Classification , 2018, IHSI.

[11]  Samit Bhattacharya,et al.  Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset , 2017, AAAI.

[12]  M. Shamim Hossain,et al.  Patient State Recognition System for Healthcare Using Speech and Facial Expressions , 2016, Journal of Medical Systems.

[13]  N. Frijda The laws of emotion. , 1988, The American psychologist.

[14]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[15]  Leonardo Trujillo,et al.  Detecting Epilepsy in EEG Signals Using Time, Frequency and Time-Frequency Domain Features , 2018 .

[16]  Bin Hu,et al.  Exploring EEG Features in Cross-Subject Emotion Recognition , 2018, Front. Neurosci..

[17]  Maria Lewicka,et al.  Positive‐negative asymmetry or ‘When the heart needs a reason’ , 1992 .

[18]  Xianxiang Chen,et al.  Respiration-based emotion recognition with deep learning , 2017, Comput. Ind..

[19]  Mustafa E. Kamasak,et al.  Multi channel brain EEG signals based emotional arousal classification with unsupervised feature learning using autoencoders , 2017, 2017 25th Signal Processing and Communications Applications Conference (SIU).

[20]  Christian Kleine-Cosack,et al.  Recognition and Simulation of Emotions , 2007 .

[21]  Yue Wang,et al.  A three-stage decision framework for multi-subject emotion recognition using physiological signals , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[22]  Yong Zhang,et al.  EEG-based classification of emotions using empirical mode decomposition and autoregressive model , 2018, Multimedia Tools and Applications.

[23]  Edward B. Royzman,et al.  Negativity Bias, Negativity Dominance, and Contagion , 2001 .

[24]  Nitin Kumar,et al.  Bispectral Analysis of EEG for Emotion Recognition , 2015, IHCI.

[25]  Kostas Karpouzis,et al.  Estimation of behavioral user state based on eye gaze and head pose—application in an e-learning environment , 2009, Multimedia Tools and Applications.

[26]  Yun Luo,et al.  WGAN Domain Adaptation for EEG-Based Emotion Recognition , 2018, ICONIP.

[27]  Egon L. van den Broek,et al.  Tune in to your emotions: a robust personalized affective music player , 2012, User Modeling and User-Adapted Interaction.

[28]  Ruifeng Xu,et al.  A novel convolutional neural networks for emotion recognition based on EEG signal , 2017, 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[29]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[30]  John J. B. Allen,et al.  Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion , 2004, Biological Psychology.