Convolutional Neural Network Approach for Eeg-Based Emotion Recognition Using Brain Connectivity and its Spatial Information

Emotion recognition based on electroencephalography (EEG) has received attention as a way to implement human-centric services. However, there is still much room for improvement, particularly in terms of the recognition accuracy. In this paper, we propose a novel deep learning approach using convolutional neural networks (CNNs) for EEG-based emotion recognition. In particular, we employ brain connectivity features that have not been used with deep learning models in previous studies, which can account for synchronous activations of different brain regions. In addition, we develop a method to effectively capture asymmetric brain activity patterns that are important for emotion recognition. Experimental results confirm the effectiveness of our approach.

[1]  Soraia M. Alarcão,et al.  Emotions Recognition Using EEG Signals: A Survey , 2019, IEEE Transactions on Affective Computing.

[2]  Vangelis Sakkalis,et al.  Review of advanced techniques for the estimation of brain connectivity measured with EEG/MEG , 2011, Comput. Biol. Medicine.

[3]  Karl J. Friston.  Functional and Effective Connectivity: A Review , 2011, Brain Connect..

[4]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[5]  C. Stam,et al.  Phase lag index: Assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources , 2007, Human brain mapping.

[6]  Seong-Eun Moon,et al.  EEG Connectivity Analysis in Perception of Tone-mapped High Dynamic Range Videos , 2015, ACM Multimedia.

[7]  Mohammed Yeasin,et al.  Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks , 2015, ICLR.

[8]  Goutam Saha,et al.  Classification of emotions induced by music videos and correlation with participants' rating , 2014, Expert Syst. Appl..

[9]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[10]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[11]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[12]  Wei Liu,et al.  Emotion Recognition Using Multimodal Deep Learning , 2016, ICONIP.

[13]  Xiaodan Zhuang,et al.  Compact unsupervised EEG response representation for emotion recognition , 2014, IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[14]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[15]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[16]  Tiago H. Falk,et al.  Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization , 2016, Neurocomputing.

[17]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[18]  Touradj Ebrahimi,et al.  Affect recognition based on physiological changes during the watching of music videos , 2012, TIIS.

[19]  C. P. Latha,et al.  A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition , 2016, APTIKOM Journal on Computer Science and Information Technologies.