Emotion Recognition and Dynamic Functional Connectivity Analysis Based on EEG

Although emotion recognition techniques have been well developed, the understanding of the neural mechanism remains rudimentary. The traditional static network approach cannot reflect the entire brain activity at the time scale. Instead, a newly introduced temporal brain network is an optimal approach which can be used to investigate the dynamic functional connectivity (FC) of the human brain in different emotion states considering the time-varying brain regional interaction. In this study, we focused on emotion recognition and dynamic FC analysis with SEED dataset. First, multiband static networks were computed by the phase lag index (PLI). Then, subject-independent discriminative connection features of such static networks were selected to recognize the positive, neutral, and negative emotion types. In addition, we constructed the temporal brain network by sorting the static network according to time sequence. The experimental results show that the beta band is the most suitable for emotion recognition due to the best accuracy of 87.03%. And, the frontal and the temporal lobes are more sensitive to brain emotion-related activities. Moreover, we find the spatio-temporal topology of dynamic FC shows the small-world structure. Notably, the positive emotion is more distinguishable in the temporal global efficiency, especially between positive and neutral emotion states. Our findings provide new insight into the emotion-related brain regional coordination evolution and show the potential of dynamic FC for the investigation of the emotion-related brain mechanism.

[1]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[2]  Serap Aydin,et al.  Emotion Recognition with Eigen Features of Frequency Band Activities Embedded in Induced Brain Oscillations Mediated by Affective Pictures , 2016, Int. J. Neural Syst..

[3]  Min Zhao,et al.  The Reorganization of Human Brain Networks Modulated by Driving Mental Fatigue , 2017, IEEE Journal of Biomedical and Health Informatics.

[4]  Josef Kittler,et al.  Floating search methods for feature selection with nonmonotonic criterion functions , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[5]  Yong-Jin Liu,et al.  Real-Time Movie-Induced Discrete Emotion Recognition from EEG Signals , 2018, IEEE Transactions on Affective Computing.

[6]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[7]  Loïc Kessous,et al.  Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis , 2010, Journal on Multimodal User Interfaces.

[8]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[9]  Oluwarotimi Williams Samuel,et al.  Towards Efficient Decoding of Multiple Classes of Motor Imagery Limb Movements Based on EEG Spectral and Time Domain Descriptors , 2017, Journal of Medical Systems.

[10]  Chi Zhang,et al.  Emotion Recognition from EEG Signals Using Multidimensional Information in EMD Domain , 2017, BioMed research international.

[11]  Cecilia Mascolo,et al.  Temporal distance metrics for social network analysis , 2009, WOSN '09.

[12]  Erol Başar,et al.  Affective pictures processing is reflected by an increased long-distance EEG connectivity , 2017, Cognitive Neurodynamics.

[13]  Yong Zhang,et al.  Classification of EEG Signals Based on Autoregressive Model and Wavelet Packet Decomposition , 2016, Neural Processing Letters.

[14]  H. Adeli,et al.  Brain functional connectivity patterns for emotional state classification in Parkinson’s disease patients without dementia , 2016, Behavioural Brain Research.

[15]  David A. Leopold,et al.  Dynamic functional connectivity: Promise, issues, and interpretations , 2013, NeuroImage.

[16]  Scott T. Grafton,et al.  Dynamic reconfiguration of human brain networks during learning , 2010, Proceedings of the National Academy of Sciences.

[17]  T. Jung,et al.  Dry and Noncontact EEG Sensors for Mobile Brain–Computer Interfaces , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Nida Itrat Abbasi,et al.  A novel real-time driving fatigue detection system based on wireless dry EEG , 2018, Cognitive Neurodynamics.

[19]  Richard M. Leahy,et al.  A note on the phase locking value and its properties , 2013, NeuroImage.

[20]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[21]  Petter Holme,et al.  Analyzing Temporal Networks in Social Media , 2014, Proceedings of the IEEE.

[22]  J. Fell,et al.  The role of phase synchronization in memory processes , 2011, Nature Reviews Neuroscience.

[23]  Wan Khairunizam,et al.  Implementation of wavelet packet transform and non linear analysis for emotion classification in stroke patient using brain signals , 2017, Biomed. Signal Process. Control..

[24]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[25]  Yu Chen,et al.  Temporal efficiency evaluation and small-worldness characterization in temporal networks , 2016, Scientific Reports.

[26]  Petter Holme,et al.  Modern temporal network theory: a colloquium , 2015, The European Physical Journal B.

[27]  Cheonshu Park,et al.  An Emotion Expression System for the Emotional Robot , 2007, 2007 IEEE International Symposium on Consumer Electronics.

[28]  Abdulhamit Subasi,et al.  EEG signal classification using PCA, ICA, LDA and support vector machines , 2010, Expert Syst. Appl..

[29]  C. Stam,et al.  Phase lag index: Assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources , 2007, Human brain mapping.

[30]  Mahmoud Hassan,et al.  Electroencephalography Source Connectivity: Aiming for High Resolution of Brain Networks in Time and Space , 2018, IEEE Signal Processing Magazine.

[31]  Olaf Sporns,et al.  THE HUMAN CONNECTOME: A COMPLEX NETWORK , 2011, Schizophrenia Research.

[32]  A. Bezerianos,et al.  Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks , 2014, Brain and Cognition.

[33]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[34]  John Suckling,et al.  Dynamic Reorganization of Functional Connectivity Reveals Abnormal Temporal Efficiency in Schizophrenia , 2018, Schizophrenia bulletin.

[35]  Bao-Liang Lu,et al.  EEG-Based Emotion Recognition Using Frequency Domain Features and Support Vector Machines , 2011, ICONIP.

[36]  S. Murugappan,et al.  Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT) , 2013, 2013 IEEE 9th International Colloquium on Signal Processing and its Applications.

[37]  Edward T. Bullmore,et al.  Efficiency and Cost of Economical Brain Functional Networks , 2007, PLoS Comput. Biol..

[38]  Oluwarotimi Williams Samuel,et al.  A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees , 2017, Journal of NeuroEngineering and Rehabilitation.

[39]  Shashidhar G. Koolagudi,et al.  Emotion recognition from speech: a review , 2012, International Journal of Speech Technology.

[40]  Olga Sourina,et al.  Real-Time EEG-Based Emotion Recognition and Its Applications , 2011, Trans. Comput. Sci..

[41]  Barry Horwitz,et al.  The elusive concept of brain connectivity , 2003, NeuroImage.

[42]  Oluwarotimi Williams Samuel,et al.  Motor imagery classification of upper limb movements based on spectral domain features of EEG patterns , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[43]  Ying Zeng,et al.  EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations , 2019, IEEE Transactions on Biomedical Engineering.

[44]  Sahar Moghimi,et al.  Toward automatic detection of brain responses to emotional music through analysis of EEG effective connectivity , 2016, Comput. Hum. Behav..

[45]  Yingying Tang,et al.  Abnormal functional connectivity of EEG gamma band in patients with depression during emotional face processing , 2015, Clinical Neurophysiology.

[46]  Hugo Silva,et al.  Multimodal biosignal sensor data handling for emotion recognition , 2011, 2011 IEEE SENSORS Proceedings.

[47]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[48]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[49]  Milan Palus,et al.  Nonlinearity in normal human EEG: cycles, temporal asymmetry, nonstationarity and randomness, not chaos , 1996, Biological Cybernetics.

[50]  Dick de Waard,et al.  Monitoring drivers' mental workload in driving simulators using physiological measures. , 2010, Accident; analysis and prevention.

[51]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Weiqi Wang,et al.  Functional connectivity analysis using fNIRS in healthy subjects during prolonged simulated driving , 2017, Neuroscience Letters.

[53]  Bao-Liang Lu,et al.  EEG-based emotion recognition during watching movies , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[54]  Peerapon Vateekul,et al.  An evaluation of feature extraction in EEG-based emotion prediction with support vector machines , 2014, 2014 11th International Joint Conference on Computer Science and Software Engineering (JCSSE).