Emotion recognition based on sparse representation of phase synchronization features

Emotion recognition based on Electroencephalogram (EEG) has attracted much attention in brain-computer interaction. However, most existing methods usually focus on amplitude and spectrum of the EEG signal, leading to sub-optimal performances due to the insufficiency in modelling the complex intrinsic information of neural integration. To address this issue, this paper proposes to capitalize on the largely neglected phase synchronization (PS) between EEG channels which reflects the intrinsic rhythmic interactions between different channels in neural integration. Specifically, this paper develops a simple and novel feature extraction method which calculates the PS based sparse representation features to analyze emotion states. First, the EEG phase synchronization indexes (PSI) of all channel pairs are estimated as features to distinguish different emotions, since certain topographical maps on PSI reveal specific emotion states. Then principal component analysis is performed to eliminate redundant and noisy features in PSI. Finally, Sparse Representation based Classification (SRC) furtherly emphasize emotion-related features and restrain useless features. For the benchmark affective EEG dataset DEAP, the proposed method based on no-overlapping EEG features achieve an average accuracy of 94.5%, 87.61%, and 67.04% for the classification tasks respectively on two, three and four emotions, demonstrating the superiority over state-of-the-art emotion classification methods.

[1]  Seong Youb Chung,et al.  Affective classification using Bayesian classifier and supervised learning , 2012, 2012 12th International Conference on Control, Automation and Systems.

[2]  Hongtao Lu,et al.  Vigilance detection based on sparse representation of EEG , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[3]  Mohammad Soleymani,et al.  Single Trial Classification of EEG and Peripheral Physiological Signals for Recognition of Emotions Induced by Music Videos , 2010, Brain Informatics.

[4]  L. Trainor,et al.  Unpredicted Pitch Modulates Beta Oscillatory Power during Rhythmic Entrainment to a Tone Sequence , 2016, Front. Psychol..

[5]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[6]  A. N. Paithane,et al.  Human disposition detection using EEG signals , 2016, 2016 International Conference on Computing, Analytics and Security Trends (CAST).

[7]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[8]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[9]  Zhijun Li,et al.  Inferring Functional Neural Connectivity with Phase Synchronization Analysis: A Review of Methodology , 2012, Comput. Math. Methods Medicine.

[10]  Fabio Babiloni,et al.  Assessment of driving fatigue based on intra/inter-region phase synchronization , 2017, Neurocomputing.

[11]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[12]  Jiahui Pan,et al.  Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition , 2017, Comput. Intell. Neurosci..

[13]  L. Schmidt,et al.  Cross-regional cortical synchronization during affective image viewing , 2010, Brain Research.

[14]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[15]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[16]  Hui Li,et al.  Induced gamma activity in EEG represents cognitive control during detecting emotional expressions , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  L. F. Barrett,et al.  Sex differences in the neural correlates of affective experience. , 2014, Social cognitive and affective neuroscience.

[18]  Jiahui Pan,et al.  An EEG-Based Brain Computer Interface for Emotion Recognition and Its Application in Patients with Disorder of Consciousness , 2019, IEEE Transactions on Affective Computing.

[19]  Min Wu,et al.  Speech emotion recognition based on an improved brain emotion learning model , 2018, Neurocomputing.

[20]  Heung-No Lee,et al.  Sparse representation-based classification scheme for motor imagery-based brain–computer interface systems , 2012, Journal of neural engineering.

[21]  Tommaso Costa,et al.  EEG phase synchronization during emotional response to positive and negative film stimuli , 2006, Neuroscience Letters.

[22]  Yuan-Pin Lin,et al.  EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Junfeng Sun,et al.  Unified framework for detecting phase synchronization in coupled time series. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[25]  T. Gasser,et al.  EEG power and coherence while male adults watch emotional video films. , 1990, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[26]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  M L Phillips,et al.  Interaction of catechol O-methyltransferase and serotonin transporter genes modulates effective connectivity in a facial emotion-processing circuitry , 2012, Translational Psychiatry.

[28]  Olga Sourina,et al.  EEG Databases for Emotion Recognition , 2013, 2013 International Conference on Cyberworlds.

[29]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[30]  J. Russell.  Affective space is bipolar. , 1979 .

[31]  Hung T. Nguyen,et al.  Recognizing emotions from EEG subbands using wavelet analysis , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[32]  Kalyana Chakravarthy Veluvolu,et al.  EEG classification of emotions using emotion-specific brain functional network , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[33]  Zhong Liu,et al.  A Novel Ensemble Method for Imbalanced Data Learning: Bagging of Extrapolation-SMOTE SVM , 2017, Comput. Intell. Neurosci..

[34]  Kalyana Chakravarthy Veluvolu,et al.  Identification of emotion associated brain functional network with phase locking value , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[35]  Seungchan Lee,et al.  Simple adaptive sparse representation based classification schemes for EEG based brain-computer interface applications , 2015, Comput. Biol. Medicine.

[36]  I. Jolliffe.  Principal Component Analysis , 2005 .

[37]  Reda A. El-Khoribi,et al.  Emotion Recognition based on EEG using LSTM Recurrent Neural Network , 2017 .

[38]  Md. Foisal Hossain,et al.  Employing PCA and t-statistical approach for feature extraction and classification of emotion from multichannel EEG signal , 2020 .

[39]  Liang Dong,et al.  Recognition of Emotions Using Multichannel EEG Data and DBN-GC-Based Ensemble Deep Learning Framework , 2018, Comput. Intell. Neurosci..

[40]  Olga Sourina,et al.  Real-Time Fractal-Based Valence Level Recognition from EEG , 2013, Trans. Comput. Sci..

[41]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[42]  I. Christie,et al.  Autonomic specificity of discrete emotion and dimensions of affective space: a multivariate approach. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[43]  Konstantinos N. Plataniotis,et al.  Affective states classification using EEG and semi-supervised deep learning approaches , 2016, 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP).

[44]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[45]  S. Hsieh,et al.  Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns , 2014, PloS one.