Feature Extraction and Selection for Emotion Recognition from EEG

Emotion recognition from EEG signals allows the direct assessment of the “inner” state of a user, which is considered an important factor in human-machine-interaction. Many methods for feature extraction have been studied and the selection of both appropriate features and electrode locations is usually based on neuro-scientific findings. Their suitability for emotion recognition, however, has been tested using a small amount of distinct feature sets and on different, usually small data sets. A major limitation is that no systematic comparison of features exists. Therefore, we review feature extraction methods for emotion recognition from EEG based on 33 studies. An experiment is conducted comparing these features using machine learning techniques for feature selection on a self recorded data set. Results are presented with respect to performance of different feature selection methods, usage of selected feature types, and selection of electrode locations. Features selected by multivariate methods slightly outperform univariate methods. Advanced feature extraction techniques are found to have advantages over commonly used spectral power bands. Results also suggest preference to locations over parietal and centro-parietal lobes.

[1]  R. Levenson.  Emotion and the autonomic nervous system: A prospectus for research on autonomic specificity. , 1988 .

[2]  Hatice Gunes,et al.  Automatic, Dimensional and Continuous Emotion Recognition , 2010, Int. J. Synth. Emot..

[3]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[4]  Olga Sourina,et al.  Real-Time Fractal-Based Valence Level Recognition from EEG , 2013, Trans. Comput. Sci..

[5]  Thierry Pun,et al.  Valence-arousal evaluation using physiological signals in an emotion recall paradigm , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[6]  Leontios J. Hadjileontiadis,et al.  Toward an EEG-Based Recognition of Music Liking Using Time-Frequency Analysis , 2012, IEEE Transactions on Biomedical Engineering.

[7]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[8]  Tanja Schultz,et al.  Towards emotion recognition from electroencephalographic signals , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[9]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[10]  Touradj Ebrahimi,et al.  EEG Correlates of Different Emotional States Elicited during Watching Music Videos , 2011, ACII.

[11]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[12]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[13]  Scott Makeig,et al.  First Demonstration of a Musical Emotion BCI , 2011, ACII.

[14]  Kai Keng Ang,et al.  EEG-based Emotion Recognition Using Self-Organizing Map for Boundary Detection , 2010, 2010 20th International Conference on Pattern Recognition.

[15]  Olga Sourina,et al.  EEG-based Dominance Level Recognition for Emotion-Enabled Interaction , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[16]  T. Higuchi.  Approach to an irregular time series on the basis of the fractal theory , 1988 .

[17]  Charalampos Bratsas,et al.  Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli , 2010, IEEE Transactions on Information Technology in Biomedicine.

[18]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[19]  K. Takahashi,et al.  Remarks on SVM-based emotion recognition from multi-modal bio-potential signals , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[20]  D. Aboutajdine,et al.  Feature Selection on Supervised Classification Using Wilks Lambda Statistic , 2007, 2007 International Symposium on Computational Intelligence and Intelligent Informatics.

[21]  Martin Buss,et al.  Towards a System-Theoretic Model for Transition of Affect , 2009 .

[22]  Christian Mühl,et al.  Valence, arousal and dominance in the EEG during game play , 2013, Int. J. Auton. Adapt. Commun. Syst..

[23]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  Julien Penders,et al.  Towards wireless emotional valence detection from EEG , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[26]  J. Cacioppo.  Feelings and emotions: roles for electrophysiological markers , 2004, Biological Psychology.

[27]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[28]  Carl J. Huberty,et al.  Applied MANOVA and discriminant analysis , 2006 .

[29]  Reza Khosrowabadi,et al.  Classification of EEG correlates on emotion using features from Gaussian mixtures of EEG spectrogram , 2010, Proceeding of the 3rd International Conference on Information and Communication Technology for the Moslem World (ICT4M) 2010.

[30]  B. Hjorth.  EEG analysis based on time domain properties. , 1970, Electroencephalography and clinical neurophysiology.

[31]  Jeffrey M. Hausdorff,et al.  Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis. , 2000, Journal of applied physiology.

[32]  Mohammad Soleymani,et al.  Continuous emotion detection in response to music videos , 2011, Face and Gesture 2011.

[33]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[34]  Toshimitsu Musha,et al.  Feature extraction from EEGs associated with emotions , 1997, Artificial Life and Robotics.

[35]  A. Brouwer,et al.  Modality-specific Affective Responses and their Implications for Affective BCI , 2011 .

[36]  Thierry Pun,et al.  A channel selection method for EEG classification in emotion assessment based on synchronization likelihood , 2007, 2007 15th European Signal Processing Conference.

[37]  Kazuhiko Takahashi.  Remarks on emotion recognition from multi-modal bio-potential signals , 2004 .

[38]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition from Brain Signals Using Hybrid Adaptive Filtering and Higher Order Crossings Analysis , 2010, IEEE Transactions on Affective Computing.

[39]  Anil K. Ghosh,et al.  Applied MANOVA and Discriminant Analysis (2nd ed.). Carl J. Huberty and Stephen Olejnik , 2007 .

[40]  Bao-Liang Lu,et al.  EEG-based emotion recognition during watching movies , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[41]  Sazali Yaacob,et al.  EEG feature extraction for classifying emotions using FCM and FKM , 2008 .

[42]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[43]  Léon J. M. Rothkrantz,et al.  Emotion recognition using brain activity , 2008, CompSysTech.

[44]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[45]  Rohit Prasad,et al.  Robust EEG emotion classification using segment level decision fusion , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Bao-Liang Lu,et al.  EEG-Based Emotion Recognition Using Frequency Domain Features and Support Vector Machines , 2011, ICONIP.

[47]  C. Parameswariah,et al.  Frequency Characteristics of Wavelets , 2002, IEEE Power Engineering Review.

[48]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[49]  K. Takahashi,et al.  Remarks on emotion recognition from multi-modal bio-potential signals , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[50]  Panagiotis D. Bamidis,et al.  Real time emotion aware applications: A case study employing emotion evocative pictures and neuro-physiological sensing enhanced by Graphic Processor Units , 2012, Comput. Methods Programs Biomed..

[51]  Olga Sourina,et al.  A Fractal-based Algorithm of Emotion Recognition from EEG using Arousal-Valence Model , 2011, BIOSIGNALS.

[52]  Hugues Bersini,et al.  A Survey on Filter Techniques for Feature Selection in Gene Expression Microarray Analysis , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[53]  M.H. Moradi,et al.  Emotion detection using brain and peripheral signals , 2008, 2008 Cairo International Biomedical Engineering Conference.

[54]  M. Akin,et al.  Comparison of Wavelet Transform and FFT Methods in the Analysis of EEG Signals , 2002, Journal of Medical Systems.

[55]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[56]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Pavel Pudil,et al.  Efficient Feature Subset Selection and Subset Size Optimization , 2010 .

[58]  Martin Buss,et al.  Effect-size-based electrode and feature selection for emotion recognition from EEG , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[60]  S. A. Hosseini,et al.  Higher Order Spectra Analysis of EEG Signals in Emotional Stress States , 2010, 2010 Second International Conference on Information Technology and Computer Science.

[61]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .