Emotion Recognition from Brain Signals Using Hybrid Adaptive Filtering and Higher Order Crossings Analysis

This paper aims at providing a new feature extraction method for a user-independent emotion recognition system, namely, HAF-HOC, from electroencephalograms (EEGs). A novel filtering procedure, namely, Hybrid Adaptive Filtering (HAF), for an efficient extraction of the emotion-related EEG-characteristics was developed by applying Genetic Algorithms to the Empirical Mode Decomposition-based representation of EEG signals. In addition, Higher Order Crossings (HOCs) analysis was employed for feature extraction realization from the HAF-filtered signals. The introduced HAF-HOC scheme incorporated four different classification methods to accomplish a robust emotion recognition performance. Through a series of facial-expression image projection, as a Mirror Neuron System-based emotion elicitation process, EEG data related to six basic emotions (happiness, surprise, anger, fear, disgust, and sadness) have been acquired from 16 healthy subjects using three EEG channels. Experimental results from the application of the HAF-HOC to the collected EEG data and comparison with previous approaches have shown that the HAF-HOC scheme clearly surpasses the latter in the field of emotion recognition from brain signals for the discrimination of up to six distinct emotions, providing higher classification rates up to 85.17 percent. The promising performance of the HAF-HOC surfaces the value of EEG signals within the endeavor of realizing more pragmatic, affective human-machine interfaces.

[1]  M. J. Katz,et al.  Fractals and the analysis of waveforms. , 1988, Computers in biology and medicine.

[2]  Gabriel Rilling,et al.  Empirical mode decomposition as a filter bank , 2004, IEEE Signal Processing Letters.

[3]  David G. Stork,et al.  Pattern Classification , 1973 .

[4]  Rosalind W. Picard.  Affective Computing , 1997 .

[5]  Kazuhiko Takahashi.  Remarks on Emotion Recognition from Bio-Potential Signals , 2004 .

[6]  Christine L. Lisetti,et al.  Emotion recognition from physiological signals using wireless sensors for presence technologies , 2004, Cognition, Technology & Work.

[7]  P. Agostino Accardo,et al.  Use of the fractal dimension for the analysis of electroencephalographic time series , 1997, Biological Cybernetics.

[8]  Hugo D Critchley,et al.  Imitating expressions: emotion-specific neural substrates in facial mimicry. , 2006, Social cognitive and affective neuroscience.

[9]  Guillaume Chanel,et al.  Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals , 2006, MRCS.

[10]  E Stålberg,et al.  Frequency and time domain characteristics of single muscle fibre action potentials. , 1975, Electroencephalography and clinical neurophysiology.

[11]  P. Ekman,et al.  Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I. , 1990, Journal of personality and social psychology.

[12]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[13]  Björn W. Schuller,et al.  Speaker Independent Speech Emotion Recognition by Ensemble Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[14]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[15]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[16]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[17]  P. Ekman.  Expression and the Nature of Emotion , 1984 .

[18]  E. Naumann,et al.  EEG asymmetry, dispositional mood and personality , 1999 .

[19]  Scotty D. Craig,et al.  AutoTutor Detects and Responds to Learners Affective and Cognitive States , 2008 .

[20]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[21]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[22]  M.H. Moradi,et al.  Emotion detection using brain and peripheral signals , 2008, 2008 Cairo International Biomedical Engineering Conference.

[23]  L. Aftanas,et al.  Affective picture processing: event-related synchronization within individually defined human theta band is modulated by valence dimension. , 2001, Neuroscience Letters.

[24]  R. Ward,et al.  EMG and EOG artifacts in brain computer interface systems: A survey , 2007, Clinical Neurophysiology.

[25]  Alicia Heraz,et al.  Predicting the Three Major Dimensions of the Learner-s Emotions from Brainwaves , 2007 .

[26]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[27]  Leontios J. Hadjileontiadis,et al.  EEG-based emotion recognition using hybrid filtering and higher order crossings , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[28]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[29]  Christine L. Lisetti,et al.  Emotion Recognition from Physiological Signals for User Modeling of Affect , 2003 .

[30]  D. O. Bos,et al.  EEG-based Emotion Recognition The Influence of Visual and Auditory Stimuli , 2007 .

[31]  Brian Everitt,et al.  Principles of Multivariate Analysis , 2001 .

[32]  Léon J. M. Rothkrantz,et al.  Emotion recognition using brain activity , 2008, CompSysTech.

[33]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[34]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[35]  W. Cannon.  The James-Lange theory of emotions: a critical examination and an alternative theory. By Walter B. Cannon, 1927. , 1927, American Journal of Psychology.

[36]  P. Mahalanobis.  On the generalized distance in statistics , 1936 .

[37]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[38]  W. Nauta.  The problem of the frontal lobe: a reinterpretation. , 1971, Journal of psychiatric research.

[39]  L M Patnaik,et al.  Feature Extraction using Fuzzy C - Means Clustering for Data Mining Systems , 2006 .

[40]  S. Yaacob,et al.  Lifting scheme for human emotion recognition using EEG , 2008, 2008 International Symposium on Information Technology.

[41]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[42]  Tanja Schultz,et al.  Towards an EEG-based emotion recognizer for humanoid robots , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[43]  Claude C. Chibelushi,et al.  Robust facial expression recognition using a state-based model of spatially-localised facial dynamics , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[44]  J. Mazziotta,et al.  Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[45]  R. Davidson.  What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research , 2004, Biological Psychology.

[46]  P. Ekman.  Pictures of Facial Affect , 1976 .

[47]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[48]  Kerry L. Coburn,et al.  Facts and artifacts in brain electrical activity mapping , 2005, Brain Topography.

[49]  G Pfurtscheller,et al.  Visualization of significant ERD/ERS patterns in multichannel EEG and ECoG data , 2002, Clinical Neurophysiology.

[50]  Travis Bradberry,et al.  Emotional Intelligence 2.0 , 2009 .

[51]  Takeo Kanade,et al.  Automated facial expression recognition based on FACS action units , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[52]  Thomas S. Huang,et al.  Emotion Recognition from Facial Expressions using Multilevel HMM , 2000 .

[53]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[54]  Arthur Petrosian,et al.  Kolmogorov complexity of finite sequences and recognition of different preictal EEG patterns , 1995, Proceedings Eighth IEEE Symposium on Computer-Based Medical Systems.

[55]  H. Critchley,et al.  Controlling emotional expression: behavioral and neural correlates of nonimitative emotional responses. , 2008, Cerebral cortex.

[56]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[57]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[58]  C. Darwin.  The Expression of Emotion in Man and Animals , 2020 .

[59]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  D. Spalding.  The Principles of Psychology , 1873, Nature.

[61]  T. Higuchi.  Approach to an irregular time series on the basis of the fractal theory , 1988 .

[62]  Ed. McKenzie.  10. Time Series Analysis by Higher Order Crossings , 1996 .

[63]  G. Rizzolatti,et al.  Both of Us Disgusted in My Insula The Common Neural Basis of Seeing and Feeling Disgust , 2003, Neuron.

[64]  Michael A. Arbib,et al.  Mirror neurons and imitation: A computationally guided review , 2006, Neural Networks.

[65]  P. Winkielman,et al.  When the social mirror breaks: deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. , 2006, Developmental science.

[66]  P. Ekman,et al.  Autonomic nervous system activity distinguishes among emotions. , 1983, Science.

[67]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.