Real-time EEG-based emotion monitoring using stable features

In human–computer interaction (HCI), electroencephalogram (EEG) signals can be added as an additional input to computer. An integration of real-time EEG-based human emotion recognition algorithms in human–computer interfaces can make the users experience more complete, more engaging, less emotionally stressful or more stressful depending on the target of the applications. Currently, the most accurate EEG-based emotion recognition algorithms are subject-dependent, and a training session is needed for the user each time right before running the application. In this paper, we propose a novel real-time subject-dependent algorithm with the most stable features that gives a better accuracy than other available algorithms when it is crucial to have only one training session for the user and no re-training is allowed subsequently. The proposed algorithm is tested on an affective EEG database that contains five subjects. For each subject, four emotions (pleasant, happy, frightened and angry) are induced, and the affective EEG is recorded for two sessions per day in eight consecutive days. Testing results show that the novel algorithm can be used in real-time emotion recognition applications without re-training with the adequate accuracy. The proposed algorithm is integrated with real-time applications “Emotional Avatar” and “Twin Girls” to monitor the users emotions in real time.

[1]  Olga Sourina,et al.  Real-Time EEG-Based Emotion Recognition and Its Applications , 2011, Trans. Comput. Sci..

[2]  Johan Hagelbäck,et al.  Evaluating Classifiers for Emotion Recognition Using EEG , 2013, HCI.

[3]  S. Sigurdsson,et al.  Reliability of quantitative EEG features , 2007, Clinical Neurophysiology.

[4]  Olga Sourina,et al.  Real-Time Fractal-Based Valence Level Recognition from EEG , 2013, Trans. Comput. Sci..

[5]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[6]  Yuan-Pin Lin,et al.  EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Olga Sourina,et al.  Stability of Features in Real-Time EEG-based Emotion Recognition Algorithm , 2014, 2014 International Conference on Cyberworlds.

[8]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[9]  Tanja Schultz,et al.  Towards an EEG-based emotion recognizer for humanoid robots , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[10]  Olga Sourina,et al.  Real-time EEG-based emotion recognition for music therapy , 2011, Journal on Multimodal User Interfaces.

[11]  Masafumi Hagiwara,et al.  A feeling estimation system using a simple electroencephalograph , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[12]  Saeid Sanei,et al.  EEG signal processing , 2000, Clinical Neurophysiology.

[13]  Ed. McKenzie.  10. Time Series Analysis by Higher Order Crossings , 1996 .

[14]  Charalampos Bratsas,et al.  Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli , 2010, IEEE Transactions on Information Technology in Biomedicine.

[15]  K. Stevens,et al.  Emotions and speech: some acoustical correlates. , 1972, The Journal of the Acoustical Society of America.

[16]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[17]  Heather L. Urry,et al.  The stability of resting frontal electroencephalographic asymmetry in depression. , 2004, Psychophysiology.

[18]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[19]  M. Andrés.  Learning and behavior: A contemporary synthesis , 2008 .

[20]  R. E. Wheeler,et al.  Psychometric properties of resting anterior EEG asymmetry: temporal stability and internal consistency. , 1992, Psychophysiology.

[21]  P. Lang,et al.  International Affective Picture System (IAPS): Instruction Manual and Affective Ratings (Tech. Rep. No. A-4) , 1999 .

[22]  Julien Penders,et al.  Towards wireless emotional valence detection from EEG , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[23]  R Verleger,et al.  EEG coherence at rest and during a visual task in two groups of children. , 1987, Electroencephalography and clinical neurophysiology.

[24]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  T. Higuchi.  Approach to an irregular time series on the basis of the fractal theory , 1988 .

[26]  A. Kondacs,et al.  Long-term intra-individual variability of the background EEG in normals , 1999, Clinical Neurophysiology.

[27]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[28]  Olga Sourina,et al.  Real-Time Subject-Dependent EEG-Based Emotion Recognition Algorithm , 2014, Trans. Comput. Sci..

[29]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[30]  Bao-Liang Lu,et al.  Emotion classification based on gamma-band EEG , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Bao-Liang Lu,et al.  EEG-Based Emotion Recognition Using Frequency Domain Features and Support Vector Machines , 2011, ICONIP.

[32]  B. Oken,et al.  Test-retest reliability in EEG frequency analysis. , 1991, Electroencephalography and clinical neurophysiology.

[33]  Olga Sourina,et al.  EEG Databases for Emotion Recognition , 2013, 2013 International Conference on Cyberworlds.

[34]  Leontios J. Hadjileontiadis,et al.  Adaptive Emotional Information Retrieval From EEG Signals in the Time-Frequency Domain , 2012, IEEE Transactions on Signal Processing.

[35]  T. Gasser,et al.  Test-retest reliability of spectral parameters of the EEG. , 1985, Electroencephalography and clinical neurophysiology.

[36]  R. Nagarajan,et al.  Combining Spatial Filtering and Wavelet Transform for Classifying Human Emotions Using EEG Signals , 2011 .

[37]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[38]  S. A. Hosseini,et al.  Higher Order Spectra Analysis of EEG Signals in Emotional Stress States , 2010, 2010 Second International Conference on Information Technology and Computer Science.

[39]  Leontios J. Hadjileontiadis,et al.  A Novel Emotion Elicitation Index Using Frontal Brain Asymmetry for Enhanced EEG-Based Emotion Recognition , 2011, IEEE Transactions on Information Technology in Biomedicine.