An investigation on visual and audiovisual stimulus based emotion recognition using EEG

In this paper, we investigate the possibility of using visual and audio visual stimulus for detecting the human emotion by measuring electroencephalogram (EEG). Visual and audiovisual stimulus based protocols is designed to acquire the EEG signals over five healthy subjects using 63 biosensors. We propose the analysis of EEG signals using discrete wavelet transform and classification using neural network. EEG signals were decomposed into five frequency sub-bands using 'db4' wavelet function and two statistical features were extracted from the alpha band. These statistical features were used as input to the neural network for classifying five emotions (disgust, happy, surprise, sad and anger). In the experiments of recognising human emotions from visual and audiovisual stimulus, the average recognition rate of 56.66% and 66.67% is obtained. The experimental result shows that the audiovisual stimulus based emotion recognition gives better classification accuracy over visual stimulus.

[1]  M Adjouadi,et al.  A practical EMG-based human-computer interface for users with motor disabilities. , 2000, Journal of rehabilitation research and development.

[2]  Shaogang Gong,et al.  Beyond Facial Expressions: Learning Human Emotion from Body Gestures , 2007, BMVC.

[3]  Zhou Jianzhong,et al.  Recognition and analyses of EEG & ERP signals related to emotion: from the perspective of psychology , 2005, Proceedings. 2005 First International Conference on Neural Interface and Control, 2005..

[4]  F. Babiloni,et al.  Adaptive brain interfaces for physically-disabled people , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[5]  J.R. LaCourse,et al.  Brainiac: a brain computer link , 1995, Proceedings of 1995 IEEE Instrumentation and Measurement Technology Conference - IMTC '95.

[6]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[7]  O. Rioul,et al.  Wavelets and signal processing , 1991, IEEE Signal Processing Magazine.

[8]  M. Stokes,et al.  Cognitive tasks for driving a brain-computer interfacing system: a pilot study , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  K. Takahashi,et al.  Remarks on hands-free manipulation system using simple brain-computer interface , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[10]  Chris A. Russel,et al.  Selecting Salient Features of Psychophysiological Measures , 2001 .

[11]  Minoru Sasaki,et al.  Brain-wave bio potentials based mobile robot control: wavelet-neural network pattern recognition approach , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[12]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Léon J. M. Rothkrantz,et al.  Emotion recognition using brain activity , 2008, CompSysTech.

[14]  Zhong Ji,et al.  Multi-resolution time-frequency analysis for detection of rhythms of EEG signals , 2004, 3rd IEEE Signal Processing Education Workshop. 2004 IEEE 11th Digital Signal Processing Workshop, 2004..

[15]  M. Murugappan,et al.  Time-Frequency Analysis of EEG Signals for Human Emotion Detection , 2008 .

[16]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[17]  Toshio Tsuji,et al.  A human-assisting manipulator teleoperated by EMG signals and arm motions , 2003, IEEE Trans. Robotics Autom..

[18]  Charles W. Anderson,et al.  Classification of EEG Signals from Four Subjects During Five Mental Tasks , 2007 .

[19]  C. Parameswariah,et al.  Frequency Characteristics of Wavelets , 2002, IEEE Power Engineering Review.

[20]  S. Yaacob,et al.  Lifting scheme for human emotion recognition using EEG , 2008, 2008 International Symposium on Information Technology.

[21]  Wei-Xing He,et al.  Nonlinear Feature Extraction of Sleeping EEG Signals , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[22]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[23]  J. Wolpaw,et al.  Multichannel EEG-based brain-computer communication. , 1994, Electroencephalography and clinical neurophysiology.

[24]  G. Pfurtscheller,et al.  Conversion of EEG activity into cursor movement by a brain-computer interface (BCI) , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Joe-Air Jiang,et al.  Wavelet-Based Processing and Adaptive Fuzzy Clustering For Automated Long-Term Polysomnography Analysis , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[26]  R. Palaniappan,et al.  Brain Computer Interface Design Using Band Powers Extracted During Mental Tasks , 2005, Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering, 2005..

[27]  Elena L. Glassman,et al.  A wavelet-like filter based on neuron action potentials for analysis of human scalp electroencephalographs , 2005, IEEE Transactions on Biomedical Engineering.

[28]  Thierry Pun,et al.  Valence-arousal evaluation using physiological signals in an emotion recall paradigm , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[29]  James L. McClelland.  Explorations In Parallel Distributed Processing , 1988 .

[30]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[31]  Mehran Jahed,et al.  Real-time intelligent pattern recognition algorithm for surface EMG signals , 2007, Biomedical engineering online.

[32]  K. Takahashi,et al.  Remarks on multimodal nonverbal interface and its application to controlling mobile robot , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[33]  Fusheng Yang,et al.  Imagined hand movement identification based on spatio-temporal pattern recognition of EEG , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[34]  Ming-Hu Ha,et al.  Applications of wavelet transform in medical image processing , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).