Emotion Recognition by Heart Rate Variability

Background: Emotion plays an important role when people face difficult social problems in their daily activities. This study explores the application of sensors and mobile technologies to detect and recognize school bullying. Many databases offer data for emotion recognition research. One of these is the Mahnob-HCI-Tagging database, which yields a baseline accuracy for emotion recognition based on EEG, eye gaze, and a combination of EEG and eye gaze. Because EEG and eye gaze are not suitable for emotion recognition in the mobile gadget environment, it is interesting to investigate other physiological signals individually, such as ECG, galvanic skin conductance (GSR), body temperature and respiration rate. Objective: This paper focussed on the ECG signal and, more specifically, on heart rate variability (HRV), derived from ECG, to identify certain standard features used in emotion recognition. Instead of using discrete emotions as labels, we transferred emotions, such as fear, anger, happiness and anxiety, to an arousal-valence space. Results:For arousal and valence based on HRV, the baselines are 47.69 and 42.55, respectively, while those for arousal and valence in all physiological signals were 46.2 and 45.5, respectively. The most challenging label in this experiment turned out to be #65533neutral#65533 in the valence scale, as the SVM classified all results as either #65533unpleasant#65533 or #65533pleasant#65533. Conclusion: This work provided a baseline for emotion recognition research based on ECG signals. It also encourages experimental trials using GSR, body temperature and respiration rate individually.

[1]  Antonella Brighi,et al.  The emotional impact of bullying and cyberbullying on victims: a European cross-national study. , 2012, Aggressive behavior.

[2]  Pierre-Yves Oudeyer,et al.  The production and recognition of emotions in speech: features and algorithms , 2003, Int. J. Hum. Comput. Stud..

[3]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[4]  Marc Escalona Mena.  Emotion recognition from speech signals , 2012 .

[5]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[6]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[7]  Mark S. Nixon,et al.  Gait Feature Subset Selection by Mutual Information , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[8]  Muhammad Waseem,et al.  Assessment and Management of Bullied Children in the Emergency Department , 2013, Pediatric emergency care.

[9]  Giovanna Castellano,et al.  Biologically inspired emotion recognition from speech , 2011, EURASIP J. Adv. Signal Process..

[10]  J. Mcnames,et al.  Accuracy of ultra-short heart rate variability measures , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[11]  Bhumika Chandrakar,et al.  A SURVEY OF NOISE REMOVAL TECHNIQUES FOR ECG SIGNALS , 2013 .

[12]  Chiun-Li Chin,et al.  The emotion recognition system with Heart Rate Variability and facial image features , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[13]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[14]  Ian B Hickie,et al.  Heart rate variability is associated with emotion recognition: direct evidence for a relationship between the autonomic nervous system and social cognition. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[15]  Say Wei Foo,et al.  Speech emotion recognition using hidden Markov models , 2003, Speech Commun..

[16]  Albino Nogueiras,et al.  Speech emotion recognition using hidden Markov models , 2001, INTERSPEECH.

[17]  Hung-Wen Chiu,et al.  Frequency-domain heart rate variability analysis performed by digital filters , 2010, 2010 Computing in Cardiology.

[18]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[19]  J. Sztajzel.  Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system. , 2004, Swiss medical weekly.

[20]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[21]  M. Borg,et al.  The Emotional Reactions of School Bullies and their Victims , 1998 .

[22]  Jukka Kortelainen,et al.  EEG-based recognition of video-induced emotions: Selecting subject-independent feature set , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[23]  Simon C. Hunter,et al.  The Influence of Emotional Reaction on Help Seeking by Victims of School Bullying , 2006 .

[24]  Oudeyer Pierre-Yves,et al.  The production and recognition of emotions in speech: features and algorithms , 2003 .