Automatic ECG-Based Emotion Recognition in Music Listening

This paper presents an automatic ECG-based emotion recognition algorithm for human emotion recognition. First, we adopt a musical induction method to induce participants’ real emotional states and collect their ECG signals without any deliberate laboratory setting. Afterward, we develop an automatic ECG-based emotion recognition algorithm to recognize human emotions elicited by listening to music. Physiological ECG features extracted from the time-, and frequency-domain, and nonlinear analyses of ECG signals are used to find emotion-relevant features and to correlate them with emotional states. Subsequently, we develop a sequential forward floating selection-kernel-based class separability-based (SFFS-KBCS-based) feature selection algorithm and utilize the generalized discriminant analysis (GDA) to effectively select significant ECG features associated with emotions and to reduce the dimensions of the selected features, respectively. Positive/negative valence, high/low arousal, and four types of emotions (joy, tension, sadness, and peacefulness) are recognized using least squares support vector machine (LS-SVM) recognizers. The results show that the correct classification rates for positive/negative valence, high/low arousal, and four types of emotion classification tasks are 82.78, 72.91, and 61.52 percent, respectively.

[1]  Christos D. Katsis,et al.  A User Independent, Biosignal Based, Emotion Recognition Method , 2007, User Modeling.

[2]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[3]  T. Seppänen,et al.  Quantitative beat-to-beat analysis of heart rate dynamics during exercise. , 1996, The American journal of physiology.

[4]  Luiz Carlos de Abreu,et al.  The effects of auditory stimulation with music on heart rate variability in healthy women , 2013, Clinics.

[5]  Pablo Laguna,et al.  A robust method for ECG-based estimation of the respiratory frequency during stress testing , 2006, IEEE Transactions on Biomedical Engineering.

[6]  Yeon-Sik Noh,et al.  An improved algorithm for respiration signal extraction from electrocardiogram measured by conductive textile electrodes using instantaneous frequency estimation , 2008, Medical & Biological Engineering & Computing.

[7]  K. H. Kim,et al.  Emotion recognition system using short-term monitoring of physiological signals , 2004, Medical and Biological Engineering and Computing.

[8]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[9]  Enzo Pasquale Scilingo,et al.  Recognizing Emotions Induced by Affective Sounds through Heart Rate Variability , 2015, IEEE Transactions on Affective Computing.

[10]  Luiz Carlos de Abreu,et al.  Symbolic Analysis of Heart Rate Variability During Exposure to Musical Auditory Stimulation. , 2016, Alternative therapies in health and medicine.

[11]  R. Mccraty,et al.  Cardiac coherence: a new, noninvasive measure of autonomic nervous system order. , 1996, Alternative therapies in health and medicine.

[12]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[13]  J. Russell A circumplex model of affect. , 1980 .

[14]  Dimitrios Hatzinakos,et al.  ECG Pattern Analysis for Emotion Detection , 2012, IEEE Transactions on Affective Computing.

[15]  Joyce H. D. M. Westerink,et al.  Directing Physiology and Mood through Music: Validation of an Affective Music Player , 2013, IEEE Transactions on Affective Computing.

[16]  Mika P. Tarvainen,et al.  Software for advanced HRV analysis , 2004, Comput. Methods Programs Biomed..

[17]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Stefan Koelsch,et al.  Med Biol Eng Comput (2010) 48:423–433 DOI 10.1007/s11517-010-0592-3 ORIGINAL ARTICLE , 2022 .

[19]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[20]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[21]  Dana Kulic,et al.  Affective State Estimation for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[22]  Mohsen Naji,et al.  Classification of Music-Induced Emotions Based on Information Fusion of Forehead Biosignals and Electrocardiogram , 2014, Cognitive Computation.

[23]  Yu-Liang Hsu,et al.  ECG arrhythmia classification using a probabilistic neural network with a feature reduction method , 2013, Neurocomputing.

[24]  Conor Heneghan,et al.  Automated processing of the single-lead electrocardiogram for the detection of obstructive sleep apnoea , 2003, IEEE Transactions on Biomedical Engineering.

[25]  C. Krumhansl An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[26]  James McNames,et al.  Reliability and accuracy of heart rate variability metrics versus ECG segment duration , 2006, Medical and Biological Engineering and Computing.

[27]  P. Ekman An argument for basic emotions , 1992 .

[28]  M.A. Mneimneh,et al.  An adaptive kalman filter for removing baseline wandering in ECG signals , 2006, 2006 Computers in Cardiology.

[29]  Enzo Pasquale Scilingo,et al.  The Role of Nonlinear Dynamics in Affective Valence and Arousal Recognition , 2012, IEEE Transactions on Affective Computing.

[30]  S. Galloway,et al.  Effects of central sympathetic inhibition on heart rate variability during steady‐state exercise in healthy humans , 2002, Clinical physiology and functional imaging.

[31]  Lei Wang,et al.  Feature Selection with Kernel Class Separability , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Katarzyna Wac,et al.  Ambulatory Assessment of Affect: Survey of Sensor Systems for Monitoring of Autonomic Nervous Systems Activation in Emotion , 2014, IEEE Transactions on Affective Computing.

[33]  Su-Lim Tan,et al.  A biometric signature based system for improved emotion recognition using physiological responses from multiple subjects , 2010, 2010 8th IEEE International Conference on Industrial Informatics.

[34]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[35]  A. Damasio,et al.  Basic emotions are associated with distinct patterns of cardiorespiratory activity. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[36]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[37]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[38]  Wanhui Wen,et al.  Emotion Recognition Based on Multi-Variant Correlation of Physiological Signals , 2014, IEEE Transactions on Affective Computing.

[39]  Eric C. C. Tsang,et al.  Nesting One-Against-One Algorithm Based on SVMs for Pattern Classification , 2008, IEEE Transactions on Neural Networks.

[40]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[41]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[42]  U. Rajendra Acharya,et al.  Heart rate variability: a review , 2006, Medical and Biological Engineering and Computing.

[43]  Gerhard Tröster,et al.  Modeling arousal phases in daily living using wearable sensors , 2013, IEEE Transactions on Affective Computing.

[44]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[45]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.