Fast Emotion Recognition Based on Single Pulse PPG Signal with Convolutional Neural Network

Physiological signals contain considerable information regarding emotions. This paper investigated the ability of photoplethysmogram (PPG) signals to recognize emotion, adopting a two-dimensional emotion model based on valence and arousal to represent human feelings. The main purpose was to recognize short term emotion using a single PPG signal pulse. We used a one-dimensional convolutional neural network (1D CNN) to extract PPG signal features to classify the valence and arousal. We split the PPG signal into a single 1.1 s pulse and normalized it for input to the neural network based on the personal maximum and minimum values. We chose the dataset for emotion analysis using physiological (DEAP) signals for the experiment and tested the 1D CNN as a binary classification (high or low valence and arousal), achieving the short-term emotion recognition of 1.1 s with 75.3% and 76.2% valence and arousal accuracies, respectively, on the DEAP data.

[1]  Ilia Uma.  physiological signals based human emotion recognition a review , 2014 .

[2]  Seungwan Ryu,et al.  Soft robot review , 2017 .

[3]  Qunfei Zhao,et al.  A Sociable Human-robot Interaction Scheme Based on Body Emotion Analysis , 2019, International Journal of Control, Automation and Systems.

[4]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[5]  H. Prendinger,et al.  Emotion Recognition from Electromyography and Skin Conductance , 2005 .

[6]  V. Ramu Reddy,et al.  Emotion detection and recognition using HRV features derived from photoplethysmogram signals , 2016, ERM4CT@ICMI.

[7]  Ming Qiu,et al.  Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[8]  Xiaodan Zhuang,et al.  Compact unsupervised EEG response representation for emotion recognition , 2014, IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[9]  Liang Dong,et al.  Emotion Recognition from Multiband EEG Signals Using CapsNet , 2019, Sensors.

[10]  J. N. Bassili.  Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[11]  Richard S. Lazarus,et al.  The Relationship Between Autonomic Indicators of Psychological Stress: Heart Rate and Skin Conductance , 1963 .

[12]  Dimitrios Tzovaras,et al.  Automatic Recognition of Boredom in Video Games Using Novel Biosignal Moment-Based Features , 2011, IEEE Transactions on Affective Computing.

[13]  Chien-Hung Lin,et al.  Heart Rate Variability Signal Features for Emotion Recognition by Using Principal Component Analysis and Support Vectors Machine , 2016, 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE).

[14]  Bao-Liang Lu,et al.  Emotional state classification from EEG data using machine learning approach , 2014, Neurocomputing.

[15]  Aydin Akan,et al.  Emotion recognition from EEG signals by using multivariate empirical mode decomposition , 2018, Pattern Analysis and Applications.

[16]  Giancarlo Fortino,et al.  Human emotion recognition using deep belief network architecture , 2019, Inf. Fusion.

[17]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[18]  Elmar Nöth,et al.  Real-time Recognition of the Affective User State with Physiological Signals , 2022 .

[19]  Hyeoncheol Kim,et al.  Emotion extraction based on multi bio-signal using back-propagation neural network , 2018, Multimedia Tools and Applications.

[20]  Fakhri Karray,et al.  Survey on speech emotion recognition: Features, classification schemes, and databases , 2011, Pattern Recognit..

[21]  Atefeh Goshvarpour,et al.  Poincaré's section analysis for PPG-based automatic emotion recognition , 2018, Chaos, Solitons & Fractals.

[22]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[23]  J. Russell.  A circumplex model of affect. , 1980 .

[24]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[26]  Olga Sourina,et al.  Real-Time EEG-Based Emotion Recognition and Its Applications , 2011, Trans. Comput. Sci..

[27]  Wei Liu,et al.  Emotion Recognition Using Multimodal Deep Learning , 2016, ICONIP.

[28]  Xianxiang Chen,et al.  Respiration-based emotion recognition with deep learning , 2017, Comput. Ind..

[29]  Yoshua Bengio,et al.  Learning deep physiological models of affect , 2013, IEEE Computational Intelligence Magazine.

[30]  Reda A. El-Khoribi,et al.  Emotion Recognition based on EEG using LSTM Recurrent Neural Network , 2017 .

[31]  Cristian A. Torres-Valencia,et al.  Electric propagation modeling of Deep Brain Stimulation (DBS) using the finite element method (FEM) , 2014, 2014 XIX Symposium on Image, Signal Processing and Artificial Vision.

[32]  Yue Zhou,et al.  Progressive Filtering Approach for Early Human Action Recognition , 2018, International Journal of Control, Automation and Systems.

[33]  Mann Oo. Hay.  Emotion recognition in human-computer interaction , 2012 .

[34]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[35]  Yong Peng,et al.  EEG-based emotion classification using deep belief networks , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[36]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[37]  Konstantinos N. Plataniotis,et al.  EEG-based affect states classification using Deep Belief Networks , 2016, 2016 Digital Media Industry & Academic Forum (DMIAF).

[38]  Chao Li,et al.  Analysis of physiological for emotion recognition with the IRS model , 2016, Neurocomputing.