A Globally Generalized Emotion Recognition System Involving Different Physiological Signals

Machine learning approaches for human emotion recognition have recently demonstrated high performance. However, only/mostly for subject-dependent approaches, in a variety of applications like advanced driver assisted systems, smart homes and medical environments. Therefore, now the focus is shifted more towards subject-independent approaches, which are more universal and where the emotion recognition system is trained using a specific group of subjects and then tested on totally new persons and thereby possibly while using other sensors of same physiological signals in order to recognize their emotions. In this paper, we explore a novel robust subject-independent human emotion recognition system, which consists of two major models. The first one is an automatic feature calibration model and the second one is a classification model based on Cellular Neural Networks (CNN). The proposed system produces state-of-the-art results with an accuracy rate between 80% and 89% when using the same elicitation materials and physiological sensors brands for both training and testing and an accuracy rate of 71.05% when the elicitation materials and physiological sensors brands used in training are different from those used in training. Here, the following physiological signals are involved: ECG (Electrocardiogram), EDA (Electrodermal activity) and ST (Skin-Temperature).

[1]  Mouhannad Ali,et al.  Real-time raindrop detection based on cellular neural networks for ADAS , 2016, Journal of Real-Time Image Processing.

[2]  P C Burns,et al.  E-DISTRACTION: THE CHALLENGES FOR SAFE AND USABLE INTERNET SERVICES IN VEHICLES , 2001 .

[3]  Yuan-Pin Lin,et al.  EEG-Based Emotion Recognition in Music Listening , 2010, IEEE Transactions on Biomedical Engineering.

[4]  Alex Pentland,et al.  A vision system for observing and extracting facial action parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Maya R. Gupta,et al.  Bayesian Quadratic Discriminant Analysis , 2007, J. Mach. Learn. Res..

[6]  John J. B. Allen,et al.  The handbook of emotion elicitation and assessment , 2007 .

[7]  Wanhui Wen,et al.  Emotion Recognition Based on Multi-Variant Correlation of Physiological Signals , 2014, IEEE Transactions on Affective Computing.

[8]  Dario Petri,et al.  Dynamic Compensation of Nonlinear Sensors by a Learning-From-Examples Approach , 2008, IEEE Transactions on Instrumentation and Measurement.

[9]  R. Köhler,et al.  The International Vocabulary of Metrology, 3rd Edition: Basic and General Concepts and Associated Terms. Why? How? , 2010 .

[10]  Karl-Franzens-Universitaet,et al.  SOCIOLOGY OF EMOTIONS , 2011 .

[11]  Christos D. Katsis,et al.  Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[12]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[13]  Ulrich Büker,et al.  Object recognition in image sequences with cellular neural networks , 2000, Neurocomputing.

[14]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[15]  P. Lang The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[16]  Mika P. Tarvainen,et al.  Kubios HRV - Heart rate variability analysis software , 2014, Comput. Methods Programs Biomed..

[17]  Bruno Verschuere,et al.  Psychopathy and Physiological Detection of Concealed Information: A review , 2006 .

[18]  Kyandoghere Kyamakya,et al.  A Novel Real-Time Emotion Detection System for Advanced Driver Assistance Systems , 2012, Autonomous Systems: Developments and Trends.

[19]  Gonzalo Bailador,et al.  A Stress-Detection System Based on Physiological Signals and Fuzzy Logic , 2011, IEEE Transactions on Industrial Electronics.

[20]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[21]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[22]  Wen Wan-Hui,et al.  Electrocardiography Recording, Feature Extraction and Classification for Emotion Recognition , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[23]  Jonghwa Kim,et al.  Bimodal Emotion Recognition using Speech and Physiological Changes , 2007 .

[24]  C. Maaoui,et al.  Emotion Recognition through Physiological Signals for Human-Machine Communication , 2010 .

[25]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[26]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[27]  C. Cryer,et al.  The Numerical Solution of Volterra Functional Di erential Equations by Euler''s Method , 1972 .

[28]  Regan L. Mandryk,et al.  Using psychophysiological techniques to measure user experience with entertainment technologies , 2006, Behav. Inf. Technol..

[29]  Murat Kaya Yapici,et al.  Intelligent Medical Garments with Graphene-Functionalized Smart-Cloth ECG Sensors , 2017, Sensors.

[30]  A. K. Rigler,et al.  Accelerating the convergence of the back-propagation method , 1988, Biological Cybernetics.

[31]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[32]  A. Schaefer,et al.  Please Scroll down for Article Cognition & Emotion Assessing the Effectiveness of a Large Database of Emotion-eliciting Films: a New Tool for Emotion Researchers , 2022 .

[33]  M. Benedek,et al.  Decomposition of skin conductance data by means of nonnegative deconvolution , 2010, Psychophysiology.

[34]  Dana Kulic,et al.  Affective State Estimation for Human–Robot Interaction , 2007, IEEE Transactions on Robotics.

[35]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[36]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[38]  W. Watts Relative persistence of opinion change induced by active compared to passive participation. , 1967, Journal of personality and social psychology.

[39]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[40]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[41]  Weizhong Yan,et al.  Toward Automatic Time-Series Forecasting Using Neural Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Mantas Lukosevicius,et al.  A Practical Guide to Applying Echo State Networks , 2012, Neural Networks: Tricks of the Trade.

[43]  Kyandoghere Kyamakya,et al.  A Driver State Detection System—Combining a Capacitive Hand Detection Sensor With Physiological Sensors , 2017, IEEE Transactions on Instrumentation and Measurement.

[44]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[45]  Leon O. Chua,et al.  Cellular neural networks: applications , 1988 .

[46]  Fadi Al Machot,et al.  CNN Based Subject-Independent Driver Emotion Recognition System Involving Physiological Signals for ADAS , 2016 .

[47]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[48]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[49]  Seulki Lee,et al.  A 3.9 mW 25-Electrode Reconfigured Sensor for Wearable Cardiac Monitoring System , 2011, IEEE Journal of Solid-State Circuits.

[50]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Computing k-Nearest Neighbors , 1975, IEEE Transactions on Computers.

[51]  Kyandoghere Kyamakya,et al.  EEG-based emotion recognition approach for e-healthcare applications , 2016, 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN).

[52]  Elisa Ricci,et al.  Cellular Neural Networks With Virtual Template Expansion for Retinal Vessel Segmentation , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[53]  Ronald Rousseau,et al.  Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient , 2003, J. Assoc. Inf. Sci. Technol..

[54]  David J. Hand,et al.  Measuring classifier performance: a coherent alternative to the area under the ROC curve , 2009, Machine Learning.