Emotion Recognition based on Heart Rate and Skin Conductance

Information on a customer’s emotional states concerning a product or an advertisement is a very important aspect of marketing research. Most studies aimed at identifying emotions through speech or facial expressions. However, these two vary greatly with peoplei¾€i¾™s talking habits, which cause the data lacking continuous availability. Furthermore, bio-signal data is also required in order to fully assess a user’s emotional state in some cases. We focused on recognising the six basic primary emotions proposed by Ekman using biofeedback sensors, which measure heart rate and skin conductance. Participants were shown a series of 12 video-based stimuli that have been validated by a subjective rating protocol. Experiment results showed that the collected signals allow us to identify user's emotional state with a good ratio. In addition, a partial correlation between objective and subjective data has been observed.

[1]  Armando Barreto,et al.  Stress detection in computer users through non-invasive monitoring of physiological signals. , 2006, Biomedical sciences instrumentation.

[2]  R. Levenson.  Emotion and the autonomic nervous system: A prospectus for research on autonomic specificity. , 1988 .

[3]  Hatice Gunes,et al.  Automatic, Dimensional and Continuous Emotion Recognition , 2010, Int. J. Synth. Emot..

[4]  David S. Olton,et al.  Functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[5]  Hamza Hamdi,et al.  Plate-forme multimodale pour la reconnaissance d'émotions via l'analyse de signaux physiologiques : Application à la simulation d'entretiens d'embauche. (Multimodal paltform for emotion recognition based on physiological responses : application in the interviews simulation) , 2012 .

[6]  Zahra Khalili,et al.  Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of EEG , 2009, 2009 International Joint Conference on Neural Networks.

[7]  J. Gray,et al.  Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[8]  Jonghwa Kim,et al.  Bimodal Emotion Recognition using Speech and Physiological Changes , 2007 .

[9]  G. Bower,et al.  Cognitive perspectives on emotion and motivation , 1988 .

[10]  R. Levenson,et al.  Autonomic specificity and emotion. , 2003 .

[11]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[13]  W. Mcdougall.  An introduction to social psychology , 1909 .

[14]  Juliana Doria Bednarski.  Eliciting Seven Discrete Positive Emotions Using Film Stimuli , 2012 .

[15]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[16]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[17]  Thierry Pun,et al.  Valence-arousal evaluation using physiological signals in an emotion recall paradigm , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[18]  Roux.  7 - Transformée de Fourier discrète récursive avec fenêtre exponentielle dissymétrique , 1993 .

[19]  M. Cabanac.  What is emotion? , 2002, Behavioural Processes.

[20]  R. Plutchik.  A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION , 1980 .

[21]  Jaak Panksepp,et al.  Toward a general psychobiological theory of emotions , 1982, Behavioral and Brain Sciences.

[22]  J. Russell.  Affective space is bipolar. , 1979 .

[23]  C. Izard.  The face of emotion , 1971 .

[24]  Ellen Elizabeth Bartolini,et al.  Eliciting Emotion with Film: Development of a Stimulus Set , 2011 .

[25]  Egon L. van den Broek,et al.  Guidelines for affective signal processing (ASP): From lab to life , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[26]  Paul Richard,et al.  Virtual Reality and Affective Computing Techniques for Face-to-face Communication , 2011, GRAPP.

[27]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[28]  D. Thalmann,et al.  Using physiological measures for emotional assessment: A computer-aided tool for cognitive and behavioural therapy , 2004 .

[29]  P. Ekman,et al.  Facial Action Coding System: Manual , 1978 .

[30]  Ó. Gonçalves,et al.  The Emotional Movie Database (EMDB): A Self-Report and Psychophysiological Study , 2012, Applied psychophysiology and biofeedback.

[31]  J. H. Curtis,et al.  Learning Theory and Behavior , 1960 .

[32]  Kai Keng Ang,et al.  EEG-based Emotion Recognition Using Self-Organizing Map for Boundary Detection , 2010, 2010 20th International Conference on Pattern Recognition.

[33]  Aaron Sloman,et al.  Why Robots Will Have Emotions , 1981, IJCAI.

[34]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[35]  Paul Richard,et al.  A Multi-modal Virtual Environment to Train for Job Interview , 2011, PECCS.

[36]  P. Johnson-Laird,et al.  Towards a Cognitive Theory of Emotions , 1987 .