Unobtrusive Sensing of Emotions (USE)

Emotions are acknowledged as a crucial element for artificial intelligence; this is, as is illustrated, no different for Ambient Intelligence (AmI). Unobtrusive Sensing of Emotions (USE) is introduced to enrich AmI with empathic abilities. USE coins the combination of speech and the electrocardiogram (ECG) as a powerful and unique combination to unravel people's emotions. In a controlled study, 40 people watched film scenes, in either an office or a home-like setting. It is shown that, when people's gender is taken into account, both heart rate variability (derived from the ECG) and the standard deviation of the fundamental frequency of speech indicate people's experienced valence and arousal, in parallel. As such, both measures validate each other. Thus, through USE reliable cues can be derived that indicate the emotional state of people, in particular when also people's environment is taken into account. Since all this is crucial for both AI and true AmI, this study provides a first significant leap forward in making AmI a success.

[1]  J. Stainer,et al.  The Emotions , 1922, Nature.

[2]  H. Simon,et al.  Motivational and emotional controls of cognition. , 1967, Psychological review.

[3]  B. Malmberg.  Manual of phonetics , 1968 .

[4]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  P. Kleinginna,et al.  A categorized list of emotion definitions, with suggestions for a consensual definition , 1981 .

[6]  P. Strevens.  Iii , 1985 .

[7]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[8]  Ann Daly.  Movement Analysis: Piecing Together the Puzzle , 1988 .

[9]  M. Malik.  Heart Rate Variability , 1996, Clinical cardiology.

[10]  P. Boersma.  ACCURATE SHORT-TERM ANALYSIS OF THE FUNDAMENTAL FREQUENCY AND THE HARMONICS-TO-NOISE RATIO OF A SAMPLED SOUND , 1993 .

[11]  J. Cacioppo,et al.  Relationship between attitudes and evaluative space: A critical review, with emphasis on the separability of positive and negative substrates. , 1994 .

[12]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[13]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[14]  G. Gendolla.  On the Impact of Mood on Behavior: An Integrative Theory and a Review , 2000 .

[15]  S. A. Neumann,et al.  Similar patterns of cardiovascular response during emotional activation as a function of affective valence and arousal and gender. , 2001, Journal of psychosomatic research.

[16]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[17]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[18]  J. Thayer,et al.  The rhythm of the heart in the blink of an eye: emotion-modulated startle magnitude covaries with heart rate variability. , 2003, Psychophysiology.

[19]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[20]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[21]  Emile H. L. Aarts,et al.  Ambient intelligence: a multimedia perspective , 2004, IEEE MultiMedia.

[22]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[23]  E. L. van den Broek.  Emotional Prosody Measurement (EPM): a voice-based evaluation method for psychological therapy effectiveness. , 2004, Studies in health technology and informatics.

[24]  I. Christie,et al.  Autonomic specificity of discrete emotion and dimensions of affective space: a multivariate approach. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[25]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[26]  J. Prinz.  Gut Reactions: A Perceptual Theory of Emotion , 2004 .

[27]  Egon L. van den Broek,et al.  Emotional Prosody Measurement (EPM): a voice-based evaluation method for psychological therapy effectiveness. , 2004 .

[28]  B. Appelhans,et al.  Heart Rate Variability as an Index of Regulated Emotional Responding , 2006 .

[29]  M. Minsky.  The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind , 2006 .

[30]  Constantine Kotropoulos,et al.  Emotional speech recognition: Resources, features, and methods , 2006, Speech Commun..

[31]  A. Damasio,et al.  Basic emotions are associated with distinct patterns of cardiorespiratory activity. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  Graham Clarke,et al.  A user-independent real-time emotion recognition system for software agents in domestic environments , 2007, Eng. Appl. Artif. Intell..

[33]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[34]  桐山 伸也.  "The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind," Marvin Minsky, Simon & Schuster, 2006(私のすすめるこの一冊,コーヒーブレイク) , 2007 .

[35]  M. H. Schut,et al.  Computing emotion awareness through galvanic skin response and facial electromyography , 2008 .

[36]  Allison B. Duke,et al.  Historical foundations of social effectiveness? Dale Carnegie's principles , 2008 .

[37]  Brigitte Krenn,et al.  Fully generated scripted dialogue for embodied agents , 2008, Artif. Intell..

[38]  Maria E. Jabon,et al.  Real-time classification of evoked emotions using facial feature tracking and physiological responses , 2008, Int. J. Hum. Comput. Stud..

[39]  Paul Lukowicz.  Wearable computing and artificial intelligence for healthcare applications , 2008, Artif. Intell. Medicine.

[40]  Rodrigo Ventura,et al.  Responding efficiently to relevant stimuli using an emotion-based agent architecture , 2009, Neurocomputing.

[41]  Hatice Gunes,et al.  Automatic Temporal Segment Detection and Affect Recognition From Face and Body Display , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Michael D. Robinson,et al.  Measures of emotion: A review , 2009, Cognition & emotion.

[43]  Pedro Encarnação,et al.  BIOSIGNALS 2009 - Proceedings of the International Conference on Bio-inspired Systems and Signal Processing, Porto, Portugal, January 14-17, 2009 , 2009, BIOSIGNALS.

[44]  Gwen Littlewort,et al.  Toward Practical Smile Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[47]  Egon L. van den Broek,et al.  Prerequisites for Affective Signal Processing (ASP) , 2009, BIOSIGNALS.

[48]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  S. Tong,et al.  Long-range correlations in heart rate variability during computer-mouse work under time pressure , 2009 .

[50]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.