Toward computers that recognize and respond to user emotion

For a long time emotions have been kept out of the deliberate tools of science; scientists have expressed emotion, but no tools could sense and respond to their affective information. This paper highlights research at the MIT Media Laboratory aimed at giving computers the ability to comfortably sense, recognize, and respond to the human communication of emotion, especially affective states such as frustration, confusion, interest, distress, anger, and joy, Two main themes of sensing--self-report and concurrent expression--are described, together with examples of systems that give users new ways to communicate emotion to computers and, through computers, to other people. In addition to building systems that try to elicit and detect frustration, our research group has built a system that responds to user frustration in a way that appears to help alleviate it. This paper highlights applications of this research to interface design, wearable computing, entertainment, and education, and briefly presents some potential ethical concerns and how they might be addressed.

[1]  Rosalind W. Picard,et al.  Expression glasses: a wearable device for facial expression recognition , 1999, CHI Extended Abstracts.

[2]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[3]  C. Pollard,et al.  Center for the Study of Language and Information , 2022 .

[4]  Jennifer Healey,et al.  Digital processing of affective signals , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[5]  Rosalind W. Picard,et al.  Signal processing for recognition of human frustration , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[6]  Jennifer Healey,et al.  A New Affect-Perceiving Interface and Its Application to Personalized Music Selection , 1998 .

[7]  Teresa Marrin Nakra,et al.  Inside the conductor's jacket: analysis, interpretation and musical synthesis of expressive gesture , 2000 .

[8]  Jonathan T. Klein,et al.  Computer response to user frustration , 1998 .

[9]  Jonathan Klein,et al.  This computer responds to user frustration , 1999, CHI Extended Abstracts.

[10]  Teresa Marrin,et al.  Analysis of Affective Musical Expression With the Conductors Jacket , 2002 .

[11]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[12]  K. Blocher,et al.  Affective social quest (ASQ) : teaching emotion recognition with interactive media & wireless expressive toys , 1999 .

[13]  Rosalind W. Picard.  Affective Computing , 1997 .

[14]  J Healey,et al.  Quantifying driver stress: developing a system for collecting and processing bio-metric signals in natural situations. , 1999, Biomedical sciences instrumentation.

[15]  Jonathan Klein,et al.  Frustrating the user on purpose: using biosignals in a pilot study to detect the user's emotional state , 1998, CHI Conference Summary.

[16]  Wendy S. Ark,et al.  The Emotion Mouse , 1999, HCI.

[17]  T.B. Sheridan,et al.  Community dialog technology , 1975, Proceedings of the IEEE.

[18]  L. Cañamero.  Emotional and intelligent II : the tangled knot of social cognition : papers from the 2001 AAAI Fall Symposium, November 2-4, North Falmouth, Massachusetts , 2001 .

[19]  Jennifer Healey,et al.  Affective wearables , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[20]  Jennifer Healey,et al.  StartleCam: a cybernetic wearable camera , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[21]  D. Kirsch.  The Sentic Mouse: Developing a tool for Measuring Emotional Valence , 1997 .

[22]  Matthew Rowan Norwood.  Affective feedback devices for continuous usability assessment , 2000 .

[23]  Hans-Jörg Bullinger,et al.  Human-computer interaction : ergonomics and user interfaces , 1999 .

[24]  D. Goleman.  Emotional Intelligence. New York (Bantam) 1995. , 1995 .

[25]  Juan David Velásquez,et al.  Modeling Emotions and Other Motivations in Synthetic Agents , 1997, AAAI/IAAI.