Tracing Emotion: An Overview

Computational research with continuous representations depends on obtaining continuous representations from human labellers. The main method used for that purpose is tracing. Tracing raises a range of challenging issues, both psychological and statistical. Naive assumptions about these issues are easy to make, and can lead to inappropriate requirements and uses. The natural function of traces is to capture perceived affect, and as such they belong in long traditions of research on both perception and emotion. Experiments on several types of material provide information about their characteristics, particularly the ratings on which people tend to agree. Disagreement is not necessarily a problem in the technique. It may correctly show that people's impressions of emotion diverge more than commonly thought. A new system, Gtrace, is designed to let rating studies capitalise on a decade of experience and address the research questions that are opened up by the data now available.

[1]  P. Ekman,et al.  Handbook of methods in nonverbal behavior research , 1982 .

[2]  Elisabeth André,et al.  Affect sensing in speech: Studying fusion of linguistic and acoustic features , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[3]  J. Russell.  A circumplex model of affect. , 1980 .

[4]  Aristotle,et al.  The Basic Works of Aristotle , 1941 .

[5]  S. S. Stevens.  The surprising simplicity of sensory metrics. , 1962 .

[6]  R. Reisenzein.  A structuralist reconstruction of Wundt's three-dimensional theory of emotion. , 1992 .

[7]  A. Mehrabian.  Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament , 1996 .

[8]  C. Krumhansl.  An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[9]  Margaret McRorie,et al.  Cross-Cultural Patterns in Dynamic Ratings of Positive and Negative Natural Emotional Behaviour , 2011, PloS one.

[10]  Kostas Karpouzis,et al.  Emotion recognition through facial expression analysis based on a neurofuzzy network , 2005, Neural Networks.

[11]  Richard T. Cauldwell.  WHERE DID THE ANGER GO? THE ROLE OF CONTEXT IN INTERPRETING EMOTION IN SPEECH , 2000 .

[12]  H. Scholsberg.  A scale for the judgment of facial expressions. , 1941 .

[13]  Maja Pantic,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING , 2022 .

[14]  Kostas Karpouzis,et al.  The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data , 2007, ACII.

[15]  J. Russell.  Affective space is bipolar. , 1979 .

[16]  Loïc Kessous,et al.  Multimodal user’s affective state analysis in naturalistic interaction , 2010, Journal on Multimodal User Interfaces.

[17]  J. Gottman,et al.  Marital interaction: physiological linkage and affective exchange. , 1983, Journal of personality and social psychology.

[18]  Roddy Cowie,et al.  Describing the emotional states that are expressed in speech , 2003, Speech Commun..

[19]  R. Reisenzein.  Pleasure-Arousal Theory and the Intensity of Emotions , 1994 .

[20]  Petri Toiviainen,et al.  Modelling the relationships between emotional responses to, and musical content of, music therapy improvisations , 2008 .

[21]  Roddy Cowie,et al.  Emotional speech: Towards a new generation of databases , 2003, Speech Commun..

[22]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[23]  Björn W. Schuller,et al.  Building Autonomous Sensitive Artificial Listeners , 2012, IEEE Transactions on Affective Computing.

[24]  P. Salovey,et al.  Emotional intelligence meets traditional standards for an intelligence , 1999 .

[25]  Gökhan Tür,et al.  Prosody-based automatic segmentation of speech into sentences and topics , 2000, Speech Commun..

[26]  J. Cohn,et al.  Infant Smiling Dynamics and Perceived Positive Emotion , 2008, Journal of nonverbal behavior.

[27]  C. Pelachaud,et al.  Emotion-Oriented Systems: The Humaine Handbook , 2011 .

[28]  David Sander,et al.  A systems approach to appraisal mechanisms in emotion , 2005, Neural Networks.

[29]  R. Levenson,et al.  Continuous measurement of emotion: The affect rating dial. , 2007 .

[30]  Catherine J. Stevens,et al.  Cognition and the temporal arts: Investigating audience response to dance using PDAs that record continuous data during live performance , 2009, Int. J. Hum. Comput. Stud..

[31]  Cindy K. Chung,et al.  Expressive Writing: Connections to Physical and Mental Health , 2011 .

[32]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[33]  Eckart Altenmüller,et al.  EMuJoy: Software for continuous measurement of perceived emotions in music , 2007, Behavior research methods.

[34]  Emery Schubert.  Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space , 1999 .

[35]  Roddy Cowie,et al.  Real life emotions in French and English TV video clips: an integrated annotation protocol combining continuous and discrete approaches , 2006, LREC.

[36]  G. Bower,et al.  Mood effects on person-perception judgments. , 1987, Journal of personality and social psychology.

[37]  W. R. Garner.  The Processing of Information and Structure , 1974 .

[38]  Emery Schubert,et al.  Continuous Self-Report Methods , 1993 .

[39]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[40]  D. Heylen,et al.  Issues in Data Labelling , 2011 .

[41]  S. Paradiso.  The Emotional Brain: The Mysterious Underpinnings of Emotional Life , 1998 .