Creation of a corpus of multimodal spontaneous expressions of emotions in Human-Machine Interaction

This paper presents an experience in laboratory dealing with the constitution of a corpus of multimodal spontaneous expressions of emotions. The originality of this corpus resides in its characteristics (interactions between a virtual actor and humans learning a theater text), in its content (multimodal spontaneous expressions of emotions) and in its two sources of characterization (by the participant and by one of his/her close relation). The corpus collection is part of a study on the fusion of multimodal information (verbal, facial, gestural, postural, and physiological) to improve the detection and characterization of expressions of emotions in human-machine interaction (HMI).

[1]  Ellen Douglas-Cowie.  Multimodal databases of everyday emotion: content and labelling , 2005 .

[2]  Albert Rilliard,et al.  Why and how to control the authentic emotional speech corpora , 2003, INTERSPEECH.

[3]  Wade Junek,et al.  Mind Reading: The Interactive Guide to Emotions , 2007 .

[4]  Nicolas Courty,et al.  Embodied in a Look: Bridging the Gap between Humans and Avatars , 2003, IVA.

[5]  R. Cowie,et al.  A new emotion database: considerations, sources and scope , 2000 .

[6]  Marc Mersiol,et al.  The Effect of Embodied Conversational Agents' Speech Quality on Users' Attention and Emotion , 2005, ACII.

[7]  Florian Schiel,et al.  The SmartKom Multimodal Corpus at BAS , 2002, LREC.

[8]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[9]  P. Ekman.  Pictures of Facial Affect , 1976 .

[10]  Elmar Nöth,et al.  How to find trouble in communication , 2003, Speech Commun..

[11]  Roddy Cowie,et al.  Multimodal databases of everyday emotion: facing up to complexity , 2005, INTERSPEECH.

[12]  EmoTV 1 : Annotation of Real-life Emotions for the Specification of Multimodal Affective Interfaces , 2005 .

[13]  Christian Bouville,et al.  FaceEngine a 3D facial animation engine for real time applications , 2001, Web3D '01.

[14]  Roddy Cowie,et al.  Describing the emotional states that are expressed in speech , 2003, Speech Commun..

[15]  Alex Waibel,et al.  EMOTION-SENSITIVE HUMAN-COMPUTER INTERFACES , 2000 .