Multi-level Annotations of Nonverbal Behaviors in French Spontaneous Conversation

During spontaneous conversations, multiple modalities such as gesture, posture and gaze are combined in sophisticated ways for different functions such as spatial references. In the French research community, there is a lack of spontaneous multimodal corpora for the French language. This paper describes the multi-level scheme that we have defined for the annotation of gesture, posture and gaze. We explain how we applied it for the annotation of a corpus of spontaneous French conversations. Two types of analyses were made on the resulting annotations. Firstly, intercoder agreement was computed on the annotations of gesture space. Secondly, we clustered chunks of postures into categories. Such research will enable the joint study of multiple nonverbal modalities such as the relations between gesture semantics and posture.

[1]  P. Ekman,et al.  Handbook of methods in nonverbal behavior research , 1982 .

[2]  Candea Maria.  Danielle Bouvet et Mary-Annick Morel, Le ballet et la musique de la parole. Le geste et l’intonation dans le dialogue oral en français, Paris, Ophrys, 2002 , 2010 .

[3]  D. McNeill.  Hand and Mind , 1995 .

[4]  Philippe Blache,et al.  Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project , 2009, Multimodal Corpora.

[5]  S. Schermer.  Human Osteology , 2004 .

[6]  Rachel Panckhurst,et al.  Traitement automatique des langues. , 2001 .

[7]  Sotaro Kita,et al.  What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking , 2003 .

[8]  Danielle Bouvet.  Le corps et la métaphore dans les langues gestuelles, à la recherche des modes de production des signes , 1997 .

[9]  Michael Kipp,et al.  Gesture generation by imitation: from human behavior to computer character animation , 2005 .

[10]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[11]  Philippe Sero Guillaume.  La langue des signes française , 1999 .

[12]  Walter Rohmert,et al.  Influence Parameters and Assessment Methods for Evaluating Body Postures , 1986 .

[13]  R. Espesser,et al.  Le CID - Corpus of Interactional Data. Annotation et exploitation multimodale de parole conversationnelle [The “Corpus of Interactional Data” (CID) - Multimodal annotation of conversational speech”] , 2008, ICON.

[14]  John R. Wilson,et al.  Ergonomics Of Working Postures : Models, Methods And Cases: The Proceedings Of The First International Occupational Ergonomics Symposium, Zadar, Yugoslavia, 15-17 April 1985 , 2003 .

[15]  J. T. M. Kay.  Color Atlas and Textbook of Human Anatomy , 1979 .

[16]  D. McNeill.  Hand and Mind: What Gestures Reveal about Thought , 1992 .

[17]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[18]  Costanza Navarretta,et al.  The MUMIN multimodal coding scheme , 2005 .

[19]  Michael Neff,et al.  An annotation scheme for conversational gestures: how to economically capture timing and form , 2007, Lang. Resour. Evaluation.

[20]  Jean-Claude Martin,et al.  Gesture and emotion: Can basic gestural form features discriminate emotions? , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[21]  Danielle Bouvet,et al.  Le ballet et la musique de la parole : le geste et l'intonation dans le dialogue oral en français , 2002 .

[22]  D. McNeill.  Gesture and Thought , 2005 .

[23]  Clifford Nass,et al.  Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics , 2000, Int. J. Hum. Comput. Stud..

[24]  J. Mccroskey,et al.  Nonverbal Behavior in Interpersonal Relations , 1987 .