Learning to classify the feedback function of head movements in a Danish corpus of first encounters

This paper deals with the automatic classification of feedback by head movement in the Danish NOMCO corpus, a collection of dyadic interactions in which speakers get to know each other for the first time. The results show that by using a combination of features related to the shape of head movements and facial expressions, together with features of the words these gestures are related with, good results (an F-score of 0.72) are achieved in distinguishing head movements used to express feedback from those that serve a different communicative function. Moreover, the distinction between feedback give and feedback elicit, can also be learnt with very good accuracy (an F-score of 0.913), although this result should be taken with caution due to the fact that one of the behaviours is much more dominant than the other in the corpus.

[1]  Louis-Philippe Morency,et al.  A probabilistic multimodal approach for predicting listener backchannels , 2009, Autonomous Agents and Multi-Agent Systems.

[2]  Trevor Darrell,et al.  Contextual recognition of head gestures , 2005, ICMI '05.

[3]  Costanza Navarretta,et al.  Classification of Feedback Expressions in Multimodal Data , 2010, ACL.

[4]  Patrizia Paggio,et al.  Information structure and communicative functions in spoken and multimodal data , 2010 .

[5]  Costanza Navarretta,et al.  Distinguishing the Communicative Functions of Gestures , 2008, MLMI.

[6]  Trevor Darrell,et al.  Head gestures for perceptual interfaces: The role of context in improving recognition , 2007, Artif. Intell..

[7]  P. Boersma.  Praat : doing phonetics by computer (version 5.1.05) , 2009 .

[8]  S. Maynard.  Interactional functions of a nonverbal sign Head movement in japanese dyadic casual conversation , 1987 .

[9]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[10]  Evelyn Z. McClave.  Linguistic functions of head movements in the context of speech , 2000 .

[11]  Michael Kipp,et al.  Gesture generation by imitation: from human behavior to computer character animation , 2005 .

[12]  Costanza Navarretta,et al.  The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena , 2007, Lang. Resour. Evaluation.

[13]  Christian Husodo-Schulz,et al.  Exploring Features and Classifiers for Dialogue Act Segmentation , 2008, MLMI.

[14]  V. Yngve.  On getting a word in edgewise , 1970 .

[15]  Patrizia Paggio.  Information Structure and Pauses in a Corpus of Spoken Danish , 2006, EACL.

[16]  T. Kobayashi,et al.  A conversation robot using head gesture recognition as para-linguistic information , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[17]  Loredana Cerrato,et al.  Investigating Communicative Feedback Phenomena across Languages and Modalities , 2007 .

[18]  U. Hadar,et al.  Head movement during listening turns in conversation , 1985 .

[19]  Dirk Heylen,et al.  On the Contextual Analysis of Agreement Scores , 2009, Multimodal Corpora.

[20]  Steve Renals,et al.  Detecting Action Items in Meetings , 2008, MLMI.

[21]  Joakim Nivre,et al.  On the Semantics and Pragmatics of Linguistic Feedback , 1992, J. Semant..

[22]  Costanza Navarretta,et al.  The NOMCO Multimodal Nordic Resource - Goals and Characteristics , 2010, LREC.

[23]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[24]  Patrizia Paggio,et al.  Annotating Information Structure in a Corpus of Spoken Danish , 2006, LREC.

[25]  Joakim Nivre,et al.  Towards Multimodal Spoken Language Corpora: TransTool and SyncTool , 1998 .