Backchannels revisited from a multimodal perspective

In this study, we analyze the role of several linguistic cues (prosodic units, pitch contours, discourse markers, morphological categories, and gaze direction) in French turn-taking face-to-face interactions. Specifically, we investigate vocal and gestural backchannel signals (BCs) produced by a recipient to show his active listening. We show that some particular pitch contours and discursive markers play a systematic role in inducing both gestural and vocal BCs. Conversely, morphological categories and gestural cues rather play a role for gestural BCs.

[1]  R. Espesser,et al.  Le C.I.D. -Corpus of interactional data- : Protocoles, conventions, annotations , 2006 .

[2]  Beatrice Szczepek Reed.  Prosody in conversation , 2007 .

[3]  Jean E. Fox Tree,et al.  Listening in on monologues and dialogues , 1999 .

[4]  Julia Hirschberg,et al.  Prosody and Conversation: An Introduction , 1998 .

[5]  A. Di Cristo,et al.  Le cadre accentuel du français contemporain : essai de modélisation. Première partie , 1999 .

[6]  Kazuko Matsumoto,et al.  Unit linking in conversational Japanese , 2003 .

[7]  Philippe Blache,et al.  Intensive gestures in French and their multimodal correlates , 2007, INTERSPEECH.

[8]  Margret Selting,et al.  The construction of units in conversational talk , 2000, Language in Society.

[9]  Costanza Navarretta,et al.  The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena , 2007, Lang. Resour. Evaluation.

[10]  C. Fougeron,et al.  Realizations of accentual phrase in French intonation , 2002 .

[11]  Nigel G. Ward,et al.  Prosodic features which cue back-channel responses in English and Japanese , 2000 .

[12]  Jean-Marie Marandin.  Contours as constructions , 2007 .

[13]  Johanneke Caspers,et al.  Melodic characteristics of backchannels in Dutch map task dialogues , 2000, INTERSPEECH.

[14]  John J. Gumperz,et al.  Discourse strategies: Prosody in conversation , 1982 .

[15]  Roxane Bertrand,et al.  Some cues about the interactional value of the «continuation» contour in French , 2005 .

[16]  Nigel Ward,et al.  Using prosodic clues to decide when to produce back-channel utterances , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[17]  J. Allwood,et al.  A study of gestural feedback expressions , 2006 .

[18]  R. Espesser,et al.  Contribution to a grammar of intonation in French Form and function of three rising patterns , 2007 .

[19]  Tristan van Rullen,et al.  Vers une analyse syntaxique à granularité variable , 2003 .

[20]  J. Allwood,et al.  A coding scheme for the annotation of feedback , turn management and sequencing phenomena , 2006 .

[21]  A. Ichikawa,et al.  An Analysis of Turn-Taking and Backchannels Based on Prosodic and Syntactic Features in Japanese Map Task Dialogs , 1998, Language and speech.

[22]  Elizabeth Couper-Kuhlen,et al.  Prosody in conversation : interactional studies , 1996 .

[23]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[24]  Emiel Krahmer,et al.  How auditory and visual prosody is used in end-of-utterance detection , 2006, INTERSPEECH.