Annotation of negotiation processes in joint-action dialogues

Situated dialogic corpora are invaluable resources for understanding the complex relationship between language, perception, and action as they are based on naturalistic dialogue situations in which the interactants are given shared goals to be accomplished in the real world. In such situations, verbal interactions are intertwined with actions, and shared goals can only be achieved via dynamic negotiation processes based on common ground constructed from discourse history as well as the interactants' knowledge about the status of actions. In this paper, we propose four major dimensions of collaborative tasks that affect the negotiation processes among interactants, and, hence, the structure of the dialogue. Based on a review of available dialogue corpora and annotation manuals, we show that existing annotation schemes so far do not adequately account for the complex dialogue processes in situated task-based scenarios. We illustrate the effects of specific features of a scenario using annotated samples of dialogue taken from the literature as well as our own corpora, and end with a brief discussion of the challenges ahead.

[1]  Craig Martell.  FORM: An Extensible, Kinematically-based Gesture Annotation Scheme , 2002, LREC.

[2]  Michael F. Schober,et al.  Spatial Dialogue between Partners with Mismatched Abilities , 2009, Spatial Language and Dialogue.

[3]  Cecilia E. Ford,et al.  Interaction and grammar: Interactional units in conversation: syntactic, intonational, and pragmatic resources for the management of turns , 1996 .

[4]  James F. Allen,et al.  Tagging Speech Repairs , 1994, HLT.

[5]  C. Raymond Perrault,et al.  Analyzing Intention in Utterances , 1986, Artif. Intell..

[6]  Rod Gardner,et al.  When Listeners Talk: Response Tokens and Listener Stance , 2001 .

[7]  Gwyneth Doherty-Sneddon,et al.  Comparison of Face-to-Face and Video-Mediated Interaction , 1996, Interact. Comput..

[8]  James F. Allen,et al.  Toward Conversational Human-Computer Interaction , 2001, AI Mag..

[9]  Stefan Kopp,et al.  The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation , 2007, Lang. Resour. Evaluation.

[10]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[11]  Christopher A. Dickinson,et al.  Coordinating cognition: The costs and benefits of shared gaze during collaborative search , 2008, Cognition.

[12]  Jan Alexanderssony,et al.  Dialogue acts in VERBMOBIL-2 , 1997 .

[13]  Julia Hirschberg,et al.  Turn-taking and affirmative cue words in task-oriented dialogue , 2009 .

[14]  Matthias Scheutz,et al.  The Indiana “Cooperative Remote Search Task” (CReST) Corpus , 2010, LREC.

[15]  Michael F. Schober,et al.  Speakers, addressees, and frames of reference: Whose effort is minimized in conversations about locations? , 1995 .

[16]  Johanna D. Moore,et al.  Implications for Generating Clarification Requests in Task-Oriented Dialogues , 2005, ACL.

[17]  James F. Allen,et al.  The TRAINS 93 Dialogues , 1995 .

[18]  James F. Allen,et al.  TRAINS as an Embodied Natural Language Dialogue System , 2001 .

[19]  Yiya Chen,et al.  Let’s you do that: Sharing the cognitive burdens of dialogue , 2007 .

[20]  Anders Green,et al.  Developing a ContextualizedMultimodal Corpus for Human-Robot Interaction , 2006, LREC.

[21]  Michael Neff,et al.  An annotation scheme for conversational gestures: how to economically capture timing and form , 2007, Lang. Resour. Evaluation.

[22]  Harry Bunt,et al.  The DIT++ taxanomy for functional dialogue markup , 2009 .

[23]  Justine Cassell,et al.  Knowledge Representation for Generating Locating Gestures in Route Directions , 2009, Spatial Language and Dialogue.

[24]  Staffan Larsson,et al.  Information state and dialogue management in the TRINDI dialogue move engine toolkit , 2000, Natural Language Engineering.

[25]  Gwyneth Doherty-Sneddon,et al.  The Reliability of a Dialogue Structure Coding Scheme , 1997, CL.

[26]  David Traum,et al.  CONVERSATIONAL AGENCY: THE TRAINS-93 DIALOGUE MANAGER , 2007 .

[27]  M. F. Schober.  Spatial perspective-taking in conversation , 1993, Cognition.

[28]  G. Beattie.  Turn-taking and interruption in political interviews: Margaret Thatcher and Jim Callaghan compared and contrasted , 1982 .

[29]  Nina Dethlefs,et al.  Route instructions in map-based human-human and human-computer dialogue: A comparative analysis , 2010, J. Vis. Lang. Comput..

[30]  Ilana Mushin,et al.  Representational issues in annotation: Using the Australian map task corpus to relate prosody and discourse structure , 2001, Speech Commun..

[31]  M. Safarova.  Rises and Falls. Studies in the Semantics and Pragmatics of Intonation , 2001 .

[32]  Thora Tenbrink,et al.  Modelling Illocutionary Structure: Combining Empirical Studies with Formal Model Analysis , 2010, CICLing.

[33]  Craig H. Martell,et al.  Corpus-Based Gesture Analysis: an Extension of the Form Dataset for the Automatic Detection of Phases in a Gesture , 2007, Int. J. Semantic Comput..

[34]  Christopher A. Dickinson,et al.  Coordinating spatial referencing using shared gaze , 2010, Psychonomic bulletin & review.

[35]  Anne H. Anderson,et al.  The DCIEM map task corpus: spontaneous dialogue under sleep deprivation and drug treatment , 1996, ICSLP.

[36]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[37]  S. Brennan,et al.  Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation , 2007 .

[38]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[39]  H. Branigan,et al.  Non-linguistic influences on rates of disfluency in spontaneous speech , 1999 .

[40]  Jonathan Ginzburg,et al.  On the Means for Clarification in Dialogue , 2001, SIGDIAL Workshop.

[41]  D. McNeill.  Language and Gesture: Gesture in action , 2000 .

[42]  James F. Allen,et al.  A Task-Based Evaluation of the TRAINS-95 Dialogue System , 1996, ECAI Workshop on Dialogue Processing in Spoken Language Systems.

[43]  Stephen Isard,et al.  Conversational Games within Dialogue , 1991 .

[44]  Hannes Rieser,et al.  Multi-speaker utterances and co-ordination in task-oriented dialogue , 2006 .

[45]  Dimitra Anastasiou.  A Speech and Gesture Spatial Corpus in Assisted Living , 2012, LREC.

[46]  Staffan Larsson,et al.  Information States and Dialogue Move Engines , 1999, Electron. Trans. Artif. Intell..

[47]  A. Anderson,et al.  The Effects of Visibility on Dialogue and Performance in a Cooperative Problem Solving Task , 1994 .

[48]  Mark G. Core,et al.  Coding Dialogs with the DAMSL Annotation Scheme , 1997 .

[49]  S. Brennan.  Eye gaze cues for coordination in collaborative tasks , 2011 .

[50]  Stanley Peters,et al.  Collaborative activities and multi-tasking in dialogue systems , 2002 .

[51]  Philip R. Cohen,et al.  Referring as a Collaborative Process , 2003 .

[52]  Anne H. Anderson,et al.  The Hcrc Map Task Corpus , 1991 .

[53]  A. Bangerter,et al.  Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue , 2004, Psychological science.

[54]  James F. Allen,et al.  Draft of DAMSL Dialog Act Markup in Several Layers , 2007 .

[55]  Matthew Purver,et al.  Incrementality, Alignment and Shared Utterances , 2004 .

[56]  Abigail Sellen,et al.  Remote Conversations: The Effects of Mediating Talk With Technology , 1995, Hum. Comput. Interact..

[57]  Ipke Wachsmuth,et al.  Introduction: Situated Communication , 2006 .

[58]  H. H. Clark,et al.  Referring as a collaborative process , 1986, Cognition.

[59]  Michael F. Schober,et al.  How addressees affect spatial perspective choice in dialogue , 1998 .

[60]  David McNeill,et al.  Gesture and language dialectic , 2002 .

[61]  Stefanie Shattuck-Hufnagel,et al.  The original ToBI system and the evolution of the ToBI framework , 2003 .

[62]  Kerstin Fischer,et al.  Video conferencing in a transregional research cooperation : Turn − taking in a new medium , 2002 .