Analyzing dialogue breakdowns in chat-oriented dialogue systems

Although there has been much analysis of dialogue breakdown in task-oriented dialogue systems, little attention has been paid to the same issue in chat-oriented dialogue systems, making it difficult to produce improvements. We have therefore analyzed dialogue breakdown, which we define as problematic situations in which conversational participants cannot proceed with the dialogue, in chat-oriented dialogue systems. Specifically, we collected dialogue data and annotated dialogue breakdowns with many annotators and then analyzed the system utterances that led to such breakdowns in detail. Our manual and automated analysis revealed the possible causes of dialogue breakdown in chat-oriented dialogue systems.

[1]  Ryuichiro Higashinaka,et al.  Towards an open-domain conversational system fully based on natural language processing , 2014, COLING.

[2]  Kristiina Jokinen,et al.  WikiTalk human-robot interactions , 2013, ICMI '13.

[3]  Gary Geunbae Lee,et al.  Example-based chat-oriented dialogue system with personalized long-term memory , 2015, 2015 International Conference on Big Data and Smart Computing (BIGCOMP).

[4]  Kanako Onishi,et al.  Achieving Natural Dialog with Computers , 2014 .

[5]  Haizhou Li,et al.  IRIS: a Chat-oriented Dialogue System based on the Vector Space Model , 2012, ACL.

[6]  Justine Cassell,et al.  Relational agents: a model and implementation of building user trust , 2001, CHI.

[7]  Sebastian Möller,et al.  Analysis of communication failures for spoken dialogue systems , 2007, INTERSPEECH.

[8]  Ellen M. Voorhees,et al.  Overview of TREC 2001 , 2001, TREC.

[9]  Kugatsu Sadamitsu,et al.  Unsupervised Clustering of Utterances Using Non-Parametric Bayesian Methods , 2011, INTERSPEECH.

[10]  Tim Paek,et al.  Toward a Taxonomy of Communication Errors , 2003 .

[11]  M. Rey.  The Error Is the Clue: Breakdown In Human-Machine Interaction , 2003 .

[12]  Carole Lailler,et al.  Human annotation of ASR error regions: Is "gravity" a sharable concept for human annotators? , 2014, LREC.

[13]  Alexander I. Rudnicky,et al.  Sorry and I Didn’t Catch That! - An Investigation of Non-understanding Errors and Recovery Strategies , 2005, SIGDIAL.

[14]  Antoine Raux,et al.  The Dialog State Tracking Challenge , 2013, SIGDIAL Conference.

[15]  Niels Ole Bernsen,et al.  Principles for the design of cooperative spoken human-machine dialogue , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[16]  Yang Xiang,et al.  Problematic Situation Analysis and Automatic Recognition for Chinese Online Conversational System , 2014, CIPS-SIGHAN.

[17]  Ryuichiro Higashinaka,et al.  Evaluating coherence in open domain conversational systems , 2014, INTERSPEECH.

[18]  Richard S. Wallace,et al.  The Anatomy of A.L.I.C.E. , 2009 .

[19]  Siobhan Chapman.  Logic and Conversation , 2005 .