A framework for improving error detection and correction in spoken dialog systems

Despite the recent improvements in performance and reliably of the different components of dialog systems, it is still crucial to devise strategies to avoid error propagation from one another. In this paper, we contribute a framework for improved error detection and correction in spoken conversational interfaces. The framework combines user behavior and error modeling to estimate the probability of the presence of errors in the user utterance. This estimation is forwarded to the dialog manager and used to compute whether it is necessary to correct possible errors. We have designed an strategy differentiating between the main misunderstanding and non-understanding scenarios, so that the dialog manager can provide an acceptable tailored response when entering the error correction state. As a proof of concept, we have applied our proposal to a customer support dialog system. Our results show the appropriateness of our technique to correctly detect and react to errors, enhancing the system performance and user satisfaction.

[1]  R. McCrae,et al.  An introduction to the five-factor model and its applications. , 1992, Journal of personality.

[2]  Gökhan Tür,et al.  Beyond ASR 1-best: Using word confusion networks in spoken language understanding , 2006, Comput. Speech Lang..

[3]  Joel R. Tetreault,et al.  Using system and user performance features to improve emotion detection in spoken tutoring dialogs , 2006, INTERSPEECH.

[4]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[5]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[6]  Hui Ye,et al.  Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System , 2007, NAACL.

[7]  Sebastian Möller,et al.  Sequential classifiers for the prediction of user judgments about spoken dialog systems , 2010, Speech Commun..

[8]  Ye-Yi Wang,et al.  Is word error rate a good indicator for spoken language understanding accuracy , 2003, 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721).

[9]  Seiichi Nakagawa,et al.  Detection and recognition of correction utterance in spontaneously spoken dialog , 2003, INTERSPEECH.

[10]  Shrikanth S. Narayanan,et al.  Analysis of user behavior under error conditions in spoken dialogs , 2002, INTERSPEECH.

[11]  David Traum,et al.  The Error Is the Clue: Breakdown In Human-Machine Interaction , 2006 .

[12]  J. Gratch,et al.  The Oxford Handbook of Affective Computing , 2014 .

[13]  Mari Ostendorf,et al.  Error-correction detection and response generation in a spoken dialogue system , 2005, Speech Commun..

[14]  Fangju Wang,et al.  Modeling user behavior online for disambiguating user input in a spoken dialogue system , 2013, Speech Commun..

[15]  Steve Young,et al.  Statistical User Simulation with a Hidden Agenda , 2007, SIGDIAL.

[16]  David Griol,et al.  AN AUTOMATIC DIALOG SIMULATION TECHNIQUE TO DEVELOP AND EVALUATE INTERACTIVE CONVERSATIONAL AGENTS , 2013, Appl. Artif. Intell..

[17]  Diane Horton,et al.  Repairing conversational misunderstandings and non-understandings , 1994, Speech Communication.

[18]  Laurent Karsenty,et al.  Transparency strategies to help users handle system errors , 2005, Speech Commun..

[19]  David Griol,et al.  The Conversational Interface: Talking to Smart Devices , 2016 .

[20]  Michael Picheny,et al.  Using semantic analysis to improve speech recognition performance , 2005, Comput. Speech Lang..

[21]  Gabriel Skantze,et al.  Exploring human error recovery strategies: Implications for spoken dialogue systems , 2005, Speech Communication.

[22]  Nina Dethlefs,et al.  Hierarchical reinforcement learning for situated natural language generation , 2014, Natural Language Engineering.

[23]  Tetsuya Ogata,et al.  Dynamic help generation by estimating user²s mental model in spoken dialogue systems , 2006, INTERSPEECH.

[24]  Fernando Fernández Martínez,et al.  A satisfaction-based model for affect recognition from conversational features in spoken dialog systems , 2013, Speech Commun..

[25]  Björn Schuller,et al.  Computational Paralinguistics , 2013 .

[26]  Elmar Nöth,et al.  A Taxonomy of Applications that Utilize Emotional Awareness , 2006 .

[27]  Roberto Pieraccini,et al.  Automating spoken dialogue management design using machine learning: An industry perspective , 2008, Speech Commun..

[28]  Gary Geunbae Lee,et al.  Hybrid approach to robust dialog management using agenda and dialog examples , 2010, Comput. Speech Lang..

[29]  David Griol,et al.  A statistical simulation technique to develop and evaluate conversational agents , 2013, AI Commun..

[30]  Evgeny A. Stepanov,et al.  The Development of the Multilingual LUNA Corpus for Spoken Language System Porting , 2014, LREC.

[31]  Stefan Ultes,et al.  Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts - And how it relates to user satisfaction , 2015, Speech Commun..

[32]  Hua Ai,et al.  Comparing Spoken Dialog Corpora Collected with Recruited Subjects versus Real Users , 2007, SIGDIAL.

[33]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[34]  Ramón López-Cózar,et al.  Using knowledge of misunderstandings to increase the robustness of spoken dialogue systems , 2010, Knowl. Based Syst..

[35]  Sven Hellbach,et al.  Imitating Dialog Strategies Under Uncertainty , 2014, IHCI.

[36]  Jason D. Williams,et al.  The best of both worlds: unifying conventional dialog systems and POMDPs , 2008, INTERSPEECH.

[37]  Maxine Eskénazi,et al.  Spoken Dialog Challenge 2010 , 2010, 2010 IEEE Spoken Language Technology Workshop.

[38]  Timothy Bickmore,et al.  Some Novel Aspects of Health Communication from a Dialogue Systems Perspective , 2004, AAAI Technical Report.

[39]  Ramón López-Cózar,et al.  A domain-independent statistical methodology for dialog management in spoken dialog systems , 2014, Comput. Speech Lang..

[40]  Roberto Pieraccini,et al.  User modeling for spoken dialogue system evaluation , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[41]  Kallirroi Georgila,et al.  Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems , 2005, SIGDIAL.

[42]  Renato De Mori,et al.  Multiple resolution analysis for robust automatic speech recognition , 2006, Comput. Speech Lang..