Automating spoken dialogue management design using machine learning: An industry perspective

In designing a spoken dialogue system, developers need to specify the actions a system should take in response to user speech input and the state of the environment based on observed or inferred events, states, and beliefs. This is the fundamental task of dialogue management. Researchers have recently pursued methods for automating the design of spoken dialogue management using machine learning techniques such as reinforcement learning. In this paper, we discuss how dialogue management is handled in industry and critically evaluate to what extent current state-of-the-art machine learning methods can be of practical benefit to application developers who are deploying commercial production systems. In examining the strengths and weaknesses of these methods, we highlight what academic researchers need to know about commercial deployment if they are to influence the way industry designs and practices dialogue management.

[1]  Jason Williams,et al.  Applying POMDPs to Dialog Systems in the Troubleshooting Domain , 2007, HLT-NAACL 2007.

[2]  Jeremy H. Wright,et al.  Automatically Training a Problematic Dialogue Predictor for a Spoken Dialogue System , 2011, J. Artif. Intell. Res..

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[5]  Pascal Poupart,et al.  Factored partially observable Markov decision processes for dialogue management , 2005 .

[6]  Shimei Pan,et al.  Designing and Evaluating an Adaptive Spoken Dialogue System , 2002, User Modeling and User-Adapted Interaction.

[7]  Tim Paek,et al.  The Markov Assumption in Spoken Dialogue Management , 2005, SIGDIAL Workshop.

[8]  Roberto Pieraccini,et al.  Where do we go from here? Research and Commercial Spoken Dialog Systems , 2005, SIGDIAL.

[9]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[10]  James F. Allen,et al.  An architecture for a generic dialogue shell , 2000, Natural Language Engineering.

[11]  Roberto Pieraccini,et al.  Using Markov decision process for learning dialogue strategies , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[12]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[13]  Khalil Sima'an,et al.  Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship , 2006, Computational Linguistics.

[14]  Roberto Pieraccini,et al.  User modeling for spoken dialogue system evaluation , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  David Heckerman,et al.  Decision-theoretic troubleshooting , 1995, CACM.

[17]  R. Rosenfeld,et al.  Two decades of statistical language modeling: where do we go from here? , 2000, Proceedings of the IEEE.

[18]  Eric Horvitz,et al.  Optimizing Automated Call Routing by Integrating Spoken Dialog Models with Queuing Models , 2004, NAACL.

[19]  Alexander I. Rudnicky,et al.  Sorry and I Didn’t Catch That! - An Investigation of Non-understanding Errors and Recovery Strategies , 2005, SIGDIAL.

[20]  David Maxwell Chickering,et al.  Personalizing influence diagrams: applying online learning strategies to dialogue management , 2006, User Modeling and User-Adapted Interaction.

[21]  M. Ferguson,et al.  Automatic Evaluation , 2009 .

[22]  Joelle Pineau,et al.  Spoken Dialogue Management Using Probabilistic Reasoning , 2000, ACL.

[23]  Roberto Pieraccini,et al.  Technical Support Dialog Systems:Issues, Problems, and Solutions , 2007, HLT-NAACL 2007.

[24]  Marilyn A. Walker,et al.  Quantitative and Qualitative Evaluation of Darpa Communicator Spoken Dialogue Systems , 2001, ACL.

[25]  Alexander I. Rudnicky,et al.  Sorry, I Didn’t Catch That! , 2008 .

[26]  Giuseppe Di Fabbrizio,et al.  Prompt selection with reinforcement learning in an AT&t call routing application , 2006, INTERSPEECH.

[27]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[28]  Steve Young,et al.  The statistical approach to the design of spoken dialogue systems , 2003 .

[29]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[30]  J.D. Williams,et al.  Scaling up POMDPs for Dialog Management: The ``Summary POMDP'' Method , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[31]  Joseph Polifroni,et al.  Galaxy-II as an Architecture for Spoken Dialogue Evaluation , 2000, LREC.

[32]  Bert Esselink,et al.  A Practical Guide to Software Localization , 1998 .

[33]  Joel R. Tetreault,et al.  Comparing the Utility of State Features in Spoken Dialogue Using Reinforcement Learning , 2006, NAACL.

[34]  Steve J. Young,et al.  A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies , 2006, The Knowledge Engineering Review.

[35]  Eric A. Hansen,et al.  An Improved Policy Iteration Algorithm for Partially Observable MDPs , 1997, NIPS.

[36]  Baining Guo,et al.  Planning and Acting under Uncertainty: A New Model for Spoken Dialogue System , 2001, UAI.

[37]  Tim Paek Personalizing Influence Diagrams , 2005 .

[38]  Marilyn A. Walker,et al.  An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for Email , 2000, J. Artif. Intell. Res..

[39]  Alexander I. Rudnicky,et al.  Error handling in the RavenClaw dialog management framework , 2005, EMNLP 2005.

[40]  J. Schatztnann,et al.  Effects of the user model on simulation-based learning of dialogue strategies , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[41]  Alexander I. Rudnicky,et al.  Task-based dialog management using an agenda , 2000 .

[42]  Roberto Pieraccini,et al.  Spoken Language Communication with Machines: The Long and Winding Road from Research to Business , 2005, IEA/AIE.

[43]  Bruce Balentine,et al.  How to Build a Speech Recognition Application , 1999 .

[44]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[45]  Salim Roukos,et al.  Free-flow dialog management using forms , 1999, EUROSPEECH.

[46]  Andrew M. Kuhn,et al.  Code Complete , 2005, Technometrics.

[47]  Roberto Pieraccini,et al.  VALUE-BASED OPTIMAL DECISION FOR DIALOG SYSTEMS , 2006, 2006 IEEE Spoken Language Technology Workshop.

[48]  Marilyn A. Walker,et al.  Towards developing general models of usability with PARADISE , 2000, Natural Language Engineering.

[49]  Maxine Eskénazi,et al.  ONLINE SUPERVISED LEARNING OF NON-UNDERSTANDING RECOVERY POLICIES , 2006, 2006 IEEE Spoken Language Technology Workshop.

[50]  Thierry Dutoit,et al.  A probabilistic framework for dialog simulation and optimal strategy learning , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[51]  Roberto Pieraccini,et al.  User Modeling For Spoken Dialogue , 1997 .

[52]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[53]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..