Discovering Latent Structure in Task-Oriented Dialogues

A key challenge for computational conversation models is to discover latent structure in task-oriented dialogue, since it provides a basis for analysing, evaluating, and building conversational systems. We propose three new unsupervised models to discover latent structures in task-oriented dialogues. Our methods synthesize hidden Markov models (for underlying state) and topic models (to connect words to states). We apply them to two real, non-trivial datasets: human-computer spoken dialogues in bus query service, and humanhuman text-based chats from a live technical support service. We show that our models extract meaningful state representations and dialogue structures consistent with human annotations. Quantitatively, we show our models achieve superior performance on held-out log likelihood evaluation and an ordering task.

[1]  Max Welling,et al.  Distributed Gibbs sampling for latent variable models , 2012 .

[2]  Ruslan Salakhutdinov,et al.  Evaluation methods for topic models , 2009, ICML '09.

[3]  Daniel Marcu,et al.  Bayesian Query-Focused Summarization , 2006, ACL.

[4]  Alan Ritter,et al.  Unsupervised Modeling of Twitter Conversations , 2010, NAACL.

[5]  Maxine Eskénazi,et al.  Spoken Dialog Challenge 2010: Comparison of Live and Control Test Results , 2011, SIGDIAL Conference.

[6]  Srinivas Bangalore,et al.  Learning the Structure of Task-Driven Human-Human Dialogs , 2006, ACL.

[7]  Jason D. Williams,et al.  Estimating Probability of Correctness for ASR N-Best Lists , 2009, SIGDIAL Conference.

[8]  Jordan L. Boyd-Graber,et al.  Mr. LDA: a flexible large scale topic modeling package using variational inference in MapReduce , 2012, WWW.

[9]  Victor Zue,et al.  Dialogue-Oriented Review Summary Generation for Spoken Dialogue Recommendation Systems , 2010, NAACL.

[10]  Thomas L. Griffiths,et al.  A fully Bayesian approach to unsupervised part-of-speech tagging , 2007, ACL.

[11]  Alexander I. Rudnicky,et al.  A texttiling based approach to topic boundary detection in meetings , 2006, INTERSPEECH.

[12]  Johanna D. Moore,et al.  Automatic Segmentation of Multiparty Dialogue , 2006, EACL.

[13]  Eric Fosler-Lussier,et al.  Discourse Segmentation of Multi-Party Conversation , 2003, ACL.

[14]  Oliver Lemon,et al.  Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems , 2009, EACL.

[15]  Elizabeth Shriberg,et al.  Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual , 1997 .

[16]  Oliver Lemon,et al.  Mixture Model POMDPs for Efficient Handling of Uncertainty in Dialogue Management , 2008, ACL.

[17]  Staffan Larsson,et al.  Information state and dialogue management in the TRINDI dialogue move engine toolkit , 2000, Natural Language Engineering.

[18]  Jason D. Williams Challenges and Opportunities for State Tracking in Statistical Spoken Dialog Systems: Results From Two Public Deployments , 2012, IEEE Journal of Selected Topics in Signal Processing.

[19]  David Traum,et al.  The Information State Approach to Dialogue Management , 2003 .

[20]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[21]  Jean Carletta,et al.  Extractive summarization of meeting recordings , 2005, INTERSPEECH.

[22]  Ananlada Chotimongkol,et al.  Learning the Structure of Task-Oriented Conversations from the Corpus of In-Domain Dialogs , 2008 .

[23]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[24]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[25]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[26]  David Griol,et al.  A stochastic finite-state transducer approach to spoken dialog management , 2010, INTERSPEECH.

[27]  Hanna Wallach,et al.  Structured Topic Models for Language , 2008 .

[28]  Steve J. Young,et al.  USING POMDPS FOR DIALOG MANAGEMENT , 2006, 2006 IEEE Spoken Language Technology Workshop.

[29]  Stephen G. Pulman,et al.  Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model , 2009, SIGDIAL Conference.

[30]  M. Kendall A NEW MEASURE OF RANK CORRELATION , 1938 .

[31]  Francis R. Bach,et al.  Online Learning for Latent Dirichlet Allocation , 2010, NIPS.

[32]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[33]  Noah A. Smith,et al.  Predicting Response to Political Blog Posts with Topic Models , 2009, NAACL.

[34]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Thomas L. Griffiths,et al.  Unsupervised Topic Modelling for Multi-Party Spoken Discourse , 2006, ACL.

[36]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[37]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[38]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[39]  Andrew McCallum,et al.  Polylingual Topic Models , 2009, EMNLP.

[40]  Kallirroi Georgila,et al.  Verbal indicators of psychological distress in interactive dialogue with a virtual human , 2013, SIGDIAL Conference.

[41]  Regina Barzilay,et al.  Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization , 2004, NAACL.