Quantized Dialog - A general approach for conversational systems

Abstractr We propose Quantized Dialog, a novel approach for the development of conversational systems. The methodology relies on the semantic quantization and clustering of the dialog utterances in order to reduce the dialog interaction space, making prediction of the next utterance more tractable. The effectiveness of this method is showcased using the goal-oriented dataset of the sixth Dialog System Technology Challenge (DSTC6). We compare the performance of Quantized Dialog based on an n-gram language model for next-utterance prediction against other models that employ popular deep-learning architectures, such as multi-layer neural network classifiers, memory networks, long short-term memory recurrent neural networks and convolutional neural networks. The experimental results demonstrate the promising potential of the new quantized approach in goal-oriented dialog prediction.

[1]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[2]  Yoshua Bengio,et al.  Gated Feedback Recurrent Neural Networks , 2015, ICML.

[3]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[4]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[5]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[6]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  David Griol,et al.  The Conversational Interface: Talking to Smart Devices , 2016 .

[9]  Fionn Murtagh,et al.  A Survey of Recent Advances in Hierarchical Clustering Algorithms , 1983, Comput. J..

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[12]  Antoine Raux,et al.  The Dialog State Tracking Challenge , 2013, SIGDIAL Conference.

[13]  Oliver Lemon,et al.  A Simple and Generic Belief Tracking Mechanism for the Dialog State Tracking Challenge: On the believability of observed information , 2013, SIGDIAL Conference.

[14]  Alexander I. Rudnicky,et al.  A “K Hypotheses + Other” Belief Updating Model , 2006 .

[15]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[16]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[17]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[18]  Eric Horvitz,et al.  A computational architecture for conversation , 1999 .

[19]  Joelle Pineau,et al.  Training End-to-End Dialogue Systems with the Ubuntu Dialogue Corpus , 2017, Dialogue Discourse.

[20]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[21]  Daniel Povey,et al.  Revisiting Recurrent Neural Networks for robust ASR , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[23]  R. Real,et al.  The Probabilistic Basis of Jaccard's Index of Similarity , 1996 .

[24]  Joelle Pineau,et al.  A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues , 2016, AAAI.

[25]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[26]  Matthew Henderson,et al.  The third Dialog State Tracking Challenge , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[27]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[28]  S. Singh,et al.  Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System , 2011, J. Artif. Intell. Res..

[29]  Roberto Pieraccini,et al.  Automating spoken dialogue management design using machine learning: An industry perspective , 2008, Speech Commun..

[30]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[31]  Tomoki Toda,et al.  Improving the robustness of example-based dialog retrieval using recursive neural network paraphrase identification , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[32]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[33]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[34]  Rafael E. Banchs,et al.  The Fourth Dialog State Tracking Challenge , 2016, IWSDS.

[35]  Matthew Henderson,et al.  The Second Dialog State Tracking Challenge , 2014, SIGDIAL Conference.

[36]  Jason Weston,et al.  Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks , 2015, ICLR.

[37]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[38]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[39]  Maxine Eskénazi,et al.  LET's GO: improving spoken dialog systems for the elderly and non-natives , 2003, INTERSPEECH.

[40]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[41]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[42]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[43]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[44]  Mikio Nakano,et al.  Corpus-Based Discourse Understanding in Spoken Dialogue Systems , 2003, ACL.