Single-Model Multi-domain Dialogue Management with Deep Learning

We present a Deep Learning approach to dialogue management for multiple domains. Instead of training multiple models (e.g. one for each domain), we train a single domain-independent policy network that is applicable to virtually any information-seeking domain. We use the Deep Q-Network algorithm to train our dialogue policy, and evaluate against simulated and paid human users. The results show that our algorithm outperforms previous approaches while being more practical and scalable.

[1]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[2]  David Vandyke,et al.  Policy committee for adaptation in multi-domain spoken dialogue systems , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[3]  Mari Ostendorf,et al.  Domain Adaptation with Unlabeled Data for Dialog Act Tagging , 2010 .

[4]  Sungjin Lee,et al.  Task Lineages: Dialog State Tracking for Flexible Interaction , 2016, SIGDIAL Conference.

[5]  Maxine Eskénazi,et al.  Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning , 2016, SIGDIAL Conference.

[6]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[7]  Yannis Stylianou,et al.  Learning Domain-Independent Dialogue Policies via Ontology Parameterisation , 2015, SIGDIAL Conference.

[8]  Dilek Z. Hakkani-Tür,et al.  MODEL ADAPTATION FOR DIALOG ACT TAGGING , 2006, 2006 IEEE Spoken Language Technology Workshop.

[9]  Marilyn A. Walker,et al.  Individual and Domain Adaptation in Sentence Planning for Dialogue , 2007, J. Artif. Intell. Res..

[10]  Heidi Christensen,et al.  Knowledge transfer between speakers for personalised dialogue management , 2015, SIGDIAL Conference.

[11]  Matthieu Geist,et al.  Co-adaptation in Spoken Dialogue Systems , 2012, Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice.

[12]  David Vandyke,et al.  PyDial: A Multi-domain Statistical Dialogue System Toolkit , 2017, ACL.

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Seunghak Yu,et al.  Scaling up deep reinforcement learning for multi-domain dialogue systems , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[15]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[16]  Kallirroi Georgila,et al.  EVALUATING EFFECTIVENESS AND PORTABILITY OF REINFORCEMENT LEARNED DIALOGUE STRATEGIES WITH REAL USERS: THE TALK TOWNINFO EVALUATION , 2006, 2006 IEEE Spoken Language Technology Workshop.

[17]  Lu Chen,et al.  Semantic parser enhancement for dialogue domain extension with little data , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[18]  Dongho Kim,et al.  Distributed dialogue policies for multi-domain statistical dialogue management , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).