论文信息 - A Deep Reinforcement Learning Chatbot (Short Version) - 字舞流文

A Deep Reinforcement Learning Chatbot (Short Version)

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including neural network and template-based models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than other systems. The results highlight the potential of coupling ensemble systems with deep reinforcement learning as a fruitful path for developing real-world, open-domain conversational agents.

Joelle Pineau | Sandeep Subramanian | Michael Pieper | Yoshua Bengio | Taesup Kim | Saizheng Zhang | Vincent Michalski | A. P. Sarath Chandar | Mathieu Germain | Sai Rajeswar | Alexandre de Brébisson | Iulian Serban | Chinnadhurai Sankar | Zhouhan Lin | Nan Rosemary Ke | Dendi Suhubdy | Alexandre Nguyen | Jose M. R. Sotelo | Yoshua Bengio | Joelle Pineau | A. D. Brébisson | M. Germain | Zhouhan Lin | Vincent Michalski | Iulian Serban | Saizheng Zhang | Sai Rajeswar | A. Chandar | Taesup Kim | Sandeep Subramanian | Jose M. R. Sotelo | Dendi Suhubdy | Chinnadhurai Sankar | Michael Pieper | A. Nguyen

[1] Patrick Seemann,et al. Matrix Factorization Techniques for Recommender Systems , 2014 .

[2] Yehuda Koren,et al. Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[3] Joseph Weizenbaum,et al. and Machine , 1977 .

[4] Zhou Yu,et al. Strategy and Policy Learning for Non-Task-Oriented Conversational Systems , 2016, SIGDIAL Conference.

[5] David Suendermann-Oeft,et al. HALEF: An Open-Source Standard-Compliant Telephony-Based Modular Spoken Dialog System: A Review and An Outlook , 2015, IWSDS.

[6] Joelle Pineau,et al. How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation , 2016, EMNLP.

[7] George Kurian,et al. Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[8] Andrew C. Simpson,et al. Black box and glass box evaluation of the SUNDIAL system , 1993, EUROSPEECH.

[9] R. J. Williams,et al. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[10] Doina Precup,et al. Eligibility Traces for Off-Policy Policy Evaluation , 2000, ICML.

[11] Sanjoy Dasgupta,et al. Off-Policy Temporal Difference Learning with Function Approximation , 2001, ICML.

[12] Joelle Pineau,et al. A Deep Reinforcement Learning Chatbot , 2017, ArXiv.

[13] Norman M. Fraser,et al. Dialogue Management for Telephone Information Systems , 1992, ANLP.

[14] Jennifer Chu-Carroll,et al. Building Watson: An Overview of the DeepQA Project , 2010, AI Mag..

[15] Joelle Pineau,et al. Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses , 2017, ACL.

[16] Alex Graves,et al. Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[17] Richard S. Sutton,et al. Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18] Long-Ji Lin,et al. Reinforcement learning for robots using neural networks , 1992 .

[19] Volker Steinbiss,et al. The Philips automatic train timetable information system , 1995, Speech Commun..