Learning bi-utterance for multi-turn response selection in retrieval-based chatbots

Multi-turn response selection is essential to retrieval-based chatbots. The task requires multi-turn response selection model to match a response candidate with a conversation context. Existing methods may lose relationship features in the context. In this article, we propose an improved method that extends the learning granularity of the multi-turn response selection model to enhance the model’s ability to learn relationship features of utterances in the context, which is a key to understand a conversation context for multi-turn response selection in retrieval-based chatbots. The experimental results show that our proposed method significantly improves sequential matching network for multi-turn response selection in retrieval-based chatbots.

[1]  Joelle Pineau,et al.  The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems , 2015, SIGDIAL Conference.

[2]  Qun Liu,et al.  Syntax-based Deep Matching of Short Texts , 2015, IJCAI.

[3]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[4]  Zhoujun Li,et al.  Topic Augmented Neural Network for Short Text Conversation , 2016, ArXiv.

[5]  Joelle Pineau,et al.  A Deep Reinforcement Learning Chatbot , 2017, ArXiv.

[6]  Hao Wang,et al.  A Dataset for Research on Short-Text Conversations , 2013, EMNLP.

[7]  Bowen Zhou,et al.  Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation , 2016, AAAI.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Xuan Liu,et al.  Multi-view Response Selection for Human-Computer Conversation , 2016, EMNLP.

[10]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[11]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[12]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Jianfeng Gao,et al.  A Diversity-Promoting Objective Function for Neural Conversation Models , 2015, NAACL.

[15]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Report , 1999, TREC.

[16]  Ani Nenkova,et al.  Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, NAACL 2016.

[17]  Rudolf Kadlec,et al.  Improved Deep Learning Baselines for Ubuntu Corpus Dialogs , 2015, ArXiv.

[18]  Rui Yan,et al.  Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System , 2016, SIGIR.

[19]  Bo Chen,et al.  Mechanism-Aware Neural Machine for Dialogue Response Generation , 2017, AAAI.

[20]  Lu Wang,et al.  Joint Modeling of Content and Discourse Relations in Dialogues , 2017, ACL.

[21]  Bowen Wu,et al.  Ranking Responses Oriented to Conversational Relevance in Chat-bots , 2016, COLING.

[22]  Zhoujun Li,et al.  Sequential Match Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots , 2016, ArXiv.

[23]  Harry Shum,et al.  From Eliza to XiaoIce: challenges and opportunities with social chatbots , 2018, Frontiers of Information Technology & Electronic Engineering.

[24]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[25]  Jianfeng Gao,et al.  A Persona-Based Neural Conversation Model , 2016, ACL.

[26]  Zhen Xu,et al.  Incorporating loose-structured knowledge into conversation modeling via recall-gate LSTM , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[27]  Zhoujun Li,et al.  Neural Response Generation with Dynamic Vocabularies , 2017, AAAI.

[28]  Zhoujun Li,et al.  Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots , 2018, ACL.

[29]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[30]  Hang Li,et al.  An Information Retrieval Approach to Short Text Conversation , 2014, ArXiv.

[31]  Wei-Ying Ma,et al.  Topic Augmented Neural Response Generation with a Joint Attention Mechanism , 2016, ArXiv.