A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems with Incomplete User Profiles

There is increasing interest in developing personalized Task-oriented Dialogue Systems (TDSs). Previous work on personalized TDSs often assumes that complete user profiles are available for most or even all users. This is unrealistic because In this paper, we study personalized TDSs without assuming that user profiles are complete. We propose a Cooperative Memory Network (CoMemNN) that has a novel mechanism to gradually enrich user profiles as dialogues progress and to simultaneously improve response selection based on the enriched profiles. Cooperative Memory Network (CoMemNN) consists of two core modules: User Profile Enrichment (UPE) and Dialogue Response Selection (DRS). The former enriches incomplete user profiles by utilizing collaborative information from neighbor users as well as current dialogues. The latter uses the enriched profiles to update the current user query so as to encode more useful information, based on which a personalized response to a user request is selected. We conduct extensive experiments on the personalized bAbI dialogue benchmark datasets. We find that CoMemNN is able to enrich user profiles effectively, which results in an improvement of 3.06% in terms of response selection accuracy compared to state-of-the-art methods. We also test the robustness of CoMemNN against incompleteness of user profiles by randomly discarding attribute values from user profiles. Even when discarding 50% of the attribute values, CoMemNN is able to match the performance of the best performing baseline without discarding user profiles, showing the robustness of CoMemNN.

[1]  Maarten de Rijke,et al.  Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response Generation , 2020, ECAI.

[2]  Jianfeng Gao,et al.  Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models , 2017, IJCNLP.

[3]  Haoyu Song,et al.  Exploiting Persona Information for Diverse Generation of Conversational Responses , 2019, IJCAI.

[4]  Yoav Goldberg,et al.  Controlling Linguistic Style Aspects in Neural Language Generation , 2017, ArXiv.

[5]  Richard Socher,et al.  Sketch-Fill-A-R: A Persona-Grounded Chit-Chat Generation Framework , 2019, NLP4CONVAI.

[6]  Ting Liu,et al.  Neural personalized response generation as domain adaptation , 2017, World Wide Web.

[7]  Xiang Zhang,et al.  Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems , 2015, ICLR.

[8]  Thomas Wolf,et al.  TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents , 2019, ArXiv.

[9]  Julien Perez,et al.  Gated End-to-End Memory Networks , 2016, EACL.

[10]  Jesse Hoey,et al.  ALOHA: Artificial Learning of Human Attributes for Dialogue Agents , 2020, AAAI.

[11]  Jiliang Tang,et al.  A Survey on Dialogue Systems: Recent Advances and New Frontiers , 2017, SKDD.

[12]  Haitao Zheng,et al.  Grayscale Data Construction and Multi-Level Ranking Objective for Dialogue Response Selection , 2020, ArXiv.

[13]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[14]  Jiahuan Pei,et al.  A Modular Task-oriented Dialogue System Using a Neural Mixture-of-Experts , 2019, ArXiv.

[15]  Pascale Fung,et al.  Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems , 2018, ACL.

[16]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[17]  Gerhard Weikum,et al.  Listening between the Lines: Learning Personal Attributes from Conversations , 2019, WWW.

[18]  Xu Sun,et al.  Learning Personalized End-to-End Goal-Oriented Dialog , 2018, AAAI.

[19]  Jason Weston,et al.  Learning End-to-End Goal-Oriented Dialog , 2016, ICLR.

[20]  Boi Faltings,et al.  Personalization in Goal-Oriented Dialog , 2017, ArXiv.

[21]  Qian Liu,et al.  You Impress Me: Dialogue Generation via Mutual Persona Perception , 2020, ACL.

[22]  Jianfeng Gao,et al.  A Persona-Based Neural Conversation Model , 2016, ACL.

[23]  Min Yang,et al.  Personalized Response Generation via Domain adaptation , 2017, SIGIR.

[24]  Gökhan Tür,et al.  Personal knowledge graph population from user utterances in conversational understanding , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[25]  David Vandyke,et al.  A Network-based End-to-End Trainable Task-oriented Dialogue System , 2016, EACL.

[26]  Yunming Ye,et al.  A memory network based end-to-end personalized task-oriented dialogue generation , 2020, Knowl. Based Syst..

[27]  Haitao Zheng,et al.  The World Is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection , 2020, EMNLP.

[28]  David Konopnicki,et al.  Neural Response Generation for Customer Service based on Personality Traits , 2017, INLG.

[29]  M. de Rijke,et al.  SEntNet: Source-aware Recurrent Entity Network for Dialogue Response Selection , 2019, ArXiv.

[30]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Anna Tigunova.  Extracting Personal Information from Conversations , 2020, WWW.

[33]  Pascale Fung,et al.  End-to-End Dynamic Query Memory Network for Entity-Value Independent Task-Oriented Dialog , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Yu Zhang,et al.  Personalizing a Dialogue System With Transfer Reinforcement Learning , 2016, AAAI.

[35]  Haoyu Song,et al.  Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation , 2020, ACL.

[36]  Xiaoyan Zhu,et al.  Assigning Personality/Profile to a Chatting Machine for Coherent Conversation Generation , 2018, IJCAI.

[37]  Harry Shum,et al.  The Design and Implementation of XiaoIce, an Empathetic Social Chatbot , 2018, CL.

[38]  Geoffrey Zweig,et al.  Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning , 2017, ACL.

[39]  Christopher D. Manning,et al.  Key-Value Retrieval Networks for Task-Oriented Dialogue , 2017, SIGDIAL Conference.

[40]  Matthew Henderson,et al.  Training Neural Response Selection for Task-Oriented Dialogue Systems , 2019, ACL.

[41]  Pascale Fung,et al.  Personalizing Dialogue Agents via Meta-Learning , 2019, ACL.

[42]  Antoine Bordes,et al.  Training Millions of Personalized Dialogue Agents , 2018, EMNLP.

[43]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[44]  Minlie Huang,et al.  A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data , 2019, AAAI.

[45]  Song Liu,et al.  Personalized Dialogue Generation with Diversified Traits , 2019, ArXiv.

[46]  Lei Li,et al.  Automatic Generation of Personalized Comment Based on User Profile , 2019, ACL.

[47]  Peng Xu,et al.  Getting To Know You: User Attribute Extraction from Dialogues , 2019, LREC.

[48]  Jason Weston,et al.  Personalizing Dialogue Agents: I have a dog, do you have pets too? , 2018, ACL.

[49]  Yichao Lu,et al.  Goal-Oriented End-to-End Conversational Models with Profile Features in a Real-World Setting , 2019, NAACL.

[50]  Jatin Ganhotra,et al.  Learning End-to-End Goal-Oriented Dialog with Multiple Answers , 2018, EMNLP.

[51]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[52]  Piji Li,et al.  A Neural Topical Expansion Framework for Unstructured Persona-oriented Dialogue Generation , 2020, ECAI.