Enhancing Dialogue Generation with Conversational Concept Flows

Human conversations contain natural and reasonable topic shifts, reflected as the concept flows across utterances.Previous researches prove that explicitly modeling concept flows with a large commonsense knowledge graph effectively improves response quality.However, we argue that there exists a gap between the knowledge graph and the conversation.The knowledge graph has limited commonsense knowledge and ignores the characteristics of natural conversations.Thus, many concepts and relations in conversations are not included.To bridge this gap, we propose to enhance dialogue generation with conversational concept flows.Specifically, we extract abundant concepts and relations from natural conversations and build a new conversation-aware knowledge graph.In addition, we design a novel relation-aware graph encoder to capture the concept flows guided by the knowledge graph.Experimental results on the large-scale Reddit conversation dataset indicate that our method performs better than strong baselines, andfurther analysis verifies the effectiveness of each component.All our code and data will be publicly available after acceptance.

[1]  Mao Yan Chen,et al.  EmpHi: Generating Empathetic Responses with Human-like Intents , 2022, NAACL.

[2]  Yanran Li,et al.  MISC: A Mixed Strategy-Aware Model integrating COMET for Emotional Support Conversation , 2022, ACL.

[3]  Renelito Delos Santos,et al.  LaMDA: Language Models for Dialog Applications , 2022, ArXiv.

[4]  Sachindra Joshi,et al.  MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents , 2021, EMNLP.

[5]  Minlie Huang,et al.  CEM: Commonsense-aware Empathetic Response Generation , 2021, AAAI.

[6]  Qi Zhang,et al.  Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems , 2021, EMNLP.

[7]  Shuzi Niu,et al.  Position Enhanced Mention Graph Attention Network for Dialogue Relation Extraction , 2021, SIGIR.

[8]  E. Chng,et al.  GDPNet: Refining Latent Multi-View Graph for Relation Extraction , 2020, AAAI.

[9]  J. Weston,et al.  Recipes for Safety in Open-domain Chatbots , 2020, ArXiv.

[10]  Yejin Choi,et al.  COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs , 2020, AAAI.

[11]  Zheng-Yu Niu,et al.  Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation , 2020, ACL.

[12]  Mary Williamson,et al.  Recipes for Building an Open-Domain Chatbot , 2020, EACL.

[13]  Claire Cardie,et al.  Dialogue-Based Relation Extraction , 2020, ACL.

[14]  Quoc V. Le,et al.  Towards a Human-like Open-Domain Chatbot , 2020, ArXiv.

[15]  Jeremy Blackburn,et al.  The Pushshift Reddit Dataset , 2020, ICWSM.

[16]  Zhiyuan Liu,et al.  Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs , 2019, ACL.

[17]  Jianfeng Gao,et al.  DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation , 2019, ACL.

[18]  Eric P. Xing,et al.  Target-Guided Open-Domain Conversation , 2019, ACL.

[19]  Jianfeng Gao,et al.  Challenges in Building Intelligent Open-domain Dialog Systems , 2019, ACM Trans. Inf. Syst..

[20]  Sungjin Lee,et al.  Jointly Optimizing Diversity and Relevance in Neural Response Generation , 2019, NAACL.

[21]  Y-Lan Boureau,et al.  Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset , 2018, ACL.

[22]  J. Weston,et al.  Wizard of Wikipedia: Knowledge-Powered Conversational agents , 2018, ICLR.

[23]  Zhe Gan,et al.  Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization , 2018, NeurIPS.

[24]  Ruslan Salakhutdinov,et al.  Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text , 2018, EMNLP.

[25]  Xiaoyan Zhu,et al.  Commonsense Knowledge Aware Conversation Generation with Graph Attention , 2018, IJCAI.

[26]  Fumin Shen,et al.  Chat More: Deepening and Widening the Chatting Topic via A Deep Model , 2018, SIGIR.

[27]  Mari Ostendorf,et al.  Sounding Board: A User-Centric and Content-Driven Social Chatbot , 2018, NAACL.

[28]  Jason Weston,et al.  Personalizing Dialogue Agents: I have a dog, do you have pets too? , 2018, ACL.

[29]  Yu Zhang,et al.  Flexible End-to-End Dialogue System for Knowledge Grounded Conversation , 2017, ArXiv.

[30]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[31]  Minlie Huang,et al.  Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory , 2017, AAAI.

[32]  Ming-Wei Chang,et al.  A Knowledge-Grounded Neural Conversation Model , 2017, AAAI.

[33]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[34]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[35]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[36]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[37]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[38]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[39]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[40]  Colin Cherry,et al.  A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU , 2014, WMT@ACL.

[41]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[42]  Alon Lavie,et al.  METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments , 2007, WMT@ACL.

[43]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[44]  Hermann Ney,et al.  A Systematic Comparison of Various Statistical Alignment Models , 2003, CL.

[45]  George R. Doddington,et al.  Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics , 2002 .

[46]  Dilek Z. Hakkani-Tür,et al.  Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation , 2021, ArXiv.

[47]  Xiaoyan Zhu,et al.  EARL: Informative Knowledge-Grounded Conversation Generation with Entity-Agnostic Representation Learning , 2021, EMNLP.

[48]  Yejin Choi,et al.  ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning , 2019, AAAI.

[49]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[50]  Jianfeng Gao,et al.  End-to-End Conversation Modeling : Moving beyond Chitchat DSTC 7 Task 2 Description ( v 1 . 0 ) , 2018 .