Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling

In this paper, we propose an unsupervised query enhanced approach for knowledge-intensive conversations, namely QKConv. There are three modules in QKConv: a query generator, an off-the-shelf knowledge selector, and a response generator. QKConv is optimized through joint training, which produces the response by exploring multiple candidate queries and leveraging corresponding selected knowledge. The joint training solely relies on the dialogue context and target response, getting exempt from extra query annotations or knowledge provenances. To evaluate the effectiveness of the proposed QKConv, we conduct experiments on three representative knowledge-intensive conversation datasets: conversational question-answering, task-oriented dialogue, and knowledge-grounded conversation. Experimental results reveal that QKConv performs better than all unsupervised methods across three datasets and achieves competitive performance compared to supervised methods.

[1]  S. Malmasi,et al.  Reinforced Question Rewriting for Conversational Question Answering , 2022, EMNLP.

[2]  Fan Wang,et al.  Q-TOD: A Query-driven Task-oriented Dialogue System , 2022, EMNLP.

[3]  Md. Faisal Mahbub Chowdhury,et al.  Re2G: Retrieve, Rerank, Generate , 2022, NAACL.

[4]  Ryan J. Lowe,et al.  Training language models to follow instructions with human feedback , 2022, NeurIPS.

[5]  Sungdong Kim,et al.  Saving Dense Retriever from Shortcut Dependency in Conversational Search , 2022, EMNLP.

[6]  Luísa Coheur,et al.  Question rewriting? Assessing its importance for conversational question answering , 2022, ECIR.

[7]  Dragomir R. Radev,et al.  UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models , 2022, EMNLP.

[8]  Gaurav Singh Tomar,et al.  CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning , 2021, EMNLP.

[9]  Christopher Potts,et al.  Hindsight: Posterior-guided training of retrievers for improved open-ended generation , 2021, ICLR.

[10]  Wayne Xin Zhao,et al.  RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking , 2021, EMNLP.

[11]  Jason Weston,et al.  Internet-Augmented Dialogue Generation , 2021, ACL.

[12]  Jason Weston,et al.  Retrieval Augmentation Reduces Hallucination in Conversation , 2021, EMNLP.

[13]  Hermann Ney,et al.  Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog , 2021, ArXiv.

[14]  Can Xu,et al.  Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues? , 2020, ArXiv.

[15]  Zhucheng Tu,et al.  Open-Domain Question Answering Goes Conversational via Question Rewriting , 2020, NAACL.

[16]  Weinan Zhang,et al.  A Compare Aggregate Transformer for Understanding Document-grounded Dialogue , 2020, FINDINGS.

[17]  Yelong Shen,et al.  Generation-Augmented Retrieval for Open-Domain Question Answering , 2020, ACL.

[18]  Nicola De Cao,et al.  KILT: a Benchmark for Knowledge Intensive Language Tasks , 2020, NAACL.

[19]  Ming-Wei Chang,et al.  Retrieval Augmented Language Model Pre-Training , 2020, ICML.

[20]  Paul N. Bennett,et al.  Few-Shot Generative Conversational Query Rewriting , 2020, SIGIR.

[21]  Fabio Petroni,et al.  Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , 2020, NeurIPS.

[22]  S. Longpre,et al.  Question Rewriting for Conversational Question Answering , 2020, WSDM.

[23]  Gunhee Kim,et al.  Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue , 2020, ICLR.

[24]  Gary Marcus,et al.  The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence , 2020, ArXiv.

[25]  Omer Levy,et al.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension , 2019, ACL.

[26]  Colin Raffel,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[27]  Sebastian Riedel,et al.  Language Models as Knowledge Bases? , 2019, EMNLP.

[28]  Rongzhong Lian,et al.  Learning to Select Knowledge for Response Generation in Dialog Systems , 2019, IJCAI.

[29]  J. Shane Culpepper,et al.  Joint Optimization of Cascade Ranking Models , 2019, WSDM.

[30]  J. Weston,et al.  Wizard of Wikipedia: Knowledge-Powered Conversational agents , 2018, ICLR.

[31]  Mitesh M. Khapra,et al.  Towards Exploiting Background Knowledge for Building Conversation Systems , 2018, EMNLP.

[32]  Alan W. Black,et al.  A Dataset for Document Grounded Conversations , 2018, EMNLP.

[33]  Christopher D. Manning,et al.  Key-Value Retrieval Networks for Task-Oriented Dialogue , 2017, SIGDIAL Conference.

[34]  David Vandyke,et al.  A Network-based End-to-End Trainable Task-oriented Dialogue System , 2016, EACL.

[35]  Hua Wu,et al.  PLATO-KAG: Unsupervised Knowledge-Grounded Conversation via Joint Modeling , 2021, NLP4CONVAI.

[36]  Walter Daelemans,et al.  BART for Knowledge Grounded Conversations , 2020, Converse@KDD.