Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion

Fact-centric information needs are rarely one-shot; users typically ask follow-up questions to explore a topic. In such a conversational setting, the user's inputs are often incomplete, with entities or predicates left out, and ungrammatical phrases. This poses a huge challenge to question answering (QA) systems that typically rely on cues in full-fledged interrogative sentences. As a solution, we develop CONVEX, an unsupervised method that can answer incomplete questions over a knowledge graph (KG) by maintaining conversation context using entities and predicates seen so far and automatically inferring missing or ambiguous pieces for follow-up questions. The core of our method is a graph exploration algorithm that judiciously expands a frontier to find candidate answers for the current question. To evaluate CONVEX, we release ConvQuestions, a crowdsourced benchmark with 11,200 distinct conversations from five different domains. We show that CONVEX: (i) adds conversational support to any stand-alone QA system, and (ii) outperforms state-of-the-art baselines and question completion strategies.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Gerhard Weikum,et al.  QUINT: Interpretable Question Answering over Knowledge Bases , 2017, EMNLP.

[3]  Percy Liang,et al.  Compositional Semantic Parsing on Semi-Structured Tables , 2015, ACL.

[4]  Ming-Wei Chang,et al.  Search-based Neural Structured Learning for Sequential Question Answering , 2017, ACL.

[5]  Jens Lehmann,et al.  Template-based question answering over RDF data , 2012, WWW.

[6]  Gerhard Weikum,et al.  ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters , 2018, NAACL.

[7]  Guillaume Bouchard,et al.  Interpretation of Natural Language Rules in Conversational Machine Reading , 2018, EMNLP.

[8]  Mirella Lapata,et al.  Learning to Paraphrase for Question Answering , 2017, EMNLP.

[9]  Ahmed Elgohary,et al.  A dataset and baselines for sequential open-domain question answering , 2018, EMNLP.

[10]  Gerhard Weikum,et al.  Never-Ending Learning for Open-Domain Question Answering over Knowledge Bases , 2018, WWW.

[11]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[12]  Vineet Kumar,et al.  Incomplete Follow-up Question Resolution using Retrieval based Sequence to Sequence Learning , 2017, SIGIR.

[13]  Ganesh Ramakrishnan,et al.  Neural architecture for question answering using a knowledge graph and web corpus , 2017, Information Retrieval Journal.

[14]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[15]  Kenny Q. Zhu,et al.  Knowledge Base Question Answering via Encoding of Complex Query Graphs , 2018, EMNLP.

[16]  Jens Lehmann,et al.  Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level , 2017, WWW.

[17]  Moni Naor,et al.  Optimal aggregation algorithms for middleware , 2001, PODS.

[18]  Hannah Bast,et al.  More Accurate Question Answering on Freebase , 2015, CIKM.

[19]  Ming Zhou,et al.  Dialog-to-Action: Conversational Question Answering Over a Large-Scale Knowledge Base , 2018, NeurIPS.

[20]  Thomas Pellissier Tanon,et al.  Demoing Platypus - A Multilingual Question Answering Platform for Wikidata , 2018, ESWC.

[21]  Jonathan Berant,et al.  The Web as a Knowledge-Base for Answering Complex Questions , 2018, NAACL.

[22]  Asim Kadav,et al.  A Context-aware Attention Network for Interactive Question Answering , 2016, KDD.

[23]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[24]  Gerhard Weikum,et al.  Robust question answering over the web of linked data , 2013, CIKM.

[25]  Paolo Ferragina,et al.  TAGME: on-the-fly annotation of short text fragments (by wikipedia entities) , 2010, CIKM.

[26]  Jingyuan Zhang,et al.  Knowledge Graph Embedding Based Question Answering , 2019, WSDM.

[27]  Ruslan Salakhutdinov,et al.  Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text , 2018, EMNLP.

[28]  Mitesh M. Khapra,et al.  Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph , 2018, AAAI.

[29]  Jitendra Ajmera,et al.  A statistical approach for Non-Sentential Utterance Resolution for Interactive QA System , 2015, SIGDIAL Conference.

[30]  Gerhard Weikum,et al.  Answering Complex Questions by Joining Multi-Document Evidence with Quasi Knowledge Graphs , 2019, SIGIR.

[31]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[32]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[33]  Qifa Ke,et al.  Conversational Query Understanding Using Sequence to Sequence Modeling , 2018, WWW.

[34]  Peter Clark,et al.  Learning Knowledge Graphs for Question Answering through Conversational Dialog , 2015, NAACL.

[35]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[36]  Pierre Maret,et al.  QAnswer: A Question Answering prototype bridging the gap between a considerable part of the LOD cloud and end-users , 2019, WWW.

[37]  Gerhard Weikum,et al.  Robust Disambiguation of Named Entities in Text , 2011, EMNLP.

[38]  Kevin Gimpel,et al.  Towards Universal Paraphrastic Sentence Embeddings , 2015, ICLR.