Difficulty-Controllable Multi-hop Question Generation from Knowledge Graphs

Knowledge graphs have become ubiquitous data sources and their utility has been amplified by the research on ability to answer carefully crafted questions over knowledge graphs. We investigate the problem of question generation (QG) over knowledge graphs wherein, the level of difficulty of the question can be controlled. We present an end-to-end neural network-based method for automatic generation of complex multi-hop questions over knowledge graphs. Taking a subgraph and an answer as input, our transformer-based model generates a natural language question. Our model incorporates difficulty estimation based on named entity popularity, and makes use of this estimation to generate difficulty-controllable questions. We evaluate our model on two recent multi-hop QA datasets. Our evaluation shows that our model is able to generate high-quality, fluent and relevant questions. We have released our curated QG dataset and code at https://github.com/liyuanfang/mhqg.

[1]  Enrico Motta,et al.  Evaluating question answering over linked data , 2013, J. Web Semant..

[2]  Zhiyuan Liu,et al.  OpenKE: An Open Toolkit for Knowledge Embedding , 2018, EMNLP.

[3]  Xuanjing Huang,et al.  A Question Type Driven Framework to Diversify Visual Question Generation , 2018, IJCAI.

[4]  Mitesh M. Khapra,et al.  Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph , 2018, AAAI.

[5]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[6]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[7]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[8]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[9]  Diego Marcheggiani,et al.  Deep Graph Convolutional Encoders for Structured Data to Text Generation , 2018, INLG.

[10]  Vanessa López,et al.  Core techniques of question answering systems over knowledge bases: a survey , 2017, Knowledge and Information Systems.

[11]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[12]  Ming-Wei Chang,et al.  The Value of Semantic Parse Labeling for Knowledge Base Question Answering , 2016, ACL.

[13]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[14]  Ganesh Ramakrishnan,et al.  Automating Reading Comprehension by Generating Question and Answer Pairs , 2018, PAKDD.

[15]  Navdeep Jaitly,et al.  Towards Better Decoding and Language Model Integration in Sequence to Sequence Models , 2016, INTERSPEECH.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Mitesh M. Khapra,et al.  Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model , 2017, EACL.

[18]  Mohamed Yahya,et al.  Generating Quiz Questions from Knowledge Graphs , 2015, WWW.

[19]  Shashi Narayan,et al.  Creating Training Corpora for NLG Micro-Planners , 2017, ACL.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Jonathan Berant,et al.  The Web as a Knowledge-Base for Answering Complex Questions , 2018, NAACL.

[22]  Xiaoyan Zhu,et al.  An Interpretable Reasoning Network for Multi-Relation Question Answering , 2018, COLING.

[23]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[24]  Yoshua Bengio,et al.  Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus , 2016, ACL.

[25]  Michael Günther,et al.  Introducing Wikidata to the Linked Data Web , 2014, SEMWEB.

[26]  Bolei Zhou,et al.  Visual Question Generation as Dual Task of Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Christophe Gravier,et al.  Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types , 2018, NAACL.

[29]  Wei Wang,et al.  GTR-LSTM: A Triple Encoder for Sentence Generation from RDF Data , 2018, ACL.

[30]  Xinya Du,et al.  Learning to Ask: Neural Question Generation for Reading Comprehension , 2017, ACL.

[31]  Christophe Gravier,et al.  Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples , 2017, J. Web Semant..

[32]  Jens Lehmann,et al.  LC-QuAD: A Corpus for Complex Question Answering over Knowledge Graphs , 2017, SEMWEB.

[33]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[34]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[35]  Yue Zhang,et al.  Leveraging Context Information for Natural Question Generation , 2018, NAACL.