Encoding Explanatory Knowledge for Zero-shot Science Question Answering

This paper describes N-XKT (Neural encoding based on eXplanatory Knowledge Transfer), a novel method for the automatic transfer of explanatory knowledge through neural encoding mechanisms. We demonstrate that N-XKT is able to improve accuracy and generalization on science Question Answering (QA). Specifically, by leveraging facts from background explanatory knowledge corpora, the N-XKT model shows a clear improvement on zero-shot QA. Furthermore, we show that N-XKT can be fine-tuned on a target QA dataset, enabling faster convergence and more accurate results. A systematic analysis is conducted to quantitatively analyze the performance of the N-XKT model and the impact of different categories of knowledge on the zero-shot generalization task.

[1]  Marco Valentino,et al.  Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks , 2019, TextGraphs@EMNLP.

[2]  Marco Valentino,et al.  A Survey on Explainability in Machine Reading Comprehension , 2020, ArXiv.

[3]  Gabriel Stanovsky,et al.  DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs , 2019, NAACL.

[4]  Wei Zhao,et al.  Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension , 2018, SemEval@NAACL-HLT.

[5]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[6]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[7]  Yoshua Bengio,et al.  HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , 2018, EMNLP.

[8]  Souvik Kundu,et al.  Exploiting Explicit Paths for Multi-hop Reading Comprehension , 2019, ACL.

[9]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[10]  Ming Zhou,et al.  Improving Question Answering by Commonsense-Based Pre-Training , 2018, NLPCC.

[11]  Oren Etzioni,et al.  Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , 2018, ArXiv.

[12]  Dmitry Ustalov,et al.  TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration , 2019, EMNLP.

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  Chang Zhou,et al.  Cognitive Graph for Multi-Hop Reading Comprehension at Scale , 2019, ACL.

[15]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[16]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[17]  Richard Socher,et al.  Explain Yourself! Leveraging Language Models for Commonsense Reasoning , 2019, ACL.

[18]  Kartik Talamadupula,et al.  Answering Science Exam Questions Using Query Reformulation with Background Knowledge , 2019, AKBC.

[19]  Oren Etzioni,et al.  From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project , 2020, AI Mag..

[20]  Peter Jansen,et al.  What’s in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams , 2016, COLING.

[21]  Eric P. Xing,et al.  Science Question Answering using Instructional Materials , 2016, ACL.

[22]  Chitta Baral,et al.  Self-Supervised Knowledge Triplet Learning for Zero-shot Question Answering , 2020, EMNLP.

[23]  Yang Li,et al.  Answering Elementary Science Questions by Constructing Coherent Scenes using Background Knowledge , 2015, EMNLP.

[24]  Weizhu Chen,et al.  Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering , 2018, NAACL.

[25]  Lei Li,et al.  Dynamically Fused Graph Network for Multi-hop Reasoning , 2019, ACL.

[26]  Guokun Lai,et al.  RACE: Large-scale ReAding Comprehension Dataset From Examinations , 2017, EMNLP.

[27]  Peter A. Jansen,et al.  WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference , 2020, LREC.

[28]  Simon Ostermann,et al.  MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge , 2018, LREC.

[29]  Richard Socher,et al.  Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering , 2019, ICLR.

[30]  Peter Clark,et al.  Answering Complex Questions Using Open Information Extraction , 2017, ACL.

[31]  Dan Roth,et al.  Question Answering as Global Reasoning Over Semantic Abstractions , 2018, AAAI.

[32]  Peter Clark,et al.  SciTaiL: A Textual Entailment Dataset from Science Question Answering , 2018, AAAI.

[33]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[34]  Chitta Baral,et al.  Careful Selection of Knowledge to Solve Open Book Question Answering , 2019, ACL.

[35]  Marco Valentino,et al.  Unification-based Reconstruction of Multi-hop Explanations for Science Questions , 2021, EACL.

[36]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[37]  Peter Clark,et al.  Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering , 2018, EMNLP.

[38]  Claire Cardie,et al.  Improving Machine Reading Comprehension with General Reading Strategies , 2018, NAACL.

[39]  Tushar Khot,et al.  QASC: A Dataset for Question Answering via Sentence Composition , 2020, AAAI.

[40]  Le Song,et al.  KG^2: Learning to Reason Science Exam Questions with Contextual Knowledge Graph Embeddings , 2018, ArXiv.

[41]  Mihai Surdeanu,et al.  Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering , 2018, SIGIR.

[42]  Traian Rebedea,et al.  Improving Retrieval-Based Question Answering with Deep Inference Models , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[43]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[44]  Mihai Surdeanu,et al.  Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering , 2019, EMNLP.

[45]  Iz Beltagy,et al.  SciBERT: A Pretrained Language Model for Scientific Text , 2019, EMNLP.

[46]  Clayton T. Morrison,et al.  WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference , 2018, LREC.