WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference

Developing methods of automated inference that are able to provide users with compelling human-readable justifications for why the answer to a question is correct is critical for domains such as science and medicine, where user trust and detecting costly errors are limiting factors to adoption. One of the central barriers to training question answering models on explainable inference tasks is the lack of gold explanations to serve as training data. In this paper we present a corpus of explanations for standardized science exams, a recent challenge task for question answering. We manually construct a corpus of detailed explanations for nearly all publicly available standardized elementary science question (approximately 1,680 3rd through 5th grade questions) and represent these as "explanation graphs" -- sets of lexically overlapping sentences that describe how to arrive at the correct answer to a question through a combination of domain and world knowledge. We also provide an explanation-centered tablestore, a collection of semi-structured tables that contain the knowledge to construct these elementary science explanations. Together, these two knowledge resources map out a substantial portion of the knowledge required for answering and explaining elementary science exams, and provide both structured and free-text training data for the explainable inference task.

[1]  Bethany Rittle-Johnson,et al.  Eliciting explanations: Constraints on when self-explanation aids learning , 2017, Psychonomic bulletin & review.

[2]  Percy Liang,et al.  Compositional Semantic Parsing on Semi-Structured Tables , 2015, ACL.

[3]  Peter Clark,et al.  A study of the knowledge base requirements for passing an elementary science test , 2013, AKBC '13.

[4]  Oren Etzioni,et al.  Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions , 2016, AAAI.

[5]  Ming Zhou,et al.  Answering Questions with Complex Semantic Constraints on Open Knowledge Bases , 2015, CIKM.

[6]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[7]  Peter A. Jansen.  A Study of Automatically Acquiring Explanatory Inference Patterns from Corpora of Explanations: Lessons from Elementary Science Exams , 2017, AKBC@NIPS.

[8]  Oren Etzioni,et al.  IKE - An Interactive Tool for Knowledge Extraction , 2016, AKBC@NAACL-HLT.

[9]  Peter Clark,et al.  Answering Complex Questions Using Open Information Extraction , 2017, ACL.

[10]  Hao Ma,et al.  Table Cell Search for Question Answering , 2016, WWW.

[11]  Rod D. Roscoe,et al.  Understanding Tutor Learning: Knowledge-Building and Knowledge-Telling in Peer Tutors’ Explanations and Questions , 2007 .

[12]  Oren Etzioni,et al.  Question Answering via Integer Programming over Semi-Structured Knowledge , 2016, IJCAI.

[13]  Eduard H. Hovy,et al.  Tables as Semi-structured Knowledge for Question Answering , 2016, ACL.

[14]  Oren Etzioni,et al.  Moving beyond the Turing Test with the Allen AI Science Challenge , 2016, Commun. ACM.

[15]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[16]  Cristine H. Legare,et al.  The Contributions of Explanation and Exploration to Children's Scientific Reasoning , 2014 .

[17]  Oren Etzioni,et al.  Exploring Markov Logic Networks for Question Answering , 2015, EMNLP.

[18]  Oren Etzioni,et al.  Open Information Extraction: The Second Generation , 2011, IJCAI.

[19]  Mihai Surdeanu,et al.  Tell Me Why: Using Question Answering as Distant Supervision for Answer Justification , 2017, CoNLL.

[20]  Peter Clark.  Elementary School Science and Math Tests as a Driver for AI: Take the Aristo Challenge! , 2015, AAAI.

[21]  Mihai Surdeanu,et al.  Higher-order Lexical Semantic Models for Non-factoid Answer Reranking , 2015, TACL.

[22]  Peter Jansen,et al.  What’s in an Explanation? Characterizing Knowledge and Inference Requirements for Elementary Science Exams , 2016, COLING.

[23]  Peter Jansen,et al.  Framing QA as Building and Ranking Intersentence Answer Justifications , 2017, CL.