Machine Comprehension with Syntax, Frames, and Semantics

We demonstrate significant improvement on the MCTest question answering task (Richardson et al., 2013) by augmenting baseline features with features based on syntax, frame semantics, coreference, and word embeddings, and combining them in a max-margin learning framework. We achieve the best results we are aware of on this dataset, outperforming concurrentlypublished results. These results demonstrate a significant performance gradient for the use of linguistic structure in machine comprehension.

[1]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[2]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[3]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[4]  Lynette Hirschman,et al.  Deep Read: A Reading Comprehension System , 1999, ACL.

[5]  Mats Rooth,et al.  Looking Under the Hood : Tools for Diagnosing your Question Answering Engine , 2001, ACL 2001.

[6]  Noah A. Smith,et al.  What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA , 2007, EMNLP.

[7]  Roberto Basili,et al.  Exploiting Syntactic and Shallow Semantic Kernels for Question Answer Classification , 2007, ACL.

[8]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[9]  Noah A. Smith,et al.  Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions , 2010, NAACL.

[10]  Jennifer Chu-Carroll,et al.  Building Watson: An Overview of the DeepQA Project , 2010, AI Mag..

[11]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[12]  Noah A. Smith,et al.  Probabilistic Frame-Semantic Parsing , 2010, NAACL.

[13]  Eduard H. Hovy,et al.  Overview of QA4MRE at CLEF 2011: Question Answering for Machine Reading Evaluation , 2011, CLEF.

[14]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[15]  Noah A. Smith,et al.  Automatic factual question generation from text , 2011 .

[16]  Somnath Banerjee,et al.  Question Answering System for QA4MRE@CLEF 2012 , 2012, CLEF.

[17]  Peter Clark,et al.  An Entailment-Based Approach to the QA4MRE Challenge , 2012, CLEF.

[18]  Roser Morante,et al.  QA4MRE 2011-2013: Overview of Question Answering for Machine Reading Evaluation , 2013, CLEF.

[19]  Matthew Richardson,et al.  MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text , 2013, EMNLP.

[20]  Benno Stein,et al.  Information Access Evaluation. Multilinguality, Multimodality, and Visualization , 2013, Lecture Notes in Computer Science.

[21]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[22]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[23]  Alexander I. Rudnicky,et al.  Unsupervised induction and filling of semantic slots for spoken dialogue systems using frame-semantic parsing , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[24]  Heeyoung Lee,et al.  Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules , 2013, CL.

[25]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[26]  Noah A. Smith,et al.  Frame-Semantic Parsing , 2014, CL.

[27]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[28]  Xuchen Yao,et al.  Freebase QA: Information Extraction or Semantic Parsing? , 2014, ACL 2014.

[29]  Kevin Gimpel,et al.  Tailoring Continuous Word Representations for Dependency Parsing , 2014, ACL.

[30]  Regina Barzilay,et al.  Machine Comprehension with Discourse Relations , 2015, ACL.

[31]  Jason Weston,et al.  Weakly Supervised Memory Networks , 2015, ArXiv.

[32]  Eric P. Xing,et al.  Learning Answer-Entailing Structures for Machine Comprehension , 2015, ACL.

[33]  Jason Weston,et al.  Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks , 2015, ICLR.