Simple Question Answering by Attentive Convolutional Neural Network

This work focuses on answering single-relation factoid questions over Freebase. Each question can acquire the answer from a single fact of form (subject, predicate, object) in Freebase. This task, simple question answering (SimpleQA), can be addressed via a two-step pipeline: entity linking and fact selection. In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN). This work makes two main contributions. (i) A simple and effective entity linker over Freebase is proposed. Our entity linker outperforms the state-of-the-art entity linker over SimpleQA task. (ii) A novel attentive maxpooling is stacked over word-CNN, so that the predicate representation can be matched with the predicate-focused question representation more effectively. Experiments show that our system sets new state-of-the-art in this task.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[4]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[5]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[6]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[7]  Oren Etzioni,et al.  Identifying Relations for Open Information Extraction , 2011, EMNLP.

[8]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[9]  Oren Etzioni,et al.  Paraphrase-Driven Learning for Open Question Answering , 2013, ACL.

[10]  Gerhard Weikum,et al.  Robust question answering over the web of linked data , 2013, CIKM.

[11]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[12]  Xuchen Yao,et al.  Information Extraction over Structured Data: Question Answering with Freebase , 2014, ACL.

[13]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[14]  Lei Yu,et al.  Deep Learning for Answer Sentence Selection , 2014, ArXiv.

[15]  Hinrich Schütze,et al.  FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging , 2014, TACL.

[16]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[17]  Xuchen Yao,et al.  Freebase QA: Information Extraction or Semantic Parsing? , 2014, ACL 2014.

[18]  Oren Etzioni,et al.  Open question answering over curated and extracted knowledge bases , 2014, KDD.

[19]  Ming Zhou,et al.  Question Answering over Freebase with Multi-Column Convolutional Neural Networks , 2015, ACL.

[20]  Ming-Wei Chang,et al.  Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base , 2015, ACL.

[21]  Hannah Bast,et al.  More Accurate Question Answering on Freebase , 2015, CIKM.

[22]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[23]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[24]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[25]  Pengtao Xie,et al.  On the Generalization Error Bounds of Neural Networks under Diversity-Inducing Mutual Angular Regularization , 2015, ArXiv.

[26]  Mohamed Yahya,et al.  Generating Quiz Questions from Knowledge Graphs , 2015, WWW.

[27]  Xuchen Yao,et al.  Lean Question Answering over Freebase from Scratch , 2015, NAACL.

[28]  Wenpeng Yin,et al.  Online Updating of Word Representations for Part-of-Speech Tagging , 2015, EMNLP.

[29]  Dongyan Zhao,et al.  Enhancing Freebase Question Answering Using Textual Evidence , 2016, ArXiv.

[30]  Xiaodong He,et al.  Character-Level Question Answering with Attention , 2016, EMNLP.

[31]  Wenpeng Yin,et al.  Attention-Based Convolutional Neural Network for Machine Comprehension , 2016, ArXiv.

[32]  Wei Xu,et al.  CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases , 2016, ACL.

[33]  Bowen Zhou,et al.  ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs , 2015, TACL.

[34]  Bowen Zhou,et al.  Attentive Pooling Networks , 2016, ArXiv.

[35]  Hinrich Schütze,et al.  Erratum: “ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs” , 2016, Transactions of the Association for Computational Linguistics.

[36]  Yoshua Bengio,et al.  Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus , 2016, ACL.