Adversarial Training Improved Multi-Path Multi-Scale Relation Detector for Knowledge Base Question Answering

Knowledge Base Question Answering (KBQA) is a promising approach for users to access substantial knowledge and has become a research focus in recent years. Our paper focuses on relation detection, a subtask of KBQA and proposes an adversarial training improved multi-path multi-scale relation detector (AdvT-MMRD) to improve the performance of a common KBQA system. To solve the problem of matching the casual form of a question with the logical form of a predicate, we use question pattern-relation matching, in which an attention-based bidirectional recurrent neural network with gated recurrent units (Bi-GRUs) is used to match semantic similarity and a convolutional neural network (CNN) is used to learn literal similarity between question and relation. We also explore two ways to measure the relevance of entity type-relation pairs through several level representations. Additionally, an adversarial training strategy is conducted to enhance our model. The experimental results demonstrate that our approach not only achieves a state-of-the-art accuracy of 93.8% on relation detection task, but contributes our KBQA system to reaching an outstanding accuracy of 79.0% on the SimpleQuestions benchmark.

[1]  Thomas Demeester,et al.  Adversarial training for multi-context joint entity and relation extraction , 2018, EMNLP.

[2]  Hua Wu,et al.  Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information , 2016, ArXiv.

[3]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[4]  David Bamman,et al.  Adversarial Training for Relation Extraction , 2017, EMNLP.

[5]  Sumio Fujita,et al.  Dual Constrained Question Embeddings with Relational Knowledge Bases for Simple Question Answering , 2017, IJCNLP.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[8]  Wei Xu,et al.  CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases , 2016, ACL.

[9]  Andrew M. Dai,et al.  Adversarial Training Methods for Semi-Supervised Text Classification , 2016, ICLR.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Hui Jiang,et al.  A General FOFE-net Framework for Simple and Effective Question Answering over Knowledge Bases , 2019, ArXiv.

[12]  Bowen Zhou,et al.  Improved Neural Relation Detection for Knowledge Base Question Answering , 2017, ACL.

[13]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[14]  Iryna Gurevych,et al.  Modeling Semantics with Gated Graph Neural Networks for Knowledge Base Question Answering , 2018, COLING.

[15]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.

[16]  Zhen-Hua Ling,et al.  Knowledge Base Question Answering With Attentive Pooling for Question Representation , 2019, IEEE Access.

[17]  Rajarshi Das,et al.  Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks , 2017, ACL.

[18]  Hiroyuki Shindo,et al.  Interpretable Adversarial Perturbation in Input Embedding Space for Text , 2018, IJCAI.

[19]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[20]  Wei Zhang,et al.  Knowledge Base Relation Detection via Multi-View Matching , 2018, ADBIS.

[21]  Madian Khabsa,et al.  Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching , 2018, AAAI.

[22]  Jens Lehmann,et al.  Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level , 2017, WWW.

[23]  Ming Zhou,et al.  Question Answering over Freebase with Multi-Column Convolutional Neural Networks , 2015, ACL.

[24]  Yue Wang,et al.  The APVA-TURBO Approach To Question Answering in Knowledge Base , 2018, COLING.

[25]  Guandong Xu,et al.  An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering , 2018, ArXiv.

[26]  Hao Liu,et al.  Pattern-revising Enhanced Simple Question Answering over Knowledge Bases , 2018, COLING.

[27]  Jonathan Berant,et al.  Semantic Parsing via Paraphrasing , 2014, ACL.

[28]  Dongyan Zhao,et al.  Question Answering on Freebase via Relation Extraction and Textual Evidence , 2016, ACL.

[29]  Xuchen Yao,et al.  Information Extraction over Structured Data: Question Answering with Freebase , 2014, ACL.

[30]  Hannah Bast,et al.  More Accurate Question Answering on Freebase , 2015, CIKM.

[31]  Xiaodong He,et al.  Character-Level Question Answering with Attention , 2016, EMNLP.

[32]  Luke S. Zettlemoyer,et al.  SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach , 2018, EMNLP.

[33]  Ming-Wei Chang,et al.  Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base , 2015, ACL.

[34]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[35]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[36]  Jason Weston,et al.  Large-scale Simple Question Answering with Memory Networks , 2015, ArXiv.

[37]  Jie Liu,et al.  Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN , 2018, ArXiv.

[38]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[39]  Bowen Zhou,et al.  Simple Question Answering by Attentive Convolutional Neural Network , 2016, COLING.

[40]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[41]  Xuchen Yao,et al.  Lean Question Answering over Freebase from Scratch , 2015, NAACL.

[42]  Angeliki Metallinou,et al.  Simple Question Answering with Subgraph Ranking and Joint-Scoring , 2019, NAACL.

[43]  Kenny Q. Zhu,et al.  Knowledge Base Question Answering via Encoding of Complex Query Graphs , 2018, EMNLP.

[44]  Tong Zhang,et al.  Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings , 2016, ICML.

[45]  Vishal Gupta,et al.  Retrieve and Re-rank: A Simple and Effective IR Approach to Simple Question Answering over Knowledge Graphs , 2018 .

[46]  Jungo Kasai,et al.  Robust Multilingual Part-of-Speech Tagging via Adversarial Training , 2017, NAACL.