Faithful Embeddings for Knowledge Base Queries

The deductive closure of an ideal knowledge base (KB) contains exactly the logical queries that the KB can answer. However, in practice KBs are both incomplete and over-specified, failing to answer some queries that have real-world answers. \emph{Query embedding} (QE) techniques have been recently proposed where KB entities and KB queries are represented jointly in an embedding space, supporting relaxation and generalization in KB inference. However, experiments in this paper show that QE systems may disagree with deductive reasoning on answers that do not require generalization or relaxation. We address this problem with a novel QE method that is more faithful to deductive reasoning, and show that this leads to better performance on complex queries to incomplete KBs. Finally we show that inserting this new QE module into a neural question-answering system leads to substantial improvements over the state-of-the-art.

[1]  Karl Svozil,et al.  Quantum Logic , 1998, Discrete mathematics and theoretical computer science.

[2]  Stefano Ermon,et al.  Learning and Inference via Maximum Inner Product Search , 2016, ICML.

[3]  Andrew McCallum,et al.  Word Representations via Gaussian Embedding , 2014, ICLR.

[4]  Yiming Yang,et al.  Analogical Inference for Multi-relational Embeddings , 2017, ICML.

[5]  Fan Yang,et al.  TensorLog: Deep Learning Meets Probabilistic DBs , 2017, ArXiv.

[6]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[7]  Rajarshi Das,et al.  Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks , 2016, EACL.

[8]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[9]  Meng Wang,et al.  Towards Empty Answers in SPARQL: Approximating Querying with RDF Embedding , 2018, SEMWEB.

[10]  Xiang Li,et al.  Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures , 2018, ACL.

[11]  William W. Cohen,et al.  Neural Query Language: A Knowledge Base Query Language for Tensorflow , 2019, ArXiv.

[12]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[13]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[14]  Thomas Demeester,et al.  Lifted Rule Injection for Relation Embeddings , 2016, EMNLP.

[15]  John Miller,et al.  Traversing Knowledge Graphs in Vector Space , 2015, EMNLP.

[16]  Jure Leskovec,et al.  Embedding Logical Queries on Knowledge Graphs , 2018, NeurIPS.

[17]  William W. Cohen,et al.  PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text , 2019, EMNLP.

[18]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[19]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[20]  Le Song,et al.  Variational Reasoning for Question Answering with Knowledge Graph , 2017, AAAI.

[21]  Apoorv Saxena,et al.  Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings , 2020, ACL.

[22]  Ming-Wei Chang,et al.  Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base , 2015, ACL.

[23]  William W. Cohen,et al.  Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base , 2020, ICLR.

[24]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[25]  Jure Leskovec,et al.  Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings , 2020, ICLR.

[26]  Benjamin Van Durme,et al.  Training Relation Embeddings under Logical Constraints , 2017, KG4IR@SIGIR.

[27]  Zhiyuan Liu,et al.  OpenKE: An Open Toolkit for Knowledge Embedding , 2018, EMNLP.

[28]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[29]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[30]  Ruslan Salakhutdinov,et al.  Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text , 2018, EMNLP.

[31]  Yoram Singer,et al.  Sketching and Neural Networks , 2016, ArXiv.