Quantum Embedding of Knowledge for Reasoning

Statistical Relational Learning (SRL) methods are the most widely used techniques to generate distributional representations of the symbolic Knowledge Bases (KBs). These methods embed any given KB into a vector space by exploiting statistical similarities among its entities and predicates but without any guarantee of preserving the underlying logical structure of the KB. This, in turn, results in poor performance of logical reasoning tasks that are solved using such distributional representations. We present a novel approach called Embed2Reason (E2R) that embeds a symbolic KB into a vector space in a logical structure preserving manner. This approach is inspired by the theory of Quantum Logic. Such an embedding allows answering membership based complex logical reasoning queries with impressive accuracy improvements over popular SRL baselines.

[1]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[2]  Philippe Roussel,et al.  The birth of Prolog , 1993, HOPL-II.

[3]  Dominic Widdows,et al.  Orthogonal Negation in Vector Spaces for Modelling Word-Meanings and Document Retrieval , 2003, ACL.

[4]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[5]  Jure Leskovec,et al.  Embedding Logical Queries on Knowledge Graphs , 2018, NeurIPS.

[6]  Steven Schockaert,et al.  Inductive Reasoning about Ontologies Using Conceptual Spaces , 2017, AAAI.

[7]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[8]  Jim Hefferon,et al.  Linear Algebra , 2012 .

[9]  Tim Rocktäschel,et al.  End-to-end Differentiable Proving , 2017, NIPS.

[10]  Mohammad Sadoghi,et al.  EmbedS: Scalable, Ontology-aware Graph Embeddings , 2018, EDBT.

[11]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  Hector J. Levesque,et al.  Knowledge Representation and Reasoning , 2004 .

[14]  Danqi Chen,et al.  CoQA: A Conversational Question Answering Challenge , 2018, TACL.

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  Stephen Muggleton,et al.  Inductive Logic Programming , 2011, Lecture Notes in Computer Science.

[17]  Steven Schockaert,et al.  Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoning , 2016, ECAI.

[18]  Stephen H. Bach Hinge-Loss Markov Random Fields and Probabilistic Soft Logic: A Scalable Approach to Structured Prediction , 2015, J. Mach. Learn. Res..

[19]  Bernhard Ganter,et al.  Formal Concept Analysis: Mathematical Foundations , 1998 .

[20]  Dominic Widdows,et al.  Geometry and Meaning , 2004, Computational Linguistics.

[21]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[22]  Zhiyuan Liu,et al.  Representation Learning of Knowledge Graphs with Hierarchical Types , 2016, IJCAI.

[23]  V. Varadarajan Geometry of quantum theory , 1968 .

[24]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[25]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[28]  Sameer Singh,et al.  Low-Dimensional Embeddings of Logic , 2014, ACL 2014.

[29]  Sameer Singh,et al.  Injecting Logical Background Knowledge into Embeddings for Relation Extraction , 2015, NAACL.

[30]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[31]  D. W. Cohen,et al.  An Introduction to Hilbert Space and Quantum Logic , 1989 .

[32]  Thomas Lukasiewicz,et al.  Deep Learning for Ontology Reasoning , 2017, ArXiv.

[33]  Thomas Demeester,et al.  Lifted Rule Injection for Relation Embeddings , 2016, EMNLP.

[34]  Ian Horrocks,et al.  An Introduction to Description Logic , 2017 .

[35]  J. Neumann,et al.  The Logic of Quantum Mechanics , 1936 .

[36]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[37]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.