Anytime Bottom-Up Rule Learning for Knowledge Graph Completion

Most of today’s work on knowledge graph completion is concerned with sub-symbolic approaches that focus on the concept of embedding a given graph in a low dimensional vector space. Against this trend, we propose an approach called AnyBURL that is rooted in the symbolic space. Its core algorithm is based on sampling paths, which are generalized into Horn rules. Previously published results show that the prediction quality of AnyBURL is on the same level as current state of the art with the additional benefit of offering an explanation for the predicted fact. In this paper, we are concerned with two extensions of AnyBURL. Firstly, we change AnyBURL’s interpretation of rules from Θ-subsumption into Θ-subsumption under Object Identity. Secondly, we introduce reinforcement learning to better guide the sampling process. We found out that reinforcement learning helps finding more valuable rules earlier in the search process. We measure the impact of both extensions and compare the resulting approach with current state of the art approaches. Our results show that AnyBURL outperforms most sub-symbolic methods.

[1]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[2]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[3]  Donato Malerba,et al.  Refinement of Datalog Programs , 2007 .

[4]  Fabian M. Suchanek,et al.  AMIE: association rule mining under incomplete evidence in ontological knowledge bases , 2013, WWW.

[5]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Fabian M. Suchanek,et al.  Fast rule mining in ontological knowledge bases with AMIE+\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+$$\end{docu , 2015, The VLDB Journal.

[8]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[9]  Wei Zhang,et al.  Interaction Embeddings for Prediction and Explanation in Knowledge Graphs , 2019, WSDM.

[10]  Heiner Stuckenschmidt,et al.  Fine-Grained Evaluation of Rule- and Embedding-Based Systems for Knowledge Graph Completion , 2018, SEMWEB.

[11]  Hugo Larochelle,et al.  Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality , 2015, CVSC.

[12]  Stefano Pallottino,et al.  Shortest-path methods: Complexity, interrelations and new propositions , 1984, Networks.

[13]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[14]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[15]  Rainer Gemulla,et al.  You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings , 2020, ICLR.

[16]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[17]  Vldb Endowment,et al.  The VLDB journal : the international journal on very large data bases. , 1992 .

[18]  Jens Lehmann,et al.  DBpedia: A Nucleus for a Web of Open Data , 2007, ISWC/ASWC.

[19]  Donato Malerba,et al.  Avoiding Non-Termination when Learning Logical Programs: A Case Study with FOIL and FOCL , 1994, LOPSTR.

[20]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[21]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[22]  Michael N. Katehakis,et al.  The Multi-Armed Bandit Problem: Decomposition and Computation , 1987, Math. Oper. Res..

[23]  Ni Lao,et al.  Learning Relational Features with Backward Random Walks , 2015, ACL.

[24]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[25]  Paolo Merialdo,et al.  Knowledge Graph Embedding for Link Prediction , 2020, ACM Transactions on Knowledge Discovery from Data.

[26]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[27]  Luc Dehaspe,et al.  Discovery of relational association rules , 2001 .

[28]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[29]  Li Guo,et al.  Knowledge Graph Embedding with Iterative Guidance from Soft Rules , 2017, AAAI.

[30]  Richard Socher,et al.  Multi-Hop Knowledge Graph Reasoning with Reward Shaping , 2018, EMNLP.

[31]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[32]  Fabian M. Suchanek,et al.  YAGO3: A Knowledge Base from Multilingual Wikipedias , 2015, CIDR.

[33]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[34]  J. A. Robinson,et al.  A Machine-Oriented Logic Based on the Resolution Principle , 1965, JACM.

[35]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.