Towards Learning Instantiated Logical Rules from Knowledge Graphs

Efficiently inducing high-level interpretable regularities from knowledge graphs (KGs) is an essential yet challenging task that benefits many downstream applications. In this work, we present GPFL, a probabilistic rule learner optimized to mine instantiated first-order logic rules from KGs. Instantiated rules contain constants extracted from KGs. Compared to abstract rules that contain no constants, instantiated rules are capable of explaining and expressing concepts in more details. GPFL utilizes a novel two-stage rule generation mechanism that first generalizes extracted paths into templates that are acyclic abstract rules until a certain degree of template saturation is achieved, then specializes the generated templates into instantiated rules. Unlike existing works that ground every mined instantiated rule for evaluation, GPFL shares groundings between structurally similar rules for collective evaluation. Moreover, we reveal the presence of overfitting rules, their impact on the predictive performance, and the effectiveness of a simple validation method filtering out overfitting rules. Through extensive experiments on public benchmark datasets, we show that GPFL 1.) significantly reduces the runtime on evaluating instantiated rules; 2.) discovers much more quality instantiated rules than existing works; 3.) improves the predictive performance of learned rules by removing overfitting rules via validation; 4.) is competitive on knowledge graph completion task compared to state-of-the-art baselines.

[1]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[2]  Jian-Yun Nie,et al.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space , 2018, ICLR.

[3]  Bowen Zhou,et al.  End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion , 2018, AAAI.

[4]  Simon Razniewski,et al.  Completeness-Aware Rule Learning from Knowledge Graphs , 2017, SEMWEB.

[5]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[6]  Li Guo,et al.  Knowledge Graph Embedding with Iterative Guidance from Soft Rules , 2017, AAAI.

[7]  Paolo Papotti,et al.  Robust Discovery of Positive and Negative Rules in Knowledge Bases , 2018, 2018 IEEE 34th International Conference on Data Engineering (ICDE).

[8]  Lina Yao,et al.  Quaternion Knowledge Graph Embeddings , 2019, NeurIPS.

[9]  Gerhard Weikum,et al.  ExFaKT: A Framework for Explaining Facts over Knowledge Graphs and Text , 2019, WSDM.

[10]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[11]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[12]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[13]  Fabian M. Suchanek,et al.  Fast rule mining in ontological knowledge bases with AMIE+\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+$$\end{docu , 2015, The VLDB Journal.

[14]  Bo Li,et al.  Rule-Guided Compositional Representation Learning on Knowledge Graphs , 2020, AAAI.

[15]  Jie Wang,et al.  Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction , 2020, AAAI.

[16]  Fan Yang,et al.  Differentiable Learning of Logical Rules for Knowledge Base Reasoning , 2017, NIPS.

[17]  Le Song,et al.  Variational Reasoning for Question Answering with Knowledge Graph , 2017, AAAI.

[18]  Daisy Zhe Wang,et al.  DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs , 2019, NeurIPS.

[19]  Vikram Nitin,et al.  InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions , 2020, AAAI.

[20]  Luc De Raedt Logical and Relational Learning , 2008, SBIA.

[21]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[22]  Wenhan Xiong,et al.  DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning , 2017, EMNLP.

[23]  Heiner Stuckenschmidt,et al.  Fine-Grained Evaluation of Rule- and Embedding-Based Systems for Knowledge Graph Completion , 2018, SEMWEB.

[24]  Luc De Raedt,et al.  ILP turns 20 , 2011, Machine Learning.

[25]  William W. Cohen,et al.  TensorLog: A Probabilistic Database Implemented Using Deep-Learning Infrastructure , 2020, J. Artif. Intell. Res..

[26]  Xiaojun Chen,et al.  A review: Knowledge reasoning over knowledge graph , 2020, Expert Syst. Appl..

[27]  Heiner Stuckenschmidt,et al.  Anytime Bottom-Up Rule Learning for Knowledge Graph Completion , 2019, IJCAI.

[28]  Vikram Nitin,et al.  Composition-based Multi-Relational Graph Convolutional Networks , 2020, ICLR.

[29]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[30]  Manohar Kaul,et al.  Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs , 2019, ACL.

[31]  Tom M. Mitchell,et al.  Efficient and Expressive Knowledge Base Completion Using Subgraph Feature Extraction , 2015, EMNLP.

[32]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[33]  Kewen Wang,et al.  Scalable Rule Learning via Learning Representation , 2018, IJCAI.

[34]  Ni Lao,et al.  Learning Relational Features with Backward Random Walks , 2015, ACL.

[35]  Timothy M. Hospedales,et al.  TuckER: Tensor Factorization for Knowledge Graph Completion , 2019, EMNLP.

[36]  Xinlei Chen,et al.  Never-Ending Learning , 2012, ECAI.

[37]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[38]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[39]  Evgeny Kharlamov,et al.  Rule Learning from Knowledge Graphs Guided by Embedding Models , 2018, SEMWEB.

[40]  Jignesh M. Patel,et al.  QuickFOIL: Scalable Inductive Logic Programming , 2014, Proc. VLDB Endow..

[41]  Steffen Staab,et al.  Knowledge graphs , 2020, Commun. ACM.

[42]  Daisy Zhe Wang,et al.  ScaLeKB: scalable learning and inference over large knowledge bases , 2016, The VLDB Journal.

[43]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[44]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[45]  Gerhard Weikum,et al.  YAGO2: A Spatially and Temporally Enhanced Knowledge Base from Wikipedia: Extended Abstract , 2013, IJCAI.

[46]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[47]  Daniel Gayo-Avello,et al.  "I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" - A Balanced Survey on Election Prediction using Twitter Data , 2012, ArXiv.

[48]  Bart Demoen,et al.  Improving the Efficiency of Inductive Logic Programming Through the Use of Query Packs , 2011, J. Artif. Intell. Res..