Message Passing for Hyper-Relational Knowledge Graphs

Hyper-relational knowledge graphs (KGs) (e.g., Wikidata) enable associating additional key-value pairs along with the main triple to disambiguate, or restrict the validity of a fact. In this work, we propose a message passing based graph encoder - StarE capable of modeling such hyper-relational KGs. Unlike existing approaches, StarE can encode an arbitrary number of additional information (qualifiers) along with the main triple while keeping the semantic roles of qualifiers and triples intact. We also demonstrate that existing benchmarks for evaluating link prediction (LP) performance on hyper-relational KGs suffer from fundamental flaws and thus develop a new Wikidata-based dataset - WD50K. Our experiments demonstrate that StarE based LP model outperforms existing approaches across multiple benchmarks. We also confirm that leveraging qualifiers is vital for link prediction with gains up to 25 MRR points compared to triple-based representations.

[1]  Fei Wang,et al.  Structural Deep Embedding for Hyper-Networks , 2017, AAAI.

[2]  Paolo Rosso,et al.  Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction , 2020, WWW.

[3]  Timothy M. Hospedales,et al.  TuckER: Tensor Factorization for Knowledge Graph Completion , 2019, EMNLP.

[4]  Ruochi Zhang,et al.  Hyper-SAGNN: a self-attention based graph neural network for hypergraphs , 2019, ICLR.

[5]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[6]  Philip S. Yu,et al.  A Survey on Knowledge Graphs: Representation, Acquisition and Applications , 2020, ArXiv.

[7]  Jianxin Li,et al.  On the Representation and Embedding of Knowledge Bases beyond Binary Relations , 2016, IJCAI.

[8]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[9]  Olaf Hartig,et al.  Foundations of RDF⋆ and SPARQL⋆ (An Alternative Approach to Statement-Level Metadata in RDF) , 2017, AMW.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[12]  Jian-Yun Nie,et al.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space , 2018, ICLR.

[13]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[14]  Jens Lehmann,et al.  LC-QuAD 2.0: A Large Dataset for Complex Question Answering over Wikidata and DBpedia , 2019, SEMWEB.

[15]  Graham Neubig,et al.  Latent Relation Language Models , 2019, AAAI.

[16]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[17]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[18]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[19]  Markus Krötzsch,et al.  Wikidata , 2014, Commun. ACM.

[20]  Yong Li,et al.  Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases , 2020, WWW.

[21]  Dai Quoc Nguyen,et al.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network , 2017, NAACL.

[22]  Yuanzhuo Wang,et al.  Link Prediction on N-ary Relational Data , 2019, WWW.

[23]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[24]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[25]  Jens Lehmann,et al.  Introduction to Neural Network based Approaches for Question Answering over Knowledge Graphs , 2019, ArXiv.

[26]  Jens Lehmann,et al.  Wikidata through the Eyes of DBpedia , 2015, Semantic Web.

[27]  Jens Lehmann,et al.  Incorporating Literals into Knowledge Graph Embeddings , 2018, SEMWEB.

[28]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[29]  Richong Zhang,et al.  Scalable Instance Reconstruction in Knowledge Bases via Relatedness Affiliated Embedding , 2018, WWW.

[30]  Denilson Barbosa,et al.  KnowledgeNet: A Benchmark Dataset for Knowledge Base Population , 2019, EMNLP.

[31]  Chengkai Li,et al.  Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study , 2020, SIGMOD Conference.

[32]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[33]  William W. Cohen,et al.  Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base , 2020, ICLR.

[34]  Maria-Esther Vidal,et al.  Evaluation of metadata representations in RDF stores , 2019, Semantic Web.

[35]  Pingping Huang,et al.  CoKE: Contextualized Knowledge Graph Embedding , 2019, ArXiv.

[36]  David Poole,et al.  Knowledge Hypergraphs: Prediction Beyond Binary Relations , 2019, IJCAI.

[37]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[38]  Wenhan Xiong,et al.  Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model , 2019, ICLR.

[39]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[40]  Gerhard Weikum,et al.  YAGO 4: A Reason-able Knowledge Base , 2020, ESWC.

[41]  Tianyu Gao,et al.  KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation , 2019, ArXiv.