RAGAT: Relation Aware Graph Attention Network for Knowledge Graph Completion

Knowledge graph completion (KGC) is the task of predicting missing links based on known triples for knowledge graphs. Several recent works suggest that Graph Neural Networks (GNN) that exploit graph structures achieve promising performance on KGC. These models learn information called messages from neighboring entities and relations and then aggregate messages to update central entity representations. The drawback of existing GNN based models lies in that they tend to treat relations equally and learn fixed network parameters, overlooking the distinction of each relational information. In this work, we propose a Relation Aware Graph ATtention network (RAGAT) that constructs separate message functions for different relations, which aims at exploiting the heterogeneous characteristics of knowledge graphs. Specifically, we introduce relation specific parameters to augment the expressive capability of message functions, which enables the model to extract relational information in parameter space. To validate the effect of relation aware mechanism, RAGAT is implemented with a variety of relation aware message functions. Experiments show RAGAT outperforms state-of-the-art link prediction baselines on standard FB15k-237 and WN18RR datasets.

[1]  Jens Lehmann,et al.  DBpedia: A Nucleus for a Web of Open Data , 2007, ISWC/ASWC.

[2]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[3]  Jian-Yun Nie,et al.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space , 2018, ICLR.

[4]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[5]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[6]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[7]  Yaqian Wang,et al.  Graph2Seq: Fusion Embedding Learning for Knowledge Graph Completion , 2019, IEEE Access.

[8]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[9]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[10]  Dai Quoc Nguyen,et al.  A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network , 2017, NAACL.

[11]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[12]  Rui Ye,et al.  A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment , 2019, IJCAI.

[13]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[14]  P. Talukdar,et al.  InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions , 2019, AAAI.

[15]  Wei Zhang,et al.  Interaction Embeddings for Prediction and Explanation in Knowledge Graphs , 2019, WSDM.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[18]  Michael Gamon,et al.  Representing Text for Joint Embedding of Text and Knowledge Bases , 2015, EMNLP.

[19]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[22]  Dai Quoc Nguyen,et al.  A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization , 2018, NAACL.

[23]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[24]  Jianhua Tao,et al.  ParamE: Regarding Neural Network Parameters as Relation Embeddings for Knowledge Graph Completion , 2020, AAAI.

[25]  Ruhi Sarikaya,et al.  Knowledge Graph Inference for spoken dialog systems , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[27]  Manohar Kaul,et al.  Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs , 2019, ACL.

[28]  Gerhard Weikum,et al.  WWW 2007 / Track: Semantic Web Session: Ontologies ABSTRACT YAGO: A Core of Semantic Knowledge , 2022 .

[29]  Bowen Zhou,et al.  End-to-end Structure-Aware Convolutional Networks for Knowledge Base Completion , 2018, AAAI.

[30]  Andrew McCallum,et al.  A2N: Attending to Neighbors for Knowledge Graph Inference , 2019, ACL.

[31]  Krzysztof Janowicz,et al.  TransGCN: Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction , 2019, K-CAP.

[32]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[33]  Barnabás Póczos,et al.  Contextual Parameter Generation for Knowledge Graph Link Prediction , 2020, AAAI.

[34]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[35]  Vikram Nitin,et al.  Composition-based Multi-Relational Graph Convolutional Networks , 2020, ICLR.

[36]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[37]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[38]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[39]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[40]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[41]  Yu Hu,et al.  Probabilistic Reasoning via Deep Learning: Neural Association Models , 2016, ArXiv.

[42]  Han Xiao,et al.  TransG : A Generative Model for Knowledge Graph Embedding , 2015, ACL.

[43]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.