Multiple Interaction Attention Model for Open-World Knowledge Graph Completion

Knowledge Graph Completion (KGC) aims at complementing missing relationships between entities in a Knowledge Graph (KG). While closed-world KGC approaches utilizing the knowledge within KG could only complement very limited number of missing relations, more and more approaches tend to get knowledge from open-world resources such as online encyclopedias and newswire corpus. For instance, a recent proposed open-world KGC model called ConMask learns embeddings of the entity’s name and parts of its text-description to connect unseen entities to the KG. However, this model does not make full use of the rich feature information in the text descriptions, besides, the proposed relationship-dependent content masking method may easily miss to find the target-words. In this paper, we propose to use a Multiple Interaction Attention (MIA) mechanism to model the interactions between the head entity description, head entity name, the relationship name, and the candidate tail entity descriptions, to form the enriched representations. Our empirical study conducted on two real-world data collections shows that our approach achieves significant improvements comparing to state-of-the-art KGC methods.

[1]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[2]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[3]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[4]  Rong Liu,et al.  Joint Semantic Relevance Learning with Text Data and Graph Knowledge , 2015, CVSC.

[5]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[6]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[7]  Jason Weston,et al.  Reading Wikipedia to Answer Open-Domain Questions , 2017, ACL.

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[10]  Joel Nothman,et al.  Evaluating Entity Linking with Wikipedia , 2013, Artif. Intell..

[11]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[12]  Mehmet A. Orgun,et al.  Finding the Optimal Social Trust Path for the Selection of Trustworthy Service Providers in Complex Social Networks , 2013, IEEE Trans. Serv. Comput..

[13]  Zhiyuan Liu,et al.  Representation Learning of Knowledge Graphs with Entity Descriptions , 2016, AAAI.

[14]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[15]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[16]  Tim Weninger,et al.  Open-World Knowledge Graph Completion , 2017, AAAI.

[17]  Mehmet A. Orgun,et al.  Optimal Social Trust Path Selection in Complex Social Networks , 2010, AAAI.

[18]  Wei Zhao,et al.  Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension , 2018, SemEval@NAACL-HLT.

[19]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[20]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[21]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[22]  SchmidhuberJürgen,et al.  2005 Special Issue , 2005 .

[23]  Jens Lehmann,et al.  Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level , 2017, WWW.

[24]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[25]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[26]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[27]  Kan Chen,et al.  Knowledge Graph Representation with Jointly Structural and Textual Encoding , 2016, IJCAI.

[28]  Minyi Guo,et al.  RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems , 2018, CIKM.