Knowledge Association with Hyperbolic Knowledge Graph Embeddings

Capturing associations for knowledge graphs (KGs) through entity alignment, entity type inference and other related tasks benefits NLP applications with comprehensive knowledge representations. Recent related methods built on Euclidean embeddings are challenged by the hierarchical structures and different scales of KGs. They also depend on high embedding dimensions to realize enough expressiveness. Differently, we explore with low-dimensional hyperbolic embeddings for knowledge association. We propose a hyperbolic relational graph neural network for KG embedding and capture knowledge associations with a hyperbolic transformation. Extensive experiments on entity alignment and type inference demonstrate the effectiveness and efficiency of our method.

[1]  Carlo Zaniolo,et al.  Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment , 2016, IJCAI.

[2]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[3]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  Gao Cong,et al.  Hyperbolic Recommender Systems , 2018, ArXiv.

[5]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[6]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[7]  Da Xu,et al.  Inductive Representation Learning on Temporal Graphs , 2020, ICLR.

[8]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[9]  Zhichun Wang,et al.  Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks , 2018, EMNLP.

[10]  W. Floyd,et al.  HYPERBOLIC GEOMETRY , 1996 .

[11]  Christos Faloutsos,et al.  LinkNBed: Multi-Graph Representation Learning with Entity Linkage , 2018, ACL.

[12]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[13]  Yizhou Sun,et al.  Bio-JOIE: Joint Representation Learning of Biological Knowledge Bases , 2020 .

[14]  Yuting Wu,et al.  Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs , 2019, IJCAI.

[15]  Xiangliang Zhang,et al.  Improving Cross-lingual Entity Alignment via Optimal Transport , 2019, IJCAI.

[16]  Seungwhan Moon,et al.  OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs , 2019, ACL.

[17]  Andrew M. Dai,et al.  Embedding Text in Hyperbolic Spaces , 2018, TextGraphs@NAACL-HLT.

[18]  Heiko Paulheim,et al.  Type Inference on Noisy RDF Data , 2013, SEMWEB.

[19]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Jimmy J. Lin,et al.  Aligning Cross-Lingual Entities with Multi-Aspect Information , 2019, EMNLP.

[22]  Wei Hu,et al.  TransEdge: Translating Relation-Contextualized Embeddings for Knowledge Graphs , 2019, SEMWEB.

[23]  Yizhou Sun,et al.  Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts , 2019, KDD.

[24]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[25]  Tiansi Dong,et al.  Fine-Grained Entity Typing via Hierarchical Multi Graph Convolutional Networks , 2019, EMNLP/IJCNLP.

[26]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[27]  Christopher De Sa,et al.  Representation Tradeoffs for Hyperbolic Embeddings , 2018, ICML.

[28]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[29]  Xueyan Jiang,et al.  Reducing the Rank in Relational Factorization Models by Including Observable Patterns , 2014, NIPS.

[30]  Chris Quirk,et al.  Embedding Edge-attributed Relational Hierarchies , 2019, SIGIR.

[31]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[32]  Zhiyuan Liu,et al.  Iterative Entity Alignment via Joint Knowledge Embeddings , 2017, IJCAI.

[33]  Matt Le,et al.  Inferring Concept Hierarchies from Text Corpora via Hyperbolic Embeddings , 2019, ACL.

[34]  Wenting Wang,et al.  MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-lingual Knowledge Graph , 2020, WSDM.

[35]  James W. Anderson,et al.  Hyperbolic geometry , 1999 .

[36]  Shantanu Acharya,et al.  Every Child Should Have Parents: A Taxonomy Refinement Algorithm Based on Hyperbolic Term Embeddings , 2019, ACL.

[37]  Mehwish Alam,et al.  A Survey on Knowledge Graph Embeddings with Literals: Which model links better Literal-ly? , 2019, ArXiv.

[38]  Eric P. Xing,et al.  Entity Hierarchy Embedding , 2015, ACL.

[39]  Gjergji Kasneci,et al.  SIGMa: simple greedy matching for aligning large knowledge bases , 2012, KDD.

[40]  Wei Hu,et al.  Bootstrapping Entity Alignment with Knowledge Graph Embedding , 2018, IJCAI.

[41]  Bin Wang,et al.  Guiding Cross-lingual Entity Alignment via Adversarial Knowledge Embedding , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[42]  Lu Yu,et al.  Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference , 2019, WWW.

[43]  Frank Nielsen,et al.  Visualizing hyperbolic Voronoi diagrams , 2014, SoCG.

[44]  Serge Abiteboul,et al.  PARIS: Probabilistic Alignment of Relations, Instances, and Schema , 2011, Proc. VLDB Endow..

[45]  Rui Zhang,et al.  Entity Alignment between Knowledge Graphs Using Attribute Embeddings , 2019, AAAI.

[46]  Dongyan Zhao,et al.  Jointly Learning Entity and Relation Representations for Entity Alignment , 2019, EMNLP.

[47]  Wei Hu,et al.  Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation , 2019, AAAI.

[48]  Chengjiang Li,et al.  Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model , 2019, EMNLP.

[49]  Steven Skiena,et al.  Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment , 2018, IJCAI.

[50]  Timothy M. Hospedales,et al.  Multi-relational Poincaré Graph Embeddings , 2019, NeurIPS.

[51]  Enrico Motta,et al.  Overcoming Schema Heterogeneity between Linked Semantic Repositories to Improve Coreference Resolution , 2009, ASWC.

[52]  Tiansi Dong,et al.  Triple Classification Using Regions and Fine-Grained Entity Typing , 2019, AAAI.

[53]  Wei Hu,et al.  Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs , 2019, ICML.

[54]  Yuzhong Qu,et al.  Multi-view Knowledge Graph Embedding for Entity Alignment , 2019, IJCAI.

[55]  Wei Hu,et al.  Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding , 2017, SEMWEB.

[56]  Zhou Yu,et al.  Incorporating Structured Commonsense Knowledge in Story Completion , 2018, AAAI.

[57]  Chengkai Li,et al.  A benchmarking study of embedding-based entity alignment for knowledge graphs , 2020, Proc. VLDB Endow..

[58]  Rui Ye,et al.  A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment , 2019, IJCAI.

[59]  Gary Bécigneul,et al.  Poincaré GloVe: Hyperbolic Word Embeddings , 2018, ICLR.

[60]  Zhiyuan Liu,et al.  Differentiating Concepts and Instances for Knowledge Graph Embedding , 2018, EMNLP.

[61]  William F. Reynolds,et al.  Hyperbolic geometry on a hyperboloid , 1993 .

[62]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[63]  Peter Clark,et al.  Learning Knowledge Graphs for Question Answering through Conversational Dialog , 2015, NAACL.

[64]  Xinlei Chen,et al.  Never-Ending Learning , 2012, ECAI.

[65]  Chengjiang Li,et al.  Multi-Channel Graph Neural Network for Entity Alignment , 2019, ACL.

[66]  Xiaofei Zhou,et al.  Neighborhood-Aware Attentional Representation for Multilingual Knowledge Graphs , 2019, IJCAI.

[67]  Yanghua Xiao,et al.  Modeling Multi-mapping Relations for Precise Cross-lingual Entity Alignment , 2019, EMNLP.

[68]  Alexandros Kalousis,et al.  Hyperbolic Knowledge Graph Embeddings for Knowledge Base Completion , 2019, ESWC.