Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer

Predicting missing facts in a knowledge graph (KG) is a crucial task in knowledge base construction and reasoning, and it has been the subject of much research in recent works using KG embeddings. While existing KG embedding approaches mainly learn and predict facts within a single KG, a more plausible solution would benefit from the knowledge in multiple language-specific KGs, considering that different KGs have their own strengths and limitations on data quality and coverage. This is quite challenging, since the transfer of knowledge among multiple independently maintained KGs is often hindered by the insufficiency of alignment information and the inconsistency of described facts. In this paper, we propose KEnS, a novel framework for embedding learning and ensemble knowledge transfer across a number of language-specific KGs. KEnS embeds all KGs in a shared embedding space, where the association of entities is captured based on self-learning. Then, KEnS performs ensemble inference to combine prediction results from embeddings of multiple language-specific KGs, for which multiple ensemble techniques are investigated. Experiments on five real-world language-specific KGs show that KEnS consistently improves state-of-the-art methods on KG completion, via effectively identifying and leveraging complementary knowledge.

[1]  Zhichun Wang,et al.  Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks , 2018, EMNLP.

[2]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[3]  Markus Krötzsch,et al.  Wikidata , 2014, Commun. ACM.

[4]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[5]  Wei Hu,et al.  Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding , 2017, SEMWEB.

[6]  Mirella Lapata,et al.  Text Generation from Knowledge Graphs with Graph Transformers , 2019, NAACL.

[7]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[8]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[9]  Jian-Yun Nie,et al.  RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space , 2018, ICLR.

[10]  Wei Hu,et al.  Bootstrapping Entity Alignment with Knowledge Graph Embedding , 2018, IJCAI.

[11]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[12]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[15]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[16]  Wei Hu,et al.  TransEdge: Translating Relation-Contextualized Embeddings for Knowledge Graphs , 2019, SEMWEB.

[17]  Jimmy J. Lin,et al.  Aligning Cross-Lingual Entities with Multi-Aspect Information , 2019, EMNLP.

[18]  Philip S. Yu,et al.  A Survey on Knowledge Graphs: Representation, Acquisition and Applications , 2020, ArXiv.

[19]  Yuzhong Qu,et al.  Multi-view Knowledge Graph Embedding for Entity Alignment , 2019, IJCAI.

[20]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[21]  Chengkai Li,et al.  A benchmarking study of embedding-based entity alignment for knowledge graphs , 2020, Proc. VLDB Endow..

[22]  Gerhard Weikum,et al.  YAGO: A Multilingual Knowledge Base from Wikipedia, Wordnet, and Geonames , 2016, SEMWEB.

[23]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[24]  Guillaume Lample,et al.  Word Translation Without Parallel Data , 2017, ICLR.

[25]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[26]  Yizhou Sun,et al.  Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts , 2019, KDD.

[27]  Jiawei Han,et al.  SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble , 2017, ECML/PKDD.

[28]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[29]  Jason Weston,et al.  Open Question Answering with Weakly Supervised Embedding Models , 2014, ECML/PKDD.

[30]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[31]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[32]  Steven Skiena,et al.  Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment , 2018, IJCAI.

[33]  Yizhou Sun,et al.  Bio-JOIE: Joint Representation Learning of Biological Knowledge Bases , 2020 .

[34]  Yizhou Sun,et al.  Embedding Uncertain Knowledge Graphs , 2018, AAAI.

[35]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[36]  Wei Hu,et al.  Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation , 2019, AAAI.

[37]  Chengjiang Li,et al.  Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model , 2019, EMNLP.

[38]  Carlo Zaniolo,et al.  Neural Article Pair Modeling for Wikipedia Sub-article Matching , 2018, ECML/PKDD.

[39]  Rui Zhang,et al.  Entity Alignment between Knowledge Graphs Using Attribute Embeddings , 2019, AAAI.

[40]  Chengjiang Li,et al.  Multi-Channel Graph Neural Network for Entity Alignment , 2019, ACL.

[41]  Carlo Zaniolo,et al.  Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment , 2016, IJCAI.

[42]  Alex A. T. Bui,et al.  Diagnostic Prediction with Sequence-of-setsRepresentation Learning for Clinical Events , 2020, medRxiv.

[43]  Serge Abiteboul,et al.  PARIS: Probabilistic Alignment of Relations, Instances, and Schema , 2011, Proc. VLDB Endow..

[44]  Lu Yu,et al.  Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference , 2019, WWW.