ICLEA: Interactive Contrastive Learning for Self-supervised Entity Alignment

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without seed alignments. The current SOTA self-supervised EA method draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, KGs contain rich side information (e.g., entity description), and how to effectively leverage those information has not been adequately investigated in self-supervised EA. In this paper, we propose an interactive contrastive learning model for self-supervised EA. The model encodes not only structures and semantics of entities (including entity name, entity description, and entity neighborhood), but also conducts cross-KG contrastive learning by building pseudo-aligned entity pairs. Experimental results show that our approach outperforms previous best self-supervised results by a large margin (over 9% average improvement) and performs on par with previous SOTA supervised counterparts, demonstrating the effectiveness of the interactive contrastive learning for self-supervised EA.

[1]  Xianpei Han,et al.  Global Structure and Local Semantics-Preserved Embeddings for Entity Alignment , 2020, IJCAI.

[2]  Evgeny Kharlamov,et al.  A Self-supervised Method for Entity Alignment , 2021, ArXiv.

[3]  Zhichun Wang,et al.  Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks , 2018, EMNLP.

[4]  Chengkai Li,et al.  A benchmarking study of embedding-based entity alignment for knowledge graphs , 2020, Proc. VLDB Endow..

[5]  Yuting Wu,et al.  Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs , 2019, IJCAI.

[6]  Junnan Li,et al.  Prototypical Contrastive Learning of Unsupervised Representations , 2020, ICLR.

[7]  Chengjiang Li,et al.  Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model , 2019, EMNLP.

[8]  Xiangliang Zhang,et al.  REA: Robust Cross-lingual Entity Alignment Between Knowledge Graphs , 2020, KDD.

[9]  Jiuyang Tang,et al.  Reinforced Active Entity Alignment , 2021, CIKM.

[10]  Wei Hu,et al.  Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation , 2019, AAAI.

[11]  Seung-won Hwang,et al.  KBQA: Learning Question Answering over QA Corpora and Knowledge Bases , 2019, Proc. VLDB Endow..

[12]  Guido Zuccon,et al.  ActiveEA: Active Learning for Neural Entity Alignment , 2021, EMNLP.

[13]  Yi Li,et al.  RiMOM: A Dynamic Multistrategy Ontology Alignment Framework , 2009, IEEE Transactions on Knowledge and Data Engineering.

[14]  Chengjiang Li,et al.  Multi-Channel Graph Neural Network for Entity Alignment , 2019, ACL.

[15]  Huaping Liu,et al.  Understanding the Behaviour of Contrastive Loss , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yiming Yang,et al.  Knowledge Embedding Based Graph Convolutional Network , 2021, WWW.

[17]  Zhiyuan Liu,et al.  Iterative Entity Alignment via Joint Knowledge Embeddings , 2017, IJCAI.

[18]  Yuanbin Wu,et al.  Are Negative Samples Necessary in Entity Alignment?: An Approach with High Performance, Scalability and Robustness , 2021, CIKM.

[19]  Wei Hu,et al.  Cross-Lingual Entity Alignment via Joint Attribute-Preserving Embedding , 2017, SEMWEB.

[20]  Yansong Feng,et al.  Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network , 2019, ACL.

[21]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[22]  Christos Faloutsos,et al.  Collective Multi-type Entity Alignment Between Knowledge Graphs , 2020, WWW.

[23]  Jens Lehmann,et al.  DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia , 2015, Semantic Web.

[24]  Nigel Collier,et al.  Visual Pivoting for (Unsupervised) Entity Alignment , 2020, AAAI.

[25]  Yanghua Xiao,et al.  Modeling Multi-mapping Relations for Precise Cross-lingual Entity Alignment , 2019, EMNLP.

[26]  Yang Yang,et al.  BERT-INT: A BERT-based Interaction Model For Knowledge Graph Alignment , 2020, IJCAI.

[27]  Yixin Cao,et al.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences , 2019, WWW.

[28]  Jiuyang Tang,et al.  Collective Embedding-based Entity Alignment via Adaptive Features , 2019, ArXiv.

[29]  Markus Krötzsch,et al.  Wikidata , 2014, Commun. ACM.

[30]  Naveen Arivazhagan,et al.  Language-agnostic BERT Sentence Embedding , 2020, ArXiv.

[31]  Dongyan Zhao,et al.  Jointly Learning Entity and Relation Representations for Entity Alignment , 2019, EMNLP.

[32]  Zhiyuan Liu,et al.  Exploring and Evaluating Attributes, Values, and Structure for Entity Alignment , 2020, EMNLP.

[33]  Zhichun Wang,et al.  Knowledge Graph Alignment with Entity-Pair Embedding , 2020, EMNLP.

[34]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Carlo Zaniolo,et al.  Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment , 2016, IJCAI.

[36]  Dan Roth,et al.  Cross-lingual Entity Alignment with Incidental Supervision , 2021, EACL.

[37]  Jie Tang,et al.  Self-Supervised Learning: Generative or Contrastive , 2020, IEEE Transactions on Knowledge and Data Engineering.

[38]  Wenting Wang,et al.  MRAEA: An Efficient and Robust Entity Alignment Approach for Cross-lingual Knowledge Graph , 2020, WSDM.

[39]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[40]  Wei Hu,et al.  Bootstrapping Entity Alignment with Knowledge Graph Embedding , 2018, IJCAI.

[41]  Jimmy J. Lin,et al.  Aligning Cross-Lingual Entities with Multi-Aspect Information , 2019, EMNLP.

[42]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[43]  Yuzhong Qu,et al.  Multi-view Knowledge Graph Embedding for Entity Alignment , 2019, IJCAI.

[44]  Wenting Wang,et al.  Relational Reflection Entity Alignment , 2020, CIKM.

[45]  Zhonghai Wu,et al.  Relation-Aware Neighborhood Matching Model for Entity Alignment , 2020, AAAI.