Contrastive Multi-View Multiplex Network Embedding with Applications to Robust Network Alignment

Despite its success in learning network node representations, network embedding is still relatively new for multiplex networks (MNs) with multiple types of edges. In such networks, the inter-layer anchor links are usually missing, which represent the alignment relations between nodes on different layers and are a crucial prerequisite for many cross-network applications like network alignment. For mining such anchor links between layers for MNs, multiplex network embedding (MNE) has become one of the most promising techniques. In this paper, we consider two problems for MNs: 1) edges can be missing to different extent, and data augmentation may mitigate this issue; 2) the known alignment anchor links between layers can be misleading since the behaviors of nodes on different layers are not always consistent, so the most informative ones should be emphasized compared with those misleading ones. However, most existing works neglect the two problems and simply 1) adopt one structural view for all the layers (e.g. random walk with the same window size) and 2) equally extract information from all the anchor links. We propose an end-to-end contrastive framework called cM2NE for MNE, utilizing multiple structural views for each layer and learning with several plug-in components for different scenarios. Through end-to-end optimization on three levels, the intra-view, inter-view, and inter-layer level, our framework achieves to select the fitted views for different layers and maximize the inter-layer mutual information by emphasizing those most informative anchor links. Extensive experimental results on real-world datasets for node classification and multi-network alignment show that our approach consistently outperforms peer methods.

[1]  Philip S. Yu,et al.  Inferring anchor links across multiple heterogeneous social networks , 2013, CIKM.

[2]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Yu Meng,et al.  Spherical Text Embedding , 2019, NeurIPS.

[5]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[6]  Vito Latora,et al.  Structural reducibility of multilayer networks , 2015, Nature Communications.

[7]  Rui Zhang,et al.  Cross-Network Skip-Gram Embedding for Joint Network Alignment and Link Prediction , 2020, IEEE Transactions on Knowledge and Data Engineering.

[8]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[9]  Zhangyang Wang,et al.  Graph Contrastive Learning with Augmentations , 2020, NeurIPS.

[10]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[11]  Vasant Honavar,et al.  MEGAN: A Generative Adversarial Network for Multi-View Network Embedding , 2019, IJCAI.

[12]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[13]  Katarzyna Musial,et al.  Multi-level Graph Convolutional Networks for Cross-platform Anchor Link Prediction , 2020, KDD.

[14]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[15]  Hanghang Tong,et al.  FINAL: Fast Attributed Network Alignment , 2016, KDD.

[16]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[17]  G. Karypis,et al.  DGL-KE: Training Knowledge Graph Embeddings at Scale , 2020, SIGIR.

[18]  Mark Heimann,et al.  REGAL: Representation Learning-based Graph Alignment , 2018, CIKM.

[19]  Steven C. H. Hoi,et al.  Time-Warped Sparse Non-negative Factorization for Functional Data Analysis , 2020, ACM Trans. Knowl. Discov. Data.

[20]  Junchi Yan,et al.  BTWalk: Branching Tree Random Walk for Multi-Order Structured Network Embedding , 2022, IEEE Transactions on Knowledge and Data Engineering.

[21]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[22]  Bonnie Berger,et al.  Global alignment of multiple protein interaction networks with application to functional orthology detection , 2008, Proceedings of the National Academy of Sciences.

[23]  Zengfeng Huang,et al.  SCE: Scalable Network Embedding from Sparsest Cut , 2020, KDD.

[24]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[25]  Jiawei Han,et al.  mvn2vec: Preservation and Collaboration in Multi-View Network Embedding , 2018, ArXiv.

[26]  Yan Wang,et al.  ProNE: Fast and Scalable Network Representation Learning , 2019, IJCAI.

[27]  Zhewei Wei,et al.  Scalable Graph Embeddings via Sparse Transpose Proximities , 2019, KDD.

[28]  Jiawei Han,et al.  An Attention-based Collaboration Framework for Multi-View Network Representation Learning , 2017, CIKM.

[29]  Xiaolong Jin,et al.  Predict Anchor Links across Social Networks via an Embedding Approach , 2016, IJCAI.

[30]  Phillip Isola,et al.  Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere , 2020, ICML.

[31]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[32]  Yong Yu,et al.  BASS: A Bootstrapping Approach for Aligning Heterogenous Social Networks , 2016, ECML/PKDD.

[33]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[34]  Incomplete Network Alignment: Problem Definitions and Fast Solutions , 2020 .

[35]  Donghyun Kim,et al.  Unsupervised Attributed Multiplex Network Embedding , 2020, AAAI.

[36]  Jie Tang,et al.  Representation Learning for Attributed Multiplex Heterogeneous Network , 2019, KDD.

[37]  Phillip Isola,et al.  Contrastive Multiview Coding , 2019, ECCV.

[38]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[39]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[40]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[41]  Yong Yu,et al.  ASNets: A Benchmark Dataset of Aligned Social Networks for Cross-Platform User Modeling , 2016, CIKM.

[42]  Philip S. Yu,et al.  Integrated Anchor and Social Link Predictions across Social Networks , 2015, IJCAI.

[43]  Li Liu,et al.  Aligning Users across Social Networks Using Network Embedding , 2016, IJCAI.

[44]  Hongyuan Zha,et al.  Joint Link Prediction and Network Alignment via Cross-graph Embedding , 2019, IJCAI.

[45]  Tsuyoshi Murata,et al.  MELL: Effective Embedding Method for Multiplex Networks , 2018, WWW.

[46]  Jian Li,et al.  NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization , 2019, WWW.

[47]  Weiyi Liu,et al.  Principled Multilayer Network Embedding , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[48]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[49]  Alex Smola,et al.  Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs , 2019, ArXiv.

[50]  Philip S. Yu,et al.  A Survey of Heterogeneous Information Network Analysis , 2015, IEEE Transactions on Knowledge and Data Engineering.

[51]  Liwei Qiu,et al.  Scalable Multiplex Network Embedding , 2018, IJCAI.

[52]  Jingping Bi,et al.  Cross-Network Embedding for Multi-Network Alignment , 2019, WWW.