mvn2vec: Preservation and Collaboration in Multi-View Network Embedding

Multi-view networks are ubiquitous in real-world applications. In order to extract knowledge or business value, it is of interest to transform such networks into representations that are easily machine-actionable. Meanwhile, network embedding has emerged as an effective approach to generate distributed network representations. Therefore, we are motivated to study the problem of multi-view network embedding, with a focus on the characteristics that are specific and important in embedding this type of networks. In our practice of embedding real-world multi-view networks, we identify two such characteristics, which we refer to as preservation and collaboration. We then explore the feasibility of achieving better embedding quality by simultaneously modeling preservation and collaboration, and propose the mvn2vec algorithms. With experiments on a series of synthetic datasets, an internal Snapchat dataset, and two public datasets, we further confirm the presence and importance of preservation and collaboration. These experiments also demonstrate that better embedding can be obtained by simultaneously modeling the two characteristics, while not over-complicating the model or requiring additional supervision.

[1]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[2]  Bin Wu,et al.  Representation Learning Based on Influence of Node for Multiplex Network , 2018, 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC).

[3]  Lin Liu,et al.  A Structural Representation Learning for Multi-relational Networks , 2017, IJCAI.

[4]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[5]  Jiawei Han,et al.  Large-Scale Embedding Learning in Heterogeneous Event Data , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[6]  Jianyong Wang,et al.  Coherent closed quasi-clique discovery from large dense graph databases , 2006, KDD '06.

[7]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[8]  Weiyi Liu,et al.  Principled Multilayer Network Embedding , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[9]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[10]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[11]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Steven Skiena,et al.  Don't Walk, Skip!: Online Learning of Multi-scale Network Embeddings , 2016, ASONAM.

[13]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[14]  Krzysztof Nowicki,et al.  Exploratory statistical analysis of networks , 1992 .

[15]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[16]  Derek Greene,et al.  A Matrix Factorization Approach for Integrating Multiple Data Views , 2009, ECML/PKDD.

[17]  J. Alston,et al.  Wa, Guanxi, and Inhwa: Managerial principles in Japan, China, and Korea , 1989 .

[18]  T. B. Murphy,et al.  Joint Modelling of Multiple Network Views , 2013, 1301.3759.

[19]  Christopher J. C. Burges,et al.  Spectral clustering and transductive learning with multiple views , 2007, ICML '07.

[20]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[21]  Tyler H McCormick,et al.  LATENT SPACE MODELS FOR MULTIVIEW NETWORK DATA. , 2017, The annals of applied statistics.

[22]  Wang-Chien Lee,et al.  HIN2Vec: Explore Meta-paths in Heterogeneous Information Networks for Representation Learning , 2017, CIKM.

[23]  Anna Monreale,et al.  Multidimensional networks: foundations of structural analysis , 2013, World Wide Web.

[24]  Yixin Chen,et al.  Weisfeiler-Lehman Neural Machine for Link Prediction , 2017, KDD.

[25]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[26]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[27]  Liwei Qiu,et al.  Scalable Multiplex Network Embedding , 2018, IJCAI.

[28]  Po-Wei Chan,et al.  PReP: Path-Based Relevance from a Probabilistic Perspective in Heterogeneous Information Networks , 2017, KDD.

[29]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[30]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[31]  Jiawei Han,et al.  An Attention-based Collaboration Framework for Multi-View Network Representation Learning , 2017, CIKM.

[32]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[33]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[34]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[35]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[36]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[37]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[38]  Derek Greene,et al.  Producing a unified graph representation from multiple social network views , 2013, WebSci.

[39]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[40]  Jiawei Han,et al.  Multi-View Clustering via Joint Nonnegative Matrix Factorization , 2013, SDM.

[41]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[42]  Vito Latora,et al.  Structural measures for multiplex networks. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Jie Tang,et al.  Representation Learning for Attributed Multiplex Heterogeneous Network , 2019, KDD.

[44]  Le Song,et al.  Discriminative Embeddings of Latent Variable Models for Structured Data , 2016, ICML.

[45]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[46]  Philip S. Yu,et al.  Multi-view Clustering with Graph Embedding for Connectome Analysis , 2017, CIKM.

[47]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[48]  A. Arenas,et al.  Mathematical Formulation of Multilayer Networks , 2013, 1307.4977.

[49]  Jian Pei,et al.  On mining cross-graph quasi-cliques , 2005, KDD '05.

[50]  S. Wasserman,et al.  Logit models and logistic regressions for social networks: II. Multivariate relations. , 1999, The British journal of mathematical and statistical psychology.

[51]  Jiawei Han,et al.  Mining coherent dense subgraphs across massive biological networks for functional discovery , 2005, ISMB.

[52]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[53]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[54]  Hal Daumé,et al.  Co-regularized Multi-view Spectral Clustering , 2011, NIPS.

[55]  Fei Wang,et al.  Multi-View Local Learning , 2008, AAAI.

[56]  Xiao Liu,et al.  Co-Regularized Deep Multi-Network Embedding , 2018, WWW.