Deep Mutual Encode Model for Network Embedding From Structural Identity

Network Embedding (NE) is one of the most popular learning methods in complex networks. It aims at learning the low-dimensional representations of nodes in networks and has been applied in a variety of network analytic tasks. Most existing methods of NE are designed by merely using the local, high-order or global proximity to preserve the network structure; hence they are incapable of fully capturing the structural identity of nodes, which is a concept of symmetry defined by the network structure and their relationship to other nodes. There are two limitations to existing NE models. First, the local and global node dependency information is not considered simultaneously. Second, there is no adequate framework that can reveal the role property of each node. In this paper, we propose an intuitive and unified deep learning framework named DMER, short for Deep Mutual Encode for Embedding, to learn node embeddings from structural identity. In our model, Graph Convolution Network (GCN) is adopted to model the dependency relations between nodes from a global perspective. An Auto-Encoder (AE) framework is proposed to reconstruct the features of nodes, and it can conclusively reveal the structural identity from network structure. By integrating the GCN and AE components with a shared and constrained mechanism, the proposed model implements mutual enhancement for node embedding from structural identity. Experimental results based on structural role classification and visualization demonstrate that our model achieves better performance compared with the state-of-the-art methods.

[1]  J. Kruskal Nonmetric multidimensional scaling: A numerical method , 1964 .

[2]  Jiming Liu,et al.  Unifying Structural Proximity and Equivalence for Network Embedding , 2019, IEEE Access.

[3]  Ryan A. Rossi,et al.  role2vec: Role-based Network Embeddings , 2019 .

[4]  Philip S. Yu,et al.  Joint Community and Structural Hole Spanner Detection via Harmonic Modularity , 2016, KDD.

[5]  Narciso Pizarro,et al.  Structural Identity and Equivalence of Individuals in Social Networks , 2007 .

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[8]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[9]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[10]  Ryan A. Rossi,et al.  Role Discovery in Networks , 2014, IEEE Transactions on Knowledge and Data Engineering.

[11]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[12]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[13]  P. Arabie,et al.  An algorithm for clustering relational data with applications to social network analysis and comparison with multidimensional scaling , 1975 .

[14]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[15]  Martin G. Everett,et al.  Two algorithms for computing regular equivalence , 1993 .

[16]  George C. Verghese,et al.  Graph similarity scoring and matching , 2008, Appl. Math. Lett..

[17]  T. Snijders,et al.  Estimation and Prediction for Stochastic Blockstructures , 2001 .

[18]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[19]  Philip S. Yu,et al.  Deep Recursive Network Embedding with Regular Equivalence , 2018, KDD.

[20]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[21]  Chengqi Zhang,et al.  Collective Classification via Discriminative Matrix Factorization on Sparsely Labeled Networks , 2016, CIKM.

[22]  Yizhou Sun,et al.  Integrating Community and Role Detection in Information Networks , 2016, SDM.

[23]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[24]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[25]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[26]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[27]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[28]  Bo Zhang,et al.  Discriminative Deep Random Walk for Network Classification , 2016, ACL.

[29]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[30]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[31]  BARSHA MITRA,et al.  A Survey of Role Mining , 2016, ACM Comput. Surv..

[32]  Ryan A. Rossi,et al.  Learning Role-based Graph Embeddings , 2018, ArXiv.

[33]  Roded Sharan,et al.  Comparative analysis of protein networks , 2012, Commun. ACM.

[34]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[35]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[36]  Edoardo M. Airoldi,et al.  Mixed Membership Stochastic Blockmodels , 2007, NIPS.

[37]  M. Newman,et al.  Vertex similarity in networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[38]  Philip S. Yu,et al.  Deep Dynamic Network Embedding for Link Prediction , 2018, IEEE Access.

[39]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[40]  L. Sailer Structural equivalence: Meaning and definition, computation and application , 1978 .

[41]  R. Burt,et al.  Applied Network Analysis: A Methodological Introduction , 1983 .

[42]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[43]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.