Time-Capturing Dynamic Graph Embedding for Temporal Linkage Evolution

Dynamic graph embedding learns representation vectors for vertices and edges in a graph that evolves over time. We aim to capture and embed the evolution of vertices' temporal connectivity. Existing work studies the vertices' dynamic connection changes but neglects the time it takes for edges to evolve, failing to embed temporal linkage information into the evolution of the graph. To capture vertices' temporal linkage evolution, we model dynamic graphs as a sequence of snapshot graphs, appending the respective timespans of edges (ToE). We co-train a linear regressor to embed ToE while inferring a common latent space for all snapshot graphs by a matrix-factorization-based model to embed vertices' dynamic connection changes. Vertices' temporal linkage evolution is captured as their moving trajectories within the common latent representation space. Our embedding algorithm converges quickly with our proposed training methods, which is very time efficient and scalable. Extensive evaluations on several datasets show that our model can achieve significant performance improvements, i.e. 22.98% on average across all datasets, over the state-of-the-art baselines in the tasks of vertex classification, static and time-aware link prediction, and ToE prediction.

[1]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[2]  Jiannong Cao,et al.  CANE: community-aware network embedding via adversarial training , 2020, Knowledge and Information Systems.

[3]  Bo Liu,et al.  Heterogeneous Hypergraph Embedding for Graph Classification , 2020, WSDM.

[4]  Lin Liu,et al.  Dynamic network embedding via incremental skip-gram with negative sampling , 2019, Science China Information Sciences.

[5]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[6]  Jian Pei,et al.  TIMERS: Error-Bounded SVD Restart on Dynamic Networks , 2017, AAAI.

[7]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[8]  Yilun Jin,et al.  EPNE: Evolutionary Pattern Preserving Network Embedding , 2020, ECAI.

[9]  Sander Greenland,et al.  Sparse data bias: a problem hiding in plain sight , 2016, British Medical Journal.

[10]  Palash Goyal,et al.  Capturing Edge Attributes via Network Embedding , 2018, IEEE Transactions on Computational Social Systems.

[11]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[12]  Tore Opsahl,et al.  Clustering in weighted networks , 2009, Soc. Networks.

[13]  Deli Zhao,et al.  Network Representation Learning with Rich Text Information , 2015, IJCAI.

[14]  Philip S. Yu,et al.  On Exploring Semantic Meanings of Links for Embedding Social Networks , 2018, WWW.

[15]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[16]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[17]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[18]  Xiaoming Zhang,et al.  From Properties to Links: Deep Network Embedding on Incomplete Graphs , 2017, CIKM.

[19]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[20]  Christos Faloutsos,et al.  Edge Weight Prediction in Weighted Signed Networks , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[21]  Charu C. Aggarwal,et al.  Temporally Factorized Network Modeling for Evolutionary Network Analysis , 2017, WSDM.

[22]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[23]  Chengqi Zhang,et al.  User Profile Preserving Social Network Embedding , 2017, IJCAI.

[24]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[25]  Palash Goyal,et al.  dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning , 2018, Knowl. Based Syst..

[26]  Chengqi Zhang,et al.  Collective Classification via Discriminative Matrix Factorization on Sparsely Labeled Networks , 2016, CIKM.

[27]  Wenwu Zhu,et al.  DepthLGP: Learning Embeddings of Out-of-Sample Nodes in Dynamic Networks , 2018, AAAI.

[28]  Jian Pei,et al.  High-Order Proximity Preserved Embedding for Dynamic Networks , 2018, IEEE Transactions on Knowledge and Data Engineering.

[29]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[30]  Philip S. Yu,et al.  DynGraphGAN: Dynamic Graph Embedding via Generative Adversarial Networks , 2019, DASFAA.

[31]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[32]  Wei Zhang,et al.  Dynamic Graph Representation Learning via Self-Attention Networks , 2018, ArXiv.

[33]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[34]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[35]  Wen-Chih Peng,et al.  Exploiting Centrality Information with Graph Convolutions for Network Representation Learning , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[36]  Philip S. Yu,et al.  Embedding of Embedding (EOE): Joint Embedding for Coupled Heterogeneous Networks , 2017, WSDM.

[37]  Bo Zhang,et al.  Discriminative Deep Random Walk for Network Classification , 2016, ACL.