TigeCMN: On exploration of temporal interaction graph embedding via Coupled Memory Neural Networks

With the increasing demand of mining rich knowledge in graph structured data, graph embedding has become one of the most popular research topics in both academic and industrial communities due to its powerful capability in learning effective representations. The majority of existing work overwhelmingly learn node embeddings in the context of static, plain or attributed, homogeneous graphs. However, many real-world applications frequently involve bipartite graphs with temporal and attributed interaction edges, named temporal interaction graphs. The temporal interactions usually imply different facets of interest and might even evolve over the time, thus putting forward huge challenges in learning effective node representations. Furthermore, most existing graph embedding models try to embed all the information of each node into a single vector representation, which is insufficient to characterize the node's multifaceted properties. In this paper, we propose a novel framework named TigeCMN to learn node representations from a sequence of temporal interactions. Specifically, we devise two coupled memory networks to store and update node embeddings in the external matrices explicitly and dynamically, which forms deep matrix representations and thus could enhance the expressiveness of the node embeddings. Then, we generate node embedding from two parts: a static embedding that encodes its stationary properties and a dynamic embedding induced from memory matrix that models its temporal interaction patterns. We conduct extensive experiments on various real-world datasets covering the tasks of node classification, recommendation and visualization. The experimental results empirically demonstrate that TigeCMN can achieve significant gains compared with recent state-of-the-art baselines.

[1]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[2]  Ming Gao,et al.  BiNE: Bipartite Network Embedding , 2018, SIGIR.

[3]  Nitesh V. Chawla,et al.  metapath2vec: Scalable Representation Learning for Heterogeneous Networks , 2017, KDD.

[4]  Nitesh V. Chawla,et al.  SHNE: Representation Learning for Semantic-Associated Heterogeneous Networks , 2019, WSDM.

[5]  Graham Cormode,et al.  Node Classification in Social Networks , 2011, Social Network Data Analytics.

[6]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[7]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[8]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[9]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[10]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[11]  Yongfeng Zhang,et al.  Sequential Recommendation with User Memory Networks , 2018, WSDM.

[12]  Zhao Li,et al.  MV-GCN: Multi-View Graph Convolutional Networks for Link Prediction , 2019, IEEE Access.

[13]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[16]  Jure Leskovec,et al.  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks , 2019, KDD.

[17]  Dit-Yan Yeung,et al.  Dynamic Key-Value Memory Networks for Knowledge Tracing , 2016, WWW.

[18]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[19]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[20]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[21]  Mark Heimann,et al.  node2bits: Compact Time- and Attribute-aware Node Representations for User Stitching , 2019, ECML/PKDD.

[22]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[23]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[24]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[25]  Guy Shani,et al.  A Survey of Accuracy Evaluation Metrics of Recommendation Tasks , 2009, J. Mach. Learn. Res..

[26]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[27]  Yao Zhang,et al.  Learning Node Embeddings in Interaction Graphs , 2017, CIKM.

[28]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[29]  Zi Huang,et al.  Neural Memory Streaming Recommender Networks with Adversarial Training , 2018, KDD.

[30]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[31]  Hong Cheng,et al.  Graph Clustering Based on Structural/Attribute Similarities , 2009, Proc. VLDB Endow..

[32]  Linyuan Lu,et al.  Link Prediction in Complex Networks: A Survey , 2010, ArXiv.

[33]  Chengqi Zhang,et al.  Tri-Party Deep Network Representation , 2016, IJCAI.

[34]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[35]  Doug Beeferman,et al.  Agglomerative clustering of a search engine query log , 2000, KDD '00.

[36]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[37]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[38]  Jimeng Sun,et al.  Temporal recommendation on graphs via long- and short-term preference fusion , 2010, KDD.

[39]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[40]  Phil Blunsom,et al.  Learning to Transduce with Unbounded Memory , 2015, NIPS.

[41]  Bin Shen,et al.  Collaborative Memory Network for Recommendation Systems , 2018, SIGIR.

[42]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[43]  Zhao Li,et al.  Hierarchical Bipartite Graph Neural Networks: Towards Large-Scale E-commerce Applications , 2020, 2020 IEEE 36th International Conference on Data Engineering (ICDE).