CoPE: Modeling Continuous Propagation and Evolution on Interaction Graph

Human interactions with items are being constantly logged, which enables advanced representation learning and facilitates various tasks. Instead of generating static embeddings at the end of training, several temporal embedding methods were recently proposed to learn user and item embeddings as functions of time, where each entity has a trajectory of embedding vectors aiming to encode the full dynamics. However, these methods may not be optimal to encode the dynamical behaviors on the interaction graphs in that they can not generate "fully''-temporal embeddings and do not consider information propagation. In this paper, we tackle the issues and propose CoPE (Co ntinuous P ropagation and E volution). We use an ordinary differential equation based graph neural network to model information propagation and more sophisticated evolution patterns. We train CoPE on sequences of interactions with the help of meta-learning to ensure fast adaptation to the most recent interactions. We evaluate CoPE on three tasks and prove its effectiveness.

[1]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[2]  Jure Leskovec,et al.  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks , 2019, KDD.

[3]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[4]  Zoran Obradovic,et al.  Time-Aware User Embeddings as a Service , 2020, KDD.

[5]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[6]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[7]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[8]  Martha White,et al.  Meta-Learning Representations for Continual Learning , 2019, NeurIPS.

[9]  Le Song,et al.  Deep Coevolutionary Network: Embedding User and Item Features for Recommendation , 2016, 1609.03675.

[10]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[11]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[12]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[13]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[14]  Edward De Brouwer,et al.  GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series , 2019, NeurIPS.

[15]  David Duvenaud,et al.  Latent Ordinary Differential Equations for Irregularly-Sampled Time Series , 2019, NeurIPS.

[16]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[17]  Philip S. Yu,et al.  Alleviating the Inconsistency Problem of Applying Graph Neural Network to Fraud Detection , 2020, SIGIR.

[18]  Deng Cai,et al.  What to Do Next: Modeling User Behaviors by Time-LSTM , 2017, IJCAI.

[19]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[20]  Yao Zhang,et al.  Learning Node Embeddings in Interaction Graphs , 2017, CIKM.

[21]  Yongdong Zhang,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[22]  Alex Beutel,et al.  Recurrent Recommender Networks , 2017, WSDM.

[23]  Jian Tang,et al.  Continuous Graph Neural Networks , 2019, ICML.

[24]  Jia Li,et al.  Latent Cross: Making Use of Context in Recurrent Recommender Systems , 2018, WSDM.

[25]  Philip S. Yu,et al.  Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer , 2021, SIGIR.

[26]  Jiawei Zhang,et al.  Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer , 2021, CIKM.

[27]  Nima Noorshams,et al.  TIES: Temporal Interaction Embeddings for Enhancing Social Media Integrity at Facebook , 2020, KDD.

[28]  Philip S. Yu,et al.  BasConv: Aggregating Heterogeneous Interactions for Basket Recommendation with Graph Convolutional Neural Network , 2020, SDM.

[29]  Zhen Zhang,et al.  Learning Temporal Interaction Graph Embedding via Coupled Memory Networks , 2020, WWW.

[30]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[31]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[32]  Xueqi Cheng,et al.  Graph Wavelet Neural Network , 2019, ICLR.

[33]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[34]  Yee Whye Teh,et al.  Augmented Neural ODEs , 2019, NeurIPS.

[35]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[36]  Fei Wang,et al.  Patient Subtyping via Time-Aware LSTM Networks , 2017, KDD.

[37]  Jen-Hung Huang,et al.  Herding in online product choice , 2006 .

[38]  Jiawei Han,et al.  A Data-Driven Graph Generative Model for Temporal Interaction Networks , 2020, KDD.

[39]  Yangyong Zhu,et al.  IGE+: A Framework for Learning Node Embeddings in Interaction Graphs , 2019, IEEE Transactions on Knowledge and Data Engineering.