Dynamic Graph Neural Networks for Sequential Recommendation

Modeling users' preference from his historical sequences is one of the core problem of sequential recommendation. Existing methods in such fields are widely distributed from conventional methods to deep learning methods. However, most of them only model users' interests within their own sequences and ignore the fine-grained utilization of dynamic collaborative signals among different user sequences, making them insufficient to explore users' preferences. We take inspiration from dynamic graph neural networks to cope with this challenge, unifying the user sequence modeling and dynamic interaction information among users into one framework. We propose a new method named \emph{Dynamic Graph Neural Network for Sequential Recommendation} (DGSR), which connects the sequence of different users through a dynamic graph structure, exploring the interactive behavior of users and items with time and order information. Furthermore, we design a Dynamic Graph Attention Neural Network to achieve the information propagation and aggregation among different users and their sequences in the dynamic graph. Consequently, the next-item prediction task in sequential recommendation is converted into a link prediction task for the user node to the item node in a dynamic graph. Extensive experiments on four public benchmarks show that DGSR outperforms several state-of-the-art methods. Further studies demonstrate the rationality and effectiveness of modeling user sequences through a dynamic graph.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[2]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[3]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[4]  Jure Leskovec,et al.  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks , 2019, KDD.

[5]  Hongxia Yang,et al.  Temporal Heterogeneous Interaction Graph Embedding for Next-Item Recommendation , 2020, ECML/PKDD.

[6]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[7]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[8]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[9]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[10]  James Caverlee,et al.  Next-item Recommendation with Sequential Hypergraphs , 2020, SIGIR.

[11]  Feng Yu,et al.  A Dynamic Recurrent Model for Next Basket Recommendation , 2016, SIGIR.

[12]  Shuiwang Ji,et al.  Graph U-Nets , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jian Tang,et al.  Session-Based Social Recommendation via Dynamic Graph Attention Networks , 2019, WSDM.

[14]  Yujie Wang,et al.  Time Interval Aware Self-Attention for Sequential Recommendation , 2020, WSDM.

[15]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[16]  M. de Rijke,et al.  A Collaborative Session-based Recommendation Approach with Parallel Memory Modules , 2019, SIGIR.

[17]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[18]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Da Xu,et al.  Inductive Representation Learning on Temporal Graphs , 2020, ICLR.

[21]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[22]  Chen Ma,et al.  Hierarchical Gating Networks for Sequential Recommendation , 2019, KDD.

[23]  Liang Gou,et al.  DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks , 2020, WSDM.

[24]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[25]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[26]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[27]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[28]  Ashish Vaswani,et al.  Self-Attention with Relative Position Representations , 2018, NAACL.

[29]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[30]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[31]  Hongyuan Zha,et al.  DyRep: Learning Representations over Dynamic Graphs , 2019, ICLR.

[32]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[33]  Liang Wang,et al.  Personalizing Graph Neural Networks with Attention Mechanism for Session-based Recommendation , 2019, ArXiv.

[34]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[35]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[36]  Zi Huang,et al.  Rethinking the Item Order in Session-based Recommendation with Graph Neural Networks , 2019, CIKM.

[37]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[38]  Qiao Liu,et al.  STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation , 2018, KDD.

[39]  Xing Xie,et al.  Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation , 2019, IJCAI.

[40]  Pascal Poupart,et al.  Representation Learning for Dynamic Graphs: A Survey , 2020, J. Mach. Learn. Res..

[41]  Xiao Wang,et al.  Motif-Preserving Temporal Network Embedding , 2020, IJCAI.

[42]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[43]  Mark Coates,et al.  Memory Augmented Graph Neural Networks for Sequential Recommendation , 2019, AAAI.

[44]  Xing Xie,et al.  Session-based Recommendation with Graph Neural Networks , 2018, AAAI.

[45]  Alex Smola,et al.  Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs , 2019, ArXiv.

[46]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[47]  Yanchi Liu,et al.  Graph Contextualized Self-Attention Network for Session-based Recommendation , 2019, IJCAI.

[48]  Philip S. Yu,et al.  Dynamic Graph Collaborative Filtering , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[49]  Julian J. McAuley,et al.  Translation-based Recommendation , 2017, RecSys.

[50]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[51]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[52]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.