Dynamically Expandable Graph Convolution for Streaming Recommendation

Personalized recommender systems have been widely studied and deployed to reduce information overload and satisfy users’ diverse needs. However, conventional recommendation models solely conduct a one-time training-test fashion and can hardly adapt to evolving demands, considering user preference shifts and ever-increasing users and items in the real world. To tackle such challenges, the streaming recommendation is proposed and has attracted great attention recently. Among these, continual graph learning is widely regarded as a promising approach for the streaming recommendation by academia and industry. However, existing methods either rely on the historical data replay which is often not practical under increasingly strict data regulations, or can seldom solve the over-stability issue. To overcome these difficulties, we propose a novel Dynamically Expandable Graph Convolution (DEGC) algorithm from a model isolation perspective for the streaming recommendation which is orthogonal to previous methods. Based on the motivation of disentangling outdated short-term preferences from useful long-term preferences, we design a sequence of operations including graph convolution pruning, refining, and expanding to only preserve beneficial long-term preference-related parameters and extract fresh short-term preferences. Moreover, we model the temporal user preference, which is utilized as user embedding initialization, for better capturing the individual-level preference shifts. Extensive experiments on the three most representative GCN-based recommendation models and four industrial datasets demonstrate the effectiveness and robustness of our method.

[1]  Guojie Song,et al.  Streaming Graph Neural Networks with Generative Replay , 2022, KDD.

[2]  Irwin King,et al.  Learning Binarized Graph Representations with Multi-faceted Quantization Reinforcement for Top-K Recommendation , 2022, KDD.

[3]  Paris Carbone,et al.  Learning on streaming graphs with experience replay , 2022, SAC.

[4]  Xin Wang,et al.  Multimodal Continual Graph Learning with Neural Architecture Search , 2022, WWW.

[5]  Jieming Zhu,et al.  ReLoop: A Self-Correction Continual Learning Loop for Recommender Systems , 2022, SIGIR.

[6]  Massimo Caccia,et al.  Continual Learning via Local Module Composition , 2021, NeurIPS.

[7]  Mark Coates,et al.  Graph Structure Aware Contrastive Knowledge Distillation for Incremental Learning in Recommender Systems , 2021, CIKM.

[8]  Mark Coates,et al.  Structure Aware Experience Replay for Incremental Learning in Graph-based Recommender Systems , 2021, CIKM.

[9]  Irwin King,et al.  Attentive Knowledge-aware Graph Convolutional Networks with Collaborative Guidance for Personalized Recommendation , 2021, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[10]  Irwin King,et al.  Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation , 2021, WSDM.

[11]  Jing Gao,et al.  Data Poisoning Attack against Recommender System Using Incomplete and Perturbed Data , 2021, Knowledge Discovery and Data Mining.

[12]  Myle Ott,et al.  On Anytime Learning at Macroscale , 2021, CoLLAs.

[13]  Fan Zhou,et al.  Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay , 2021, AAAI.

[14]  Yikun Ban,et al.  Local Clustering in Contextual Multi-Armed Bandits , 2021, WWW.

[15]  Bo Liu,et al.  Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks , 2021, NeurIPS.

[16]  Yiding Yang,et al.  Overcoming Catastrophic Forgetting in Graph Neural Networks , 2020, AAAI.

[17]  Alexandros Karatzoglou,et al.  One Person, One Model, One World: Learning Continual User Representation without Forgetting , 2020, SIGIR.

[18]  Yi Wu,et al.  Streaming Graph Neural Networks via Continual Learning , 2020, CIKM.

[19]  Wei Guo,et al.  GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems , 2020, CIKM.

[20]  Philip H. S. Torr,et al.  GDumb: A Simple Approach that Questions Our Progress in Continual Learning , 2020, ECCV.

[21]  Boi Faltings,et al.  ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation , 2020, RecSys.

[22]  Xue Liu,et al.  Probabilistic Metric Learning with Adaptive Margin for Top-K Recommendation , 2020, KDD.

[23]  Jie Yang,et al.  LBSN2Vec++: Heterogeneous Hypergraph Embedding for Location-Based Social Networks , 2020, IEEE Transactions on Knowledge and Data Engineering.

[24]  Xiangnan He,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[25]  Ed H. Chi,et al.  From Missing Data to Boltzmann Distributions and Time Dynamics: The Statistical Physics of Recommendation , 2020, WSDM.

[26]  Mark Coates,et al.  Multi-graph Convolution Collaborative Filtering , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[27]  Jure Leskovec,et al.  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks , 2019, KDD.

[28]  Chen Ma,et al.  Hierarchical Gating Networks for Sequential Recommendation , 2019, KDD.

[29]  Dahua Lin,et al.  Learning a Unified Classifier Incrementally via Rebalancing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Rui Zhang,et al.  Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random , 2019, ICML.

[31]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[32]  Philippe Cudré-Mauroux,et al.  Revisiting User Mobility and Social Relationships in LBSNs: A Hypergraph Embedding Approach , 2019, WWW.

[33]  Kyunghyun Cho,et al.  Continual Learning via Neural Pruning , 2019, ArXiv.

[34]  Marc'Aurelio Ranzato,et al.  Continual Learning with Tiny Episodic Memories , 2019, ArXiv.

[35]  Charles E. Leisersen,et al.  EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs , 2019, AAAI.

[36]  Shuai Li,et al.  Improved Algorithm on Online Clustering of Bandits , 2019, IJCAI.

[37]  Rama Chellappa,et al.  Learning Without Memorizing , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jiliang Tang,et al.  Streaming Graph Neural Networks , 2018, SIGIR.

[39]  Zi Huang,et al.  Streaming Ranking Based Recommender Systems , 2018, SIGIR.

[40]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[41]  Zhanxing Zhu,et al.  Reinforced Continual Learning , 2018, NeurIPS.

[42]  David Isele,et al.  Selective Experience Replay for Lifelong Learning , 2018, AAAI.

[43]  Jia Li,et al.  Latent Cross: Making Use of Context in Recurrent Recommender Systems , 2018, WSDM.

[44]  Sung Ju Hwang,et al.  Lifelong Learning with Dynamically Expandable Networks , 2017, ICLR.

[45]  James Caverlee,et al.  Multi-Aspect Streaming Tensor Completion , 2017, KDD.

[46]  Matthew B. Blaschko,et al.  Encoder Based Lifelong Learning , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Christoph H. Lampert,et al.  iCaRL: Incremental Classifier and Representation Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Charu C. Aggarwal,et al.  Recommendations For Streaming Data , 2016, CIKM.

[49]  Yiran Chen,et al.  Learning Structured Sparsity in Deep Neural Networks , 2016, NIPS.

[50]  Shuai Li,et al.  On Context-Dependent Clustering of Bandits , 2016, ICML.

[51]  Yi Chang,et al.  Streaming Recommender Systems , 2016, WWW.

[52]  Nicolas Kourtellis,et al.  Dynamic Matrix Factorization with Priors on Unknown Values , 2015, KDD.

[53]  Sahin Albayrak,et al.  Real-time recommendations for user-item streams , 2015, SAC.

[54]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[55]  Shuai Li,et al.  Collaborative Filtering Bandits , 2015, SIGIR.

[56]  Shuai Li,et al.  Online Clustering of Bandits , 2014, ICML.

[57]  Junjie Yao,et al.  TeRec: A Temporal Recommender System Over Tweet Stream , 2013, Proc. VLDB Endow..

[58]  Lars Schmidt-Thieme,et al.  Real-time top-n recommendation in social streams , 2012, RecSys.

[59]  Badrish Chandramouli,et al.  StreamRec: a real-time recommender system , 2011, SIGMOD '11.

[60]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[61]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[62]  Lars Schmidt-Thieme,et al.  Online-updating regularized kernel matrix factorization models for large-scale recommender systems , 2008, RecSys '08.

[63]  Yang Song,et al.  Real-time automatic tag recommendation , 2008, SIGIR '08.

[64]  Abhinandan Das,et al.  Google news personalization: scalable online collaborative filtering , 2007, WWW '07.

[65]  Haidong Rong,et al.  Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model Update , 2022, OSDI.

[66]  Bing Liu,et al.  BNS: Building Network Structures Dynamically for Continual Learning , 2021, NeurIPS.

[67]  J. L. Roux An Introduction to the Kalman Filter , 2003 .