Equivariant Contrastive Learning for Sequential Recommendation

Contrastive learning (CL) benefits the training of sequential recommendation models with informative self-supervision signals. Existing solutions apply general sequential data augmentation strategies to generate positive pairs and encourage their representations to be invariant. However, due to the inherent properties of user behavior sequences, some augmentation strategies, such as item substitution, can lead to changes in user intent. Learning indiscriminately invariant representations for all augmentation strategies might be sub-optimal. Therefore, we propose Equivariant Contrastive Learning for Sequential Recommendation (ECL-SR), which endows SR models with great discriminative power, making the learned user behavior representations sensitive to invasive augmentations (e.g., item substitution) and insensitive to mild augmentations (e.g., feature-level dropout masking). In detail, we use the conditional discriminator to capture differences in behavior due to item substitution, which encourages the user behavior encoder to be equivariant to invasive augmentations. Comprehensive experiments on four benchmark datasets show that the proposed ECL-SR framework achieves competitive performance compared to state-of-the-art SR models. The source code will be released.

[1]  Haohan Wang,et al.  Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network , 2022, WSDM.

[2]  Ee-Peng Lim,et al.  Explanation Guided Contrastive Learning for Sequential Recommendation , 2022, CIKM.

[3]  Yuexian Zou,et al.  Leveraging Bilinear Attention to Improve Spoken Language Understanding , 2022, ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Jinyang Gao,et al.  Contrastive Learning for Sequential Recommendation , 2022, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[5]  Peilin Zhou,et al.  Decoupled Side Information Fusion for Sequential Recommendation , 2022, SIGIR.

[6]  James R. Glass,et al.  DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings , 2022, NAACL.

[7]  Caiming Xiong,et al.  ELECRec: Training Sequential Recommenders as Discriminators , 2022, SIGIR.

[8]  Tong Chen,et al.  Self-Supervised Learning for Recommender Systems: A Survey , 2022, ArXiv.

[9]  Julian McAuley,et al.  Intent Contrastive Learning for Sequential Recommendation , 2022, WWW.

[10]  Zijian Wang,et al.  Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation , 2021, WSDM.

[11]  J. Duncan,et al.  Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation , 2021, MICCAI.

[12]  Milind Rao,et al.  MTL-SLT: Multi-Task Learning for Spoken Language Tasks , 2022, NLP4CONVAI.

[13]  Ajinkya Kale,et al.  Locker: Locally Constrained Self-Attentive Sequential Recommendation , 2021, CIKM.

[14]  Tong Chen,et al.  Lightweight Self-Attentive Sequential Recommendation , 2021, CIKM.

[15]  Hongxu Chen,et al.  Hyperbolic Hypergraphs for Sequential Recommendation , 2021, CIKM.

[16]  Julian McAuley,et al.  Contrastive Self-supervised Sequential Recommendation with Robust Augmentation , 2021, ArXiv.

[17]  Andreas Hotho,et al.  A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models , 2021, RecSys.

[18]  Lei Shi,et al.  ICAI-SR: Item Categorical Attribute Integrated Sequential Recommendation , 2021, SIGIR.

[19]  Shen Ge,et al.  Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model , 2021, ArXiv.

[20]  Chen Gao,et al.  Sequential Recommendation with Graph Neural Networks , 2021, SIGIR.

[21]  Yuexian Zou,et al.  Sentiment Injected Iteratively Co-Interactive Network for Spoken Language Understanding , 2021, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Wayne Xin Zhao,et al.  Dual Sparse Attention Network For Session-based Recommendation , 2021, AAAI.

[23]  Yuexian Zou,et al.  Audio-Oriented Multimodal Machine Comprehension via Dynamic Inter- and Intra-modality Attention , 2021, AAAI.

[24]  Danqi Chen,et al.  SimCSE: Simple Contrastive Learning of Sentence Embeddings , 2021, EMNLP.

[25]  Feng Wang,et al.  Understanding the Behaviour of Contrastive Loss , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Ji-Rong Wen,et al.  RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms , 2020, CIKM.

[27]  Neil Zeghidour,et al.  Contrastive Learning of General-Purpose Audio Representations , 2020, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Yuexian Zou,et al.  PIN: A Novel Parallel Interactive Network for Spoken Language Understanding , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[29]  Yuexian Zou,et al.  Federated Learning for Spoken Language Understanding , 2020, COLING.

[30]  Cho-Jui Hsieh,et al.  SSE-PT: Sequential Recommendation Via Personalized Transformer , 2020, RecSys.

[31]  Ji-Rong Wen,et al.  S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , 2020, CIKM.

[32]  Walid Krichene,et al.  On Sampled Metrics for Item Recommendation , 2020, KDD.

[33]  Jaesik Park,et al.  ContraGAN: Contrastive Learning for Conditional Image Generation , 2020, Neural Information Processing Systems.

[34]  Xin Jiang,et al.  DynaBERT: Dynamic BERT with Adaptive Width and Depth , 2020, NeurIPS.

[35]  Quoc V. Le,et al.  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators , 2020, ICLR.

[36]  Yujie Wang,et al.  Time Interval Aware Self-Attention for Sequential Recommendation , 2020, WSDM.

[37]  Mark Coates,et al.  Memory Augmented Graph Neural Networks for Sequential Recommendation , 2019, AAAI.

[38]  Priyanka Gupta,et al.  NISER: Normalized Item and Session Representations to Handle Popularity Bias. , 2019 .

[39]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[40]  Joemon M. Jose,et al.  A Simple Convolutional Generative Network for Next Item Recommendation , 2018, WSDM.

[41]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[42]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[43]  Edward Y. Chang,et al.  Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks , 2018, SIGIR.

[44]  Yang Sok Kim,et al.  Recommendation system development for fashion retail e-commerce , 2018, Electron. Commer. Res. Appl..

[45]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[46]  Yongfeng Zhang,et al.  Sequential Recommendation with User Memory Networks , 2018, WSDM.

[47]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[48]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[49]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[50]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[51]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[52]  Philip S. Yu,et al.  Effective Next-Items Recommendation via Personalized Sequential Pattern Mining , 2012, DASFAA.

[53]  Yu He,et al.  The YouTube video recommendation system , 2010, RecSys '10.

[54]  David Carmel,et al.  Social media recommendation based on people and tags , 2010, SIGIR.

[55]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[56]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.