DREAM: Decoupled Representation via Extraction Attention Module and Supervised Contrastive Learning for Cross-Domain Sequential Recommender

Cross-Domain Sequential Recommendation(CDSR) aims to generate accurate predictions for future interactions by leveraging users’ cross-domain historical interactions. One major challenge of CDSR is how to jointly learn the single- and cross-domain user preferences efficiently. To enhance the target domain’s performance, most existing solutions start by learning the single-domain user preferences within each domain and then transferring the acquired knowledge from the rich domain to the target domain. However, this approach ignores the inter-sequence item relationship and also limits the opportunities for target domain knowledge to enhance the rich domain performance. Moreover, it also ignores the information within the cross-domain sequence. Despite cross-domain sequences being generally noisy and hard to learn directly, they contain valuable user behavior patterns with great potential to enhance performance. Another key challenge of CDSR is data sparsity, which also exists in other recommendation system problems. In the real world, the data distribution of the recommendation system is highly skewed to the popular products, especially on the large-scale dataset with millions of users and items. One more challenge is the class imbalance problem, inherited by the sequential recommendation problem. Generally, each sample only has one positive and thousands of negative samples. To address the above problems together, an innovative Decoupled Representation via Extraction Attention Module (DREAM) is proposed for CDSR to simultaneously learn single- and cross-domain user preference via decoupled representations. A novel Supervised Contrastive Learning framework is introduced to model the inter-sequence relationship as well as address the data sparsity via data augmentations. DREAM also leverages Focal Loss to put more weight on misclassified samples to address the class-imbalance problem, with another uplift on the overall model performance. Extensive experiments had been conducted on two cross-domain recommendation datasets, demonstrating DREAM outperforms various SOTA cross-domain recommendation algorithms achieving up to a 75% uplift in Movie-Book Scenarios.

[1]  L. Yao,et al.  Dual Contrastive Transformer for Hierarchical Preference Modeling in Sequential Recommendation , 2023, SIGIR.

[2]  L. Yao,et al.  Modeling Temporal Positive and Negative Excitation for Sequential Recommendation , 2023, WWW.

[3]  Tingwen Liu,et al.  Contrastive Cross-Domain Sequential Recommendation , 2022, CIKM.

[4]  Chin-Chia Michael Yeh,et al.  Denoising Self-Attentive Sequential Recommendation , 2022, RecSys.

[5]  Zhiwen Yu,et al.  IDNP: Interest Dynamics Modeling Using Generative Neural Processes for Sequential Recommendation , 2022, WSDM.

[6]  Xiaolin Zheng,et al.  Exploiting Variational Domain-Invariant User Embedding for Partially Overlapped Cross Domain Recommendation , 2022, SIGIR.

[7]  Jinyang Gao,et al.  Contrastive Learning for Sequential Recommendation , 2022, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[8]  Tingwen Liu,et al.  Cross-Domain Recommendation to Cold-Start Users via Variational Information Bottleneck , 2022, 2022 IEEE 38th International Conference on Data Engineering (ICDE).

[9]  Bo Zhang,et al.  Contrastive Cross-domain Recommendation in Matching , 2021, KDD.

[10]  Chenyun Yu,et al.  RecGURU: Adversarial Learning of Generalized User Representations for Cross-Domain Recommendation , 2021, WSDM.

[11]  Fuzhen Zhuang,et al.  Personalized Transfer of User Preferences for Cross-domain Recommendation , 2021, WSDM.

[12]  Zijian Wang,et al.  Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation , 2021, WSDM.

[13]  Aghiles Salah,et al.  Towards Source-Aligned Variational Models for Cross-Domain Recommendation , 2021, RecSys.

[14]  Tong Chen,et al.  Lightweight Self-Attentive Sequential Recommendation , 2021, CIKM.

[15]  Julian McAuley,et al.  Contrastive Self-supervised Sequential Recommendation with Robust Augmentation , 2021, ArXiv.

[16]  Fuzhen Zhuang,et al.  Transfer-Meta Framework for Cross-domain Recommendation to Cold-Start Users , 2021, SIGIR.

[17]  Philip S. Yu,et al.  Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer , 2021, SIGIR.

[18]  Meng Liu,et al.  Cross Domain Recommendation via Bi-directional Transfer Graph Collaborative Filtering Networks , 2020, CIKM.

[19]  Ji-Rong Wen,et al.  S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , 2020, CIKM.

[20]  Xiuwu Zhang,et al.  MiNet: Mixed Interest Network for Cross-Domain Click-Through Rate Prediction , 2020, CIKM.

[21]  Walid Krichene,et al.  On Sampled Metrics for Item Recommendation , 2020, KDD.

[22]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[23]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[24]  Pan Li,et al.  DDTCDR: Deep Dual Transfer Cross Domain Recommendation , 2019, WSDM.

[25]  M. de Rijke,et al.  Parallel Split-Join Networks for Shared Account Cross-Domain Sequential Recommendations , 2019, IEEE Transactions on Knowledge and Data Engineering.

[26]  M. de Rijke,et al.  π-Net: A Parallel Information-sharing Network for Shared-account Cross-domain Sequential Recommendations , 2019, SIGIR.

[27]  Lina Yao,et al.  DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patterns , 2019, IJCAI.

[28]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[29]  Shirui Pan,et al.  DAGCN: Dual Attention Graph Convolutional Networks , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[30]  Xing Xie,et al.  Session-based Recommendation with Graph Neural Networks , 2018, AAAI.

[31]  R. Devon Hjelm,et al.  Learning deep representations by mutual information estimation and maximization , 2018, ICLR.

[32]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[33]  Lina Yao,et al.  Next Item Recommendation with Self-Attention , 2018, ArXiv.

[34]  Yu Zhang,et al.  CoNet: Collaborative Cross Networks for Cross-Domain Recommendation , 2018, UMCit@KDD.

[35]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[36]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[38]  Balázs Hidasi,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[41]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Report , 1999, TREC.