Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation

Multi-scenario recommendation is dedicated to retrieve relevant items for users in multiple scenarios, which is ubiquitous in industrial recommendation systems. These scenarios enjoy portions of overlaps in users and items, while the distribution of different scenarios is different. The key point of multi-scenario modeling is to efficiently maximize the use of whole-scenario information and granularly generate adaptive representations both for users and items among multiple scenarios. we summarize three practical challenges which are not well solved for multi-scenario modeling: (1) Lacking of fine-grained and decoupled information transfer controls among multiple scenarios. (2) Insufficient exploitation of entire space samples. (3) Item's multi-scenario representation disentanglement problem. In this paper, we propose a Scenario-Adaptive and Self-Supervised (SASS) model to solve the three challenges mentioned above. Specifically, we design a Multi-Layer Scenario Adaptive Transfer (ML-SAT) module with scenario-adaptive gate units to select and fuse effective transfer information from whole scenario to individual scenario in a quite fine-grained and decoupled way. To sufficiently exploit the power of entire space samples, a two-stage training process including pre-training and fine-tune is introduced. The pre-training stage is based on a scenario-supervised contrastive learning task with the training samples drawn from labeled and unlabeled data spaces. The model is created symmetrically both in user side and item side, so that we can get distinguishing representations of items in different scenarios. Extensive experimental results on public and industrial datasets demonstrate the superiority of the SASS model over state-of-the-art methods. This model also achieves more than 8.0% improvement on Average Watching Time Per User in online A/B tests. SASS has been successfully deployed on multi-scenario short video recommendation platform of Taobao in Alibaba.

[1]  Jian Xu,et al.  Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling , 2022, WSDM.

[2]  Bo Zhang,et al.  Contrastive Cross-domain Recommendation in Matching , 2021, KDD.

[3]  Haikuan Huang,et al.  Self-Supervised Learning on Users' Spontaneous Behaviors for Multi-Scenario Ranking in E-commerce , 2021, CIKM.

[4]  Zulong Chen,et al.  SAR-Net: A Scenario-Aware Ranking Network for Personalized Fair Recommendation in Hundreds of Travel Scenarios , 2021, CIKM.

[5]  Ruobing Xie,et al.  Adversarial Feature Translation for Multi-domain Recommendation , 2021, KDD.

[6]  Chi Zhang,et al.  One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction , 2021, CIKM.

[7]  Xinlei Chen,et al.  Exploring Simple Siamese Representation Learning , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lanfen Lin,et al.  Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce , 2020, 2020 International Conference on Data Mining Workshops (ICDMW).

[9]  Xiangnan He,et al.  Self-supervised Graph Learning for Recommendation , 2020, SIGIR.

[10]  Lijun Zhang,et al.  Improving Multi-Scenario Learning to Rank in E-commerce by Exploiting Task Relationships in the Label Space , 2020, CIKM.

[11]  Junning Liu,et al.  Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations , 2020, RecSys.

[12]  Chang Zhou,et al.  Disentangled Self-Supervision in Sequential Recommenders , 2020, KDD.

[13]  Ji-Rong Wen,et al.  S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , 2020, CIKM.

[14]  Xiuwu Zhang,et al.  MiNet: Mixed Interest Network for Cross-Domain Click-Through Rate Prediction , 2020, CIKM.

[15]  Jieqi Kang,et al.  Self-supervised Learning for Large-scale Item Recommendations , 2020, CIKM.

[16]  Bo Zhang,et al.  Internal and Contextual Attention Network for Cold-start Multi-channel Matching in Recommendation , 2020, IJCAI.

[17]  Linjun Yang,et al.  Embedding-based Retrieval in Facebook Search , 2020, KDD.

[18]  Kang Zhang,et al.  Towards Personalized and Semantic Retrieval: An End-to-End Solution for E-commerce Search via Embedding Learning , 2020, SIGIR.

[19]  Hongxia Yang,et al.  Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems , 2020, KDD.

[20]  Doug Downey,et al.  Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks , 2020, ACL.

[21]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[22]  Pan Li,et al.  DDTCDR: Deep Dual Transfer Cross Domain Recommendation , 2019, WSDM.

[23]  Fei Sun,et al.  SDM: Sequential Deep Matching Model for Online Large-scale Recommender System , 2019, CIKM.

[24]  Choon Hui Teo,et al.  Semantic Product Search , 2019, KDD.

[25]  Wei Li,et al.  Behavior sequence transformer for e-commerce recommendation in Alibaba , 2019, Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data.

[26]  Wei Li,et al.  Multi-Interest Network with Dynamic Routing for Recommendation at Tmall , 2019, CIKM.

[27]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[28]  Zhe Zhao,et al.  Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts , 2018, KDD.

[29]  Xiao Ma,et al.  Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate , 2018, SIGIR.

[30]  Yu Zhang,et al.  CoNet: Collaborative Cross Networks for Cross-Domain Recommendation , 2018, UMCit@KDD.

[31]  Kai Liu,et al.  Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction , 2017, ArXiv.

[32]  Jeff Johnson,et al.  Billion-Scale Similarity Search with GPUs , 2017, IEEE Transactions on Big Data.

[33]  Geoffrey E. Hinton,et al.  Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer , 2017, ICLR.

[34]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Martial Hebert,et al.  Cross-Stitch Networks for Multi-task Learning , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[37]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[38]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[39]  Yu He,et al.  The YouTube video recommendation system , 2010, RecSys '10.

[40]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.