Deep Multi-Representational Item Network for CTR Prediction

Click-through rate (CTR) prediction is essential in the modelling of a recommender system. Previous studies mainly focus on user behavior modelling, while few of them consider candidate item representations. This makes the models strongly dependent on user representations, and less effective when user behavior is sparse. Furthermore, most existing works regard the candidate item as one fixed embedding and ignore the multi-representational characteristics of the item. To handle the above issues, we propose a Deep multi-Representational Item NetworK (DRINK) for CTR prediction. Specifically, to tackle the sparse user behavior problem, we construct a sequence of interacting users and timestamps to represent the candidate item; to dynamically capture the characteristics of the item, we propose a transformer-based multi-representational item network consisting of a multi-CLS representation submodule and contextualized global item representation submodule. In addition, we propose to decouple the time information and item behavior to avoid information overwhelming. Outputs of the above components are concatenated and fed into a MLP layer to fit the CTR. We conduct extensive experiments on real-world datasets of Amazon and the results demonstrate the effectiveness of the proposed model.

[1]  Hui Wu,et al.  Learning Token-Based Representation for Image Retrieval , 2021, AAAI.

[2]  Wei Jiang,et al.  Dynamic Sequential Recommendation: Decoupling User Intent from Temporal Context , 2021, 2021 International Conference on Data Mining Workshops (ICDMW).

[3]  Hui Wu,et al.  Contextual Similarity Aggregation with Self-attention for Visual Re-ranking , 2021, NeurIPS.

[4]  Zai Huang,et al.  Deep User Match Network for Click-Through Rate Prediction , 2021, SIGIR.

[5]  Hongliang Fei,et al.  GemNN: Gating-enhanced Multi-task Neural Networks with Feature Interaction Learning for CTR Prediction , 2021, SIGIR.

[6]  Qingtao Tang,et al.  Deep Position-wise Interaction Network for CTR Prediction , 2021, SIGIR.

[7]  Jian Xu,et al.  Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction , 2021, SIGIR.

[8]  Mikhail Khalilov,et al.  Performance analysis of CUDA, OpenACC and OpenMP programming models on TESLA V100 GPU , 2021 .

[9]  Wen Jiang,et al.  Deep Multi-Interest Network for Click-through Rate Prediction , 2020, CIKM.

[10]  Chao Wang,et al.  Deep Time-Aware Item Evolution Network for Click-Through Rate Prediction , 2020, CIKM.

[11]  Xiaohui Zhao,et al.  A Hierarchical Attention Model for CTR Prediction Based on User Interest , 2020, IEEE Systems Journal.

[12]  Chengfu Huo,et al.  Deep Match to Rank Model for Personalized Click-Through Rate Prediction , 2020, AAAI.

[13]  Yujie Wang,et al.  Time Interval Aware Self-Attention for Sequential Recommendation , 2020, WSDM.

[14]  Yuanjie Zheng,et al.  A CTR prediction model based on user interest via attention mechanism , 2020, Applied Intelligence.

[15]  Wei Li,et al.  Multi-Interest Network with Dynamic Routing for Recommendation at Tmall , 2019, CIKM.

[16]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[17]  Chang Zhou,et al.  Deep Interest Evolution Network for Click-Through Rate Prediction , 2018, AAAI.

[18]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[19]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[20]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[21]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[22]  Weinan Zhang,et al.  Product-Based Neural Networks for User Response Prediction , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[23]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[24]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[25]  Balázs Hidasi,et al.  Recurrent neural networks , 2013, Scholarpedia.

[26]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[27]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[28]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[29]  Alex Graves.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[30]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[31]  Wei Jiang,et al.  Dynamic Popularity-Aware Contrastive Learning for Recommendation , 2021, ACML.

[32]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.