Multiplex Behavioral Relation Learning for Recommendation via Memory Augmented Transformer Network

Capturing users' precise preferences is of great importance in various recommender systems (e.g., e-commerce platforms and online advertising sites), which is the basis of how to present personalized interesting product lists to individual users. In spite of significant progress has been made to consider relations between users and items, most of existing recommendation techniques solely focus on singular type of user-item interactions. However, user-item interactive behavior is often exhibited with multi-type (e.g., page view, add-to-favorite and purchase) and inter-dependent in nature. The overlook of multiplex behavior relations can hardly recognize the multi-modal contextual signals across different types of interactions, which limit the feasibility of current recommendation methods. To tackle the above challenge, this work proposes a Memory-Augmented Transformer Networks (MATN), to enable the recommendation with multiplex behavioral relational information, and joint modeling of type-specific behavioral context and type-wise behavior inter-dependencies, in a fully automatic manner. In our MATN framework, we first develop a transformer-based multi-behavior relation encoder, to make the learned interaction representations be reflective of the cross-type behavior relations. Furthermore, a memory attention network is proposed to supercharge MATN capturing the contextual signals of different types of behavior into the category-specific latent embedding space. Finally, a cross-behavior aggregation component is introduced to promote the comprehensive collaboration across type-aware interaction behavior representations, and discriminate their inherent contributions in assisting recommendations. Extensive experiments on two benchmark datasets and a real-world e-commence user behavior data demonstrate significant improvements obtained by MATN over baselines. Codes are available at: https://github.com/akaxlh/MATN.

[1]  Florian Strub,et al.  Hybrid Recommender System based on Autoencoders , 2018 .

[2]  Lei Zheng,et al.  Joint Deep Modeling of Users and Items Using Reviews for Recommendation , 2017, WSDM.

[3]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[4]  Long Guo,et al.  Buying or Browsing?: Predicting Real-time Purchasing Intent using Attention-based Deep Network with Multiple Behavior , 2019, KDD.

[5]  Gert R. G. Lanckriet,et al.  Top-N Recommendation with Missing Implicit Feedback , 2015, RecSys.

[6]  Jie Tang,et al.  Representation Learning for Attributed Multiplex Heterogeneous Network , 2019, KDD.

[7]  Gao Cong,et al.  Interact and Decide: Medley of Sub-Attention Networks for Effective Group Recommendation , 2018, SIGIR.

[8]  Shujian Huang,et al.  Deep Matrix Factorization Models for Recommender Systems , 2017, IJCAI.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Lei Zheng,et al.  Deep Distribution Network: Addressing the Data Sparsity Issue for Top-N Recommendation , 2019, SIGIR.

[11]  S. C. Hui,et al.  Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking , 2017, WWW.

[12]  Nitesh V. Chawla,et al.  Neural Tensor Factorization for Temporal Interaction Learning , 2019, WSDM.

[13]  Siu Cheung Hui,et al.  Multi-Pointer Co-Attention Networks for Recommendation , 2018, KDD.

[14]  Yong Yu,et al.  Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling , 2019, WSDM.

[15]  Yongfeng Zhang,et al.  Relational Collaborative Filtering: Modeling Multiple Item Relations for Recommendation , 2019, SIGIR.

[16]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[17]  Hanning Zhou,et al.  Neural Autoregressive Collaborative Filtering for Implicit Feedback , 2016, DLRS@RecSys.

[18]  Chen Gao,et al.  Neural Multi-task Recommendation from Multi-behavior Data , 2018, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[19]  Jianling Sun,et al.  Compositional Coding for Collaborative Filtering , 2019, SIGIR.

[20]  Suyu Ge,et al.  Neural News Recommendation with Multi-Head Self-Attention , 2019, EMNLP.

[21]  Yongfeng Zhang,et al.  Sequential Recommendation with User Memory Networks , 2018, WSDM.

[22]  Chenggang Yan,et al.  Leveraging Multiple Implicit Feedback for Personalized Recommendation with Neural Network , 2019, AIAM.

[23]  Peng Jiang,et al.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer , 2019, CIKM.

[24]  Daniel E. Ho,et al.  Is Yelp Actually Cleaning Up the Restaurant Industry? A Re-Analysis on the Relative Usefulness of Consumer Reviews , 2019, WWW.

[25]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[26]  George Karypis,et al.  RecWalk: Nearly Uncoupled Random Walks for Top-N Recommendation , 2019, WSDM.

[27]  Mark Coates,et al.  Memory Augmented Graph Neural Networks for Sequential Recommendation , 2019, AAAI.

[28]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[29]  Martin Ester,et al.  Collaborative Denoising Auto-Encoders for Top-N Recommender Systems , 2016, WSDM.

[30]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[31]  Jure Leskovec,et al.  Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems , 2019, KDD.

[32]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[33]  James Caverlee,et al.  Time to Shop for Valentine's Day: Shopping Occasions and Sequential Recommendation in E-commerce , 2020, WSDM.

[34]  Jian Tang,et al.  Session-Based Social Recommendation via Dynamic Graph Attention Networks , 2019, WSDM.

[35]  Irwin King,et al.  STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems , 2019, IJCAI.

[36]  Jaewoo Kang,et al.  SAIN: Self-Attentive Integration Network for Recommendation , 2019, SIGIR.

[37]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[38]  Jun Wang,et al.  Learning Adaptive Display Exposure for Real-Time Advertising , 2018, CIKM.

[39]  Yiqun Liu,et al.  Social Attentional Memory Network: Modeling Aspect- and Friend-Level Differences in Recommendation , 2019, WSDM.

[40]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[41]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[42]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[43]  Nitesh V. Chawla,et al.  Online Purchase Prediction via Multi-Scale Modeling of Behavior Dynamics , 2019, KDD.

[44]  Martha Larson,et al.  Top-N Recommendation with Multi-Channel Positive Feedback using Factorization Machines , 2019, ACM Trans. Inf. Syst..

[45]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[46]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[47]  Hanning Zhou,et al.  Collaborative Filtering with User-Item Co-Autoregressive Models , 2016, AAAI.

[48]  Tommaso Di Noia,et al.  Top-N recommendations from implicit feedback leveraging linked open data , 2013, IIR.