Pair-Wise Convolution Network with Transformers for Sequential Recommendation

Sequential recommendations seek to employ the sequence of interactions between users and commodities to predict their next behavior based on the behavior they have recently made. Previously, some recommendation systems have been built on Markov chains and recurrent neural networks (among others). However, these methods have many limitations that they emphasize too much sequence change to fully emphasize the correlation between adjacent items; Besides, they generally ignore the influence of contextual information. To solve the shortcomings of the existing sequential recommendations, we try to model the relationship between items, get an effective representation of sequential features, and capture complex sequence correlations. Specifically, we propose a pair-wise convolution network with transformers for the sequential recommendation. The two-dimensional convolution networks encodes the sequence into a three-dimensional tensor and learns the relationships of features between the sequences. We adopt a residual connection to prevent the gradient from disappearing and solve the loss of feature information. The experimental results show that our method is superior to various advanced sequential models on sparse and dense data sets and different evaluation indicators.

[1]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[2]  Suncong Zheng,et al.  Hierarchical Memory Networks for Answer Selection on Unknown Words , 2016, COLING.

[3]  Chang Zhou,et al.  Deep Interest Evolution Network for Click-Through Rate Prediction , 2018, AAAI.

[4]  Feng Yu,et al.  A Dynamic Recurrent Model for Next Basket Recommendation , 2016, SIGIR.

[5]  Xiangnan He,et al.  Future Data Helps Training: Modeling Future Contexts for Session-based Recommendation , 2020, WWW.

[6]  Hui Xiong,et al.  Sequential Recommender System based on Hierarchical Attention Networks , 2018, IJCAI.

[7]  Qiao Liu,et al.  STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation , 2018, KDD.

[8]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[9]  Viswanathan Swaminathan,et al.  Feature Selection for FM-Based Context-Aware Recommendation Systems , 2017, 2017 IEEE International Symposium on Multimedia (ISM).

[10]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[11]  Joemon M. Jose,et al.  A Simple Convolutional Generative Network for Next Item Recommendation , 2018, WSDM.

[12]  Li Xu,et al.  Evolutionarily stable opportunistic spectrum access in cognitive radio networks , 2016, IET Commun..

[13]  Yunming Ye,et al.  Dual-attentional Factorization-Machines based Neural Network for User Response Prediction , 2020, WWW.

[14]  Shuo Cheng,et al.  CosRec: 2D Convolutional Neural Networks for Sequential Recommendation , 2019, CIKM.

[15]  Wei Li,et al.  Behavior sequence transformer for e-commerce recommendation in Alibaba , 2019, Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data.

[16]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[17]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[18]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[19]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.