Toward Dynamic User Intention

User intention is an important factor to be considered for recommender systems, which always changes dynamically in different contexts. Recent studies (represented by sequential recommendation) begin to focus on predicting what users want beyond what users like, which are better at capturing user intention and have attracted a surge of interest. However, user intention modeling is non-trivial, because it is generally influenced by various factors, among which item relations and their temporal evolutionary effects are of great importance. For example, consumption of a cellphone will have varying impacts on the demands for its relational items: For complements, the demands are likely to be promoted in the short term; while for substitutes, the long-term effect may take advantage, because users do not need another cellphone immediately. Moreover, the temporal evolutions of different relational effects vary across different domains, which makes it challenging to adaptively take them into consideration. As a result, most existing studies only loosely incorporate item relations by encoding their semantics into embeddings, neglecting fine-grained time-aware effects. In this work, we propose Knowledge-aware Dynamic Attention (KDA) to take both relational effects and their temporal evolutions into consideration. Specifically, to model dynamic impacts of historical relational interactions on user intention, we aggregate the history sequence into relation-specific embeddings, where the attention weight consists of two parts. First, we measure the relational intensity between historical items and the target item to model the absolute degree of influence in terms of each relation. Second, to model how the relational effects drift with time, we innovatively introduce Fourier transform with learnable frequency-domain embeddings to estimate temporal decay functions of different relations adaptively. Subsequently, the self-attention mechanism is leveraged to derive the final representation of the whole history sequence, which reflects the dynamic user intention and will be applied to generate the recommendation list. Extensive experiments in three real-world datasets indicate the proposed KDA model significantly outperforms the state-of-the-art methods on the Top-K recommendation task. Moreover, the proposed Fourier-based method opens up a new avenue to adaptively integrate temporal dynamics into general neural models.

[1]  Edward Y. Chang,et al.  Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks , 2018, SIGIR.

[2]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[3]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[4]  Xiangnan He,et al.  Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention , 2017, SIGIR.

[5]  Guibing Guo,et al.  Deep Learning-Based Sequential Recommender Systems: Concepts, Algorithms, and Evaluations , 2019, ICWE.

[6]  Yujie Wang,et al.  Time Interval Aware Self-Attention for Sequential Recommendation , 2020, WSDM.

[7]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[8]  Yiqun Liu,et al.  Make It a Chorus: Knowledge- and Time-aware Item Modeling for Sequential Recommendation , 2020, SIGIR.

[9]  Caihua Wu,et al.  Deep Learning Based Recommendation: A Survey , 2017, ICISA.

[10]  Hwanjo Yu,et al.  Do "Also-Viewed" Products Help User Rating Prediction? , 2017, WWW.

[11]  Xue Li,et al.  Time weight collaborative filtering , 2005, CIKM '05.

[12]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[13]  A. Hawkes Spectra of some self-exciting and mutually exciting point processes , 1971 .

[14]  Xu Chen,et al.  Learning over Knowledge-Base Embeddings for Recommendation , 2018, ArXiv.

[15]  Yiqun Liu,et al.  Social Attentional Memory Network: Modeling Aspect- and Friend-Level Differences in Recommendation , 2019, WSDM.

[16]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[17]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[18]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[19]  Joemon M. Jose,et al.  A Simple Convolutional Generative Network for Next Item Recommendation , 2018, WSDM.

[20]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[21]  Jie Liu,et al.  Representing and Recommending Shopping Baskets with Complementarity, Compatibility and Loyalty , 2018, CIKM.

[22]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[23]  Alejandro Bellogín,et al.  Time and sequence awareness in similarity metrics for recommendation , 2020, Inf. Process. Manag..

[24]  Yiqun Liu,et al.  An Efficient Adaptive Transfer Neural Network for Social-aware Recommendation , 2019, SIGIR.

[25]  George Karypis,et al.  FISM: factored item similarity models for top-N recommender systems , 2013, KDD.

[26]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[27]  Yiqun Liu,et al.  Modeling Item-Specific Temporal Dynamics of Repeat Consumption for Recommender Systems , 2019, WWW.

[28]  Le Song,et al.  Time-Sensitive Recommendation From Recurrent User Activities , 2015, NIPS.

[29]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[30]  Gerhard Widmer,et al.  Learning in the Presence of Concept Drift and Hidden Contexts , 1996, Machine Learning.

[31]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[32]  Yixin Cao,et al.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences , 2019, WWW.

[33]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[34]  Jiayu Zhou,et al.  Who, What, When, and Where: Multi-Dimensional Collaborative Recommendations Using Tensor Factorization on Sparse User-Generated Data , 2015, WWW.

[35]  Mengting Wan,et al.  Recommendation Through Mixtures of Heterogeneous Item Relationships , 2018, CIKM.

[36]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[37]  Alessandro Bozzon,et al.  Interacting Attention-gated Recurrent Networks for Recommendation , 2017, CIKM.

[38]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[39]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[40]  Yiqun Liu,et al.  Jointly Learning Explainable Rules for Recommendation with Knowledge Graph , 2019, WWW.

[41]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[42]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[43]  Yiqun Liu,et al.  Efficient Neural Matrix Factorization without Sampling for Recommendation , 2020, ACM Trans. Inf. Syst..

[44]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[45]  Julian J. McAuley,et al.  Translation-based Recommendation: A Scalable Method for Modeling Sequential Behavior , 2018, IJCAI.

[46]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[47]  Julian J. McAuley,et al.  Translation-based factorization machines for sequential recommendation , 2018, RecSys.

[48]  Yongfeng Zhang,et al.  Relational Collaborative Filtering: Modeling Multiple Item Relations for Recommendation , 2019, SIGIR.

[49]  Nuria Oliver,et al.  Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering , 2010, RecSys '10.

[50]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[51]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[52]  Elena Smirnova,et al.  Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks , 2017, DLRS@RecSys.

[53]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[55]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[56]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[57]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[58]  Xiangliang Zhang,et al.  Multi-Order Attentive Ranking Model for Sequential Recommendation , 2019, AAAI.

[59]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[60]  Guy Shani,et al.  An MDP-Based Recommender System , 2002, J. Mach. Learn. Res..

[61]  Julian J. McAuley,et al.  Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering , 2016, WWW.

[62]  A D Baddeley,et al.  Prior recall of newly learned items and the recency effect in free recall. , 1968, Canadian journal of psychology.

[63]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[64]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.