Knowledge-guided Deep Reinforcement Learning for Interactive Recommendation

Interactive recommendation aims to learn from dynamic interactions between items and users to achieve responsiveness and accuracy. Reinforcement learning is inherently advantageous for coping with dynamic environments and thus has attracted increasing attention in interactive recommendation research. Inspired by knowledge-aware recommendation, we proposed Knowledge-Guided deep Reinforcement learning (KGRL) to harness the advantages of both reinforcement learning and knowledge graphs for interactive recommendation. This model is implemented upon the actor-critic network framework. It maintains a local knowledge network to guide decision-making and employs the attention mechanism to capture long-term semantics between items. We have conducted comprehensive experiments in a simulated online environment with six public real-world datasets and demonstrated the superiority of our model over several state-of-the-art methods.

[1]  Yongfeng Zhang,et al.  Reinforcement Knowledge Graph Reasoning for Explainable Recommendation , 2019, SIGIR.

[2]  Minyi Guo,et al.  Knowledge Graph Convolutional Networks for Recommender Systems , 2019, WWW.

[3]  Yixin Cao,et al.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences , 2019, WWW.

[4]  Christopher C. Johnson.  Logistic Matrix Factorization for Implicit Feedback Data , 2014 .

[5]  Jian Pei,et al.  Community Preserving Network Embedding , 2017, AAAI.

[6]  Dik Lun Lee,et al.  Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks , 2017, KDD.

[7]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Liang Zhang,et al.  Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning , 2018, KDD.

[9]  Nicholas Jing Yuan,et al.  DRN: A Deep Reinforcement Learning Framework for News Recommendation , 2018, WWW.

[10]  Yong Yu,et al.  Large-scale Interactive Recommendation with Tree-structured Policy Gradient , 2018, AAAI.

[11]  Qiong Wu,et al.  Diversity-Promoting Deep Reinforcement Learning for Interactive Recommendation , 2019, ICCSE.

[12]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[13]  Lina Yao,et al.  Collaborative Location Recommendation by Integrating Multi-dimensional Contextual Information , 2018, ACM Trans. Internet Techn..

[14]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[15]  Jun Wang,et al.  Interactive collaborative filtering , 2013, CIKM.

[16]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[17]  Jun Tan,et al.  Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation , 2018, KDD.

[18]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[19]  Huazheng Wang,et al.  Factorization Bandits for Interactive Recommendation , 2017, AAAI.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[22]  Hongxia Jin,et al.  Interactive recommendation via deep neural memory augmented contextual bandits , 2018, RecSys.

[23]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[24]  Liang Zhang,et al.  Deep reinforcement learning for page-wise recommendations , 2018, RecSys.

[25]  Richard Evans,et al.  Deep Reinforcement Learning in Large Discrete Action Spaces , 2015, 1512.07679.

[26]  Artem Grotov,et al.  Online Learning to Rank for Information Retrieval: SIGIR 2016 Tutorial , 2016, SIGIR.

[27]  Larisa Shwartz,et al.  Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms , 2017, IEEE Transactions on Knowledge and Data Engineering.

[28]  Huazheng Wang,et al.  Learning Hidden Features for Contextual Bandits , 2016, CIKM.

[29]  Edward Y. Chang,et al.  Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks , 2018, SIGIR.

[30]  Jure Leskovec,et al.  Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems , 2019, KDD.

[31]  Chang Zhou,et al.  ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation , 2017, AAAI.

[32]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[33]  Lina Yao,et al.  Quaternion Knowledge Graph Embeddings , 2019, NeurIPS.

[34]  Katja Hofmann,et al.  Collective Noise Contrastive Estimation for Policy Transfer Learning , 2016, AAAI.