Sequential User-based Recurrent Neural Network Recommendations

Recurrent Neural Networks are powerful tools for modeling sequences. They are flexibly extensible and can incorporate various kinds of information including temporal order. These properties make them well suited for generating sequential recommendations. In this paper, we extend Recurrent Neural Networks by considering unique characteristics of the Recommender Systems domain. One of these characteristics is the explicit notion of the user recommendations are specifically generated for. We show how individual users can be represented in addition to sequences of consumed items in a new type of Gated Recurrent Unit to effectively produce personalized next item recommendations. Offline experiments on two real-world datasets indicate that our extensions clearly improve objective performance when compared to state-of-the-art recommender algorithms and to a conventional Recurrent Neural Network.

[1]  Jürgen Schmidhuber,et al.  Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[2]  Yoshua Bengio,et al.  The problem of learning long-term dependencies in recurrent networks , 1993, IEEE International Conference on Neural Networks.

[3]  Yoshua Bengio,et al.  Hierarchical Recurrent Neural Networks for Long-Term Dependencies , 1995, NIPS.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Ellen M. Voorhees,et al.  The TREC-8 Question Answering Track Report , 1999, TREC.

[6]  Thorsten Joachims,et al.  Detecting Concept Drift with Support Vector Machines , 2000, ICML.

[7]  F. Gers,et al.  Long short-term memory in recurrent neural networks , 2001 .

[8]  Thomas Hofmann,et al.  Unsupervised Learning by Probabilistic Latent Semantic Analysis , 2004, Machine Learning.

[9]  Xue Li,et al.  Time weight collaborative filtering , 2005, CIKM '05.

[10]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[11]  Fabio A. González,et al.  Performance of Recommendation Systems in Dynamic Streaming Environments , 2007, SDM.

[12]  Guy Shani,et al.  A Survey of Accuracy Evaluation Metrics of Recommendation Tasks , 2009, J. Mach. Learn. Res..

[13]  Linas Baltrunas,et al.  Towards Time-Dependant Recommendation based on Implicit Feedback , 2009 .

[14]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[15]  ShaniGuy,et al.  A Survey of Accuracy Evaluation Metrics of Recommendation Tasks , 2009 .

[16]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[17]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[18]  Lior Rokach,et al.  Recommender Systems Handbook , 2010 .

[19]  Lars Schmidt-Thieme,et al.  Factorizing personalized Markov chains for next-basket recommendation , 2010, WWW '10.

[20]  Yin Zhang,et al.  Exploiting temporal stability and low-rank structure for localization in mobile networks , 2010, MobiCom.

[21]  Christoph Hermann,et al.  Time-Based Recommendations for Lecture Materials , 2010 .

[22]  Nuria Oliver,et al.  Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering , 2010, RecSys '10.

[23]  Xi Chen,et al.  Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization , 2010, SDM.

[24]  C. L. Philip Chen,et al.  Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery , 2011, Int. J. Mach. Learn. Cybern..

[25]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[26]  Alexandros Karatzoglou,et al.  Collaborative temporal order modeling , 2011, RecSys '11.

[27]  MukhopadhyayTridas,et al.  A hidden Markov model for collaborative filtering , 2012 .

[28]  Tim Hussein,et al.  Hybreed: A software framework for developing context-aware hybrid recommender systems , 2012, User Modeling and User-Adapted Interaction.

[29]  Param Vir Singh,et al.  A Hidden Markov Model for Collaborative Filtering , 2010, MIS Q..

[30]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[31]  Iván Cantador,et al.  Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols , 2013, User Modeling and User-Adapted Interaction.

[32]  Geoffrey Zweig,et al.  Context dependent recurrent neural network language model , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[33]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[34]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[35]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[36]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[37]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[38]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[39]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[40]  Dietmar Jannach,et al.  Automated Generation of Music Playlists: Survey and Experiments , 2014, ACM Comput. Surv..

[41]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[42]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[43]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[44]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[45]  Inderjit S. Dhillon,et al.  High-dimensional Time Series Prediction with Missing Values , 2015, 1509.08333.

[46]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[47]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[48]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[49]  João Gama,et al.  An overview on the exploitation of time in collaborative filtering , 2015, WIREs Data Mining Knowl. Discov..

[50]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[51]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[54]  Alexandros Karatzoglou,et al.  Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations , 2016, RecSys.

[55]  Yong Liu,et al.  Improved Recurrent Neural Networks for Session-based Recommendations , 2016, DLRS@RecSys.

[56]  Junwei Wang,et al.  Recurrent neural network based recommendation for time heterogeneous feedback , 2016, Knowl. Based Syst..

[57]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[58]  Thomas Lukasiewicz,et al.  Tag-Aware Personalized Recommendation Using a Deep-Semantic Similarity Model with Negative Sampling , 2016, CIKM.

[59]  Alex Beutel,et al.  Recurrent Recommender Networks , 2017, WSDM.

[60]  Lior Rokach,et al.  Session-Based Recommendations Using Item Embedding , 2017, IUI.

[61]  Scott Sanner,et al.  Deep Sequential Recommendation for Personalized Adaptive User Interfaces , 2017, IUI.

[62]  Ivo D. Dinov,et al.  Deep learning for neural networks , 2018 .