Learning over no-Preferred and Preferred Sequence of items for Robust Recommendation

In this paper, we propose a theoretically founded sequential strategy for training large-scale Recommender Systems (RS) over implicit feedback, mainly in the form of clicks. The proposed approach consists in minimizing pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items followed by a clicked one for each user. We present two variants of this strategy where model parameters are updated using either the momentum method or a gradient-based approach. To prevent from updating the parameters for an abnormally high number of clicks over some targeted items (mainly due to bots), we introduce an upper and a lower threshold on the number of updates for each user. These thresholds are estimated over the distribution of the number of blocks in the training set. The thresholds affect the decision of RS and imply a shift over the distribution of items that are shown to the users. Furthermore, we provide a convergence analysis of both algorithms and demonstrate their practical efficiency over six large-scale collections, both regarding different ranking measures and computational time.

[1]  James Bennett,et al.  The Netflix Prize , 2007 .

[2]  Massih-Reza Amini,et al.  Heterogeneous Dyadic Multi-task Learning with Implicit Feedback , 2018, ICONIP.

[3]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[4]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[5]  Massih-Reza Amini,et al.  Sequential Learning over Implicit Feedback for Robust Large-Scale Recommender Systems , 2019, ECML/PKDD.

[6]  G. Guo,et al.  Deep Learning for Sequential Recommendation , 2019, ACM Trans. Inf. Syst..

[7]  Qiang Yang,et al.  One-Class Collaborative Filtering , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[8]  Parneet Kaur,et al.  Shilling attack models in recommender system , 2016, 2016 International Conference on Inventive Computation Technologies (ICICT).

[9]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[10]  Guy Shani,et al.  An MDP-Based Recommender System , 2002, J. Mach. Learn. Res..

[11]  Chien-Liang Liu,et al.  Large-scale recommender system with compact latent factor model , 2016, Expert Syst. Appl..

[12]  Alexandros Karatzoglou,et al.  Recurrent Neural Networks with Top-k Gains for Session-based Recommendations , 2017, CIKM.

[13]  Quan Z. Sheng,et al.  A Survey on Session-based Recommender Systems , 2019, ArXiv.

[14]  Sudhanshu Singh,et al.  Outbrain Click Prediction , 2016 .

[15]  Massih-Reza Amini,et al.  User preference and embedding learning with implicit feedback for recommender systems , 2021, Data Mining and Knowledge Discovery.

[16]  Julian J. McAuley,et al.  Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[17]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[18]  Yongdong Zhang,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[19]  Nemanja Djuric,et al.  E-commerce in Your Inbox: Product Recommendations at Scale , 2015, KDD.

[20]  Julian J. McAuley,et al.  Self-Attentive Sequential Recommendation , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[21]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[22]  Naixue Xiong,et al.  Deep Matrix Factorization With Implicit Feedback Embedding for Recommendation System , 2019, IEEE Transactions on Industrial Informatics.

[23]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[24]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[25]  P. Gallinari,et al.  A Data-dependent Generalisation Error Bound for the AUC , 2005 .

[26]  Francesco Ricci,et al.  Optimal radio channel recommendations with explicit and implicit feedback , 2012, RecSys.

[27]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[28]  Guanghui Lan,et al.  First-order and Stochastic Optimization Methods for Machine Learning , 2020 .

[29]  Ke Wang,et al.  Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding , 2018, WSDM.

[30]  Hailong Sun,et al.  Recommender systems based on ranking performance optimization , 2015, Frontiers of Computer Science.

[31]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[32]  Chuan Qin,et al.  SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback , 2020, AAAI.

[33]  Mark W. Schmidt,et al.  Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition , 2016, ECML/PKDD.

[34]  Sergey Pavlov,et al.  “Zhores” — Petaflops supercomputer for data-driven modeling, machine learning and artificial intelligence installed in Skolkovo Institute of Science and Technology , 2019, Open Engineering.

[35]  Massih-Reza Amini,et al.  KASANDR: A Large-Scale Dataset with Implicit Feedback for Recommendation , 2017, SIGIR.

[36]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[37]  Ulf Brefeld,et al.  Factored MDPs for detecting topics of user sessions , 2014, RecSys '14.

[38]  Jürgen Ziegler,et al.  Sequential User-based Recurrent Neural Network Recommendations , 2017, RecSys.

[39]  Massih-Reza Amini,et al.  Learning to recommend diverse items over implicit feedback on PANDOR , 2018, RecSys.

[40]  Yurii Nesterov,et al.  Lectures on Convex Optimization , 2018 .

[41]  Boris Polyak Some methods of speeding up the convergence of iteration methods , 1964 .

[42]  Michael J. Pazzani,et al.  Content-Based Recommendation Systems , 2007, The Adaptive Web.

[43]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[44]  A. Nemirovskii,et al.  Optimal methods of smooth convex minimization , 1986 .

[45]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.