Parallel pairwise learning to rank for collaborative filtering

Pairwise learning to rank is known to be suitable for a wide range of collaborative filtering applications. In this work, we show that its efficiency can be greatly improved with parallel stochastic gradient descent schemes. Accordingly, we first propose to extrapolate two such state‐of‐the‐art schemes to the pairwise learning to rank problem setting. We then show the versatility of these proposals by showing the applicability of several important extensions commonly desired in practice. Theoretical as well as extensive empirical analyses of our proposals show remarkable efficiency results for pairwise learning to rank in offline and stream learning settings.

[1]  Martha Larson,et al.  Bayesian Personalized Ranking with Multi-Channel User Feedback , 2016, RecSys.

[2]  Feng Niu,et al.  Million Song Dataset Challenge ! , 2012 .

[3]  Fabio Aiolli,et al.  Efficient top-n recommendation for very large scale binary rated datasets , 2013, RecSys.

[4]  Christopher Ré,et al.  Parallel stochastic gradient algorithms for large-scale matrix completion , 2013, Mathematical Programming Computation.

[5]  Léon Bottou,et al.  Stochastic Gradient Descent Tricks , 2012, Neural Networks: Tricks of the Trade.

[6]  Lars Schmidt-Thieme,et al.  Online-updating regularized kernel matrix factorization models for large-scale recommender systems , 2008, RecSys '08.

[7]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[8]  João Gama,et al.  Fast Incremental Matrix Factorization for Recommendation with Positive-Only Feedback , 2014, UMAP.

[9]  Jeffrey Scott Vitter,et al.  Random sampling with a reservoir , 1985, TOMS.

[10]  Martha Larson,et al.  CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering , 2012, RecSys.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  M. Kubát.  An Introduction to Machine Learning , 2017, Springer International Publishing.

[13]  C. Lee Giles,et al.  Learning on the border: active learning in imbalanced data classification , 2007, CIKM '07.

[14]  Jason Weston,et al.  Learning to rank recommendations with the k-order statistic loss , 2013, RecSys.

[15]  Òscar Celma,et al.  Music Recommendation and Discovery - The Long Tail, Long Fail, and Long Play in the Digital Music Space , 2010 .

[16]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[17]  Tie-Yan Liu,et al.  Learning to Rank for Information Retrieval , 2011 .

[18]  Ashwin Lall,et al.  Exponential Reservoir Sampling for Streaming Language Models , 2014, ACL.

[19]  Steffen Rendle,et al.  Improving pairwise learning for item recommendation from implicit feedback , 2014, WSDM.

[20]  David Mouillot,et al.  Introduction of Relative Abundance Distribution (RAD) Indices, Estimated from the Rank-Frequency Diagrams (RFD), to Assess Changes in Community Diversity , 2000 .

[21]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[22]  Fikret S. Gürgen,et al.  On Parallelizing SGD for Pairwise Learning to Rank in Collaborative Filtering Recommender Systems , 2017, RecSys.

[23]  Jesús S. Aguilar-Ruiz,et al.  Knowledge discovery from data streams , 2009, Intell. Data Anal..

[24]  Chih-Jen Lin,et al.  A fast parallel SGD for matrix factorization in shared memory systems , 2013, RecSys.

[25]  Dietmar Jannach,et al.  Using graded implicit feedback for bayesian personalized ranking , 2014, RecSys '14.

[26]  Massih-Reza Amini,et al.  Learning to Rank for Collaborative Filtering , 2007, ICEIS.

[27]  Lars Schmidt-Thieme,et al.  Real-time top-n recommendation in social streams , 2012, RecSys.

[28]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[29]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[30]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[31]  Benoit B. Mandelbrot,et al.  Fractal Geometry of Nature , 1984 .

[32]  Tong Zhang,et al.  Stochastic Optimization with Importance Sampling for Regularized Loss Minimization , 2014, ICML.

[33]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[34]  Martha Larson,et al.  RecSys Challenge 2016: Job Recommendations , 2016, RecSys.

[35]  João Gama,et al.  An overview on the exploitation of time in collaborative filtering , 2015, WIREs Data Mining Knowl. Discov..

[36]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[37]  Ying-Jie Wu,et al.  An efficient method for autoencoder‐based collaborative filtering , 2019, Concurr. Comput. Pract. Exp..

[38]  Christopher C. Johnson.  Logistic Matrix Factorization for Implicit Feedback Data , 2014 .

[39]  Chih-Jen Lin,et al.  A Learning-Rate Schedule for Stochastic Gradient Methods to Matrix Factorization , 2015, PAKDD.

[40]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[41]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[42]  Li Shang,et al.  AdaError: An Adaptive Learning Rate Method for Matrix Approximation-based Collaborative Filtering , 2018, WWW.

[43]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[44]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.

[45]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[46]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[47]  Junjie Yao,et al.  TeRec: A Temporal Recommender System Over Tweet Stream , 2013, Proc. VLDB Endow..

[48]  Steffen Rendle,et al.  Context-Aware Ranking with Factorization Models , 2010, Studies in Computational Intelligence.

[49]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[50]  Alexandros Karatzoglou,et al.  Learning to rank for recommender systems , 2013, RecSys.