Large-scale Collaborative Filtering with Product Embeddings

The application of machine learning techniques to large-scale personalized recommendation problems is a challenging task. Such systems must make sense of enormous amounts of implicit feedback in order to understand user preferences across numerous product categories. This paper presents a deep learning based solution to this problem within the collaborative filtering with implicit feedback framework. Our approach combines neural attention mechanisms, which allow for context dependent weighting of past behavioral signals, with representation learning techniques to produce models which obtain extremely high coverage, can easily incorporate new information as it becomes available, and are computationally efficient. Offline experiments demonstrate significant performance improvements when compared to several alternative methods from the literature. Results from an online setting show that the approach compares favorably with current production techniques used to produce personalized product recommendations.

[1]  Malcolm Slaney,et al.  Web-Scale Multimedia Analysis: Does Content Matter? , 2011, IEEE MultiMedia.

[2]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[3]  Daniel M. Roy,et al.  Neural Network Matrix Factorization , 2015, ArXiv.

[4]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[5]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Gang Fu,et al.  Deep & Cross Network for Ad Click Predictions , 2017, ADKDD@KDD.

[8]  Douglas W. Oard,et al.  Implicit Feedback for Recommender Systems , 1998 .

[9]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[10]  Nemanja Djuric,et al.  E-commerce in Your Inbox: Product Recommendations at Scale , 2015, KDD.

[11]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[12]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[15]  Hanning Zhou,et al.  A Neural Autoregressive Approach to Collaborative Filtering , 2016, ICML.

[16]  Tie-Yan Liu,et al.  Ranking Measures and Loss Functions in Learning to Rank , 2009, NIPS.

[17]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[18]  Qiang Yang,et al.  One-Class Collaborative Filtering , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[19]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[20]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[21]  Alexandros Karatzoglou,et al.  Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks , 2017, RecSys.

[22]  Jason Weston,et al.  StarSpace: Embed All The Things! , 2017, AAAI.

[23]  David M. Pennock,et al.  Categories and Subject Descriptors , 2001 .

[24]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[25]  Shan Carter,et al.  Attention and Augmented Recurrent Neural Networks , 2016 .

[26]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[27]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[28]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[29]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[30]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[31]  Martin Wattenberg,et al.  Ad click prediction: a view from the trenches , 2013, KDD.

[32]  R. Kronmal,et al.  On the Alias Method for Generating Random Variables From a Discrete Distribution , 1979 .

[33]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[34]  Haibin Cheng,et al.  Real-time Personalization using Embeddings for Search Ranking at Airbnb , 2018, KDD.

[35]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[36]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[37]  Hanning Zhou,et al.  Neural Autoregressive Collaborative Filtering for Implicit Feedback , 2016, DLRS@RecSys.

[38]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[39]  Hugo Larochelle,et al.  A Meta-Learning Perspective on Cold-Start Recommendations for Items , 2017, NIPS.

[40]  James Bennett,et al.  The Netflix Prize , 2007 .

[41]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[42]  Liana L. Fong,et al.  Faster and Cheaper: Parallelizing Large-Scale Matrix Factorization on GPUs , 2016, HPDC.

[43]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[44]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[45]  Yizhou Sun,et al.  On Sampling Strategies for Neural Network-based Collaborative Filtering , 2017, KDD.

[46]  Elena Smirnova,et al.  Meta-Prod2Vec: Product Embeddings Using Side-Information for Recommendation , 2016, RecSys.

[47]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[48]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[49]  Jason Weston,et al.  Nonlinear latent factorization by embedding multiple user interests , 2013, RecSys.

[50]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[51]  Xiangnan He,et al.  Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention , 2017, SIGIR.

[52]  Alexander M. Rush,et al.  Structured Attention Networks , 2017, ICLR.

[53]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[54]  Subhajit Sanyal,et al.  MRNet-Product2Vec: A Multi-task Recurrent Neural Network for Product Embeddings , 2017, ECML/PKDD.

[55]  Mohammed Bennamoun,et al.  How Well Sentence Embeddings Capture Meaning , 2015, ADCS.

[56]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[57]  Hong Qu,et al.  Attention based collaborative filtering , 2018, Neurocomputing.

[58]  Thomas Nedelec,et al.  Specializing Joint Representations for the task of Product Recommendation , 2017, DLRS@RecSys.

[59]  Tie-Yan Liu,et al.  A Theoretical Analysis of NDCG Type Ranking Measures , 2013, COLT.

[60]  Xiaodong He,et al.  A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems , 2015, WWW.

[61]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[62]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[63]  Cynthia Rudin,et al.  Direct Learning to Rank and Rerank , 2018, AISTATS.

[64]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[65]  Thomas Demeester,et al.  Representation learning for very short texts using weighted word embedding aggregation , 2016, Pattern Recognit. Lett..

[66]  Ryan P. Adams,et al.  Training Restricted Boltzmann Machines on Word Observations , 2012, ICML.

[67]  Christopher C. Johnson.  Logistic Matrix Factorization for Implicit Feedback Data , 2014 .