Collaborative Deep Learning for Recommender Systems

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recently advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art.

[1]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[2]  R. Strichartz A Guide to Distribution Theory and Fourier Transforms , 1994 .

[3]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[4]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[5]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[6]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[7]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[8]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[9]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[10]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[11]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[12]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[13]  Pinar Duygulu Sahin,et al.  Automatic tag expansion using visual similarity for photo sharing websites , 2010, Multimedia Tools and Applications.

[14]  Deepak Agarwal,et al.  Regression-based latent factor models , 2009, KDD.

[15]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[16]  Geoffrey E. Hinton,et al.  Semantic hashing , 2009, Int. J. Approx. Reason..

[17]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[18]  Alan Said,et al.  Identifying and utilizing contextual data in hybrid recommender systems , 2010, RecSys '10.

[19]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[20]  Ryen W. White,et al.  Recommending interesting activity-related local entities , 2011, SIGIR '11.

[21]  Chong Wang,et al.  Collaborative topic modeling for recommending scientific articles , 2011, KDD.

[22]  Zhihua Zhang,et al.  Generalized Latent Factor Models for Social Network Analysis , 2011, IJCAI.

[23]  Hongyuan Zha,et al.  Learning binary codes for collaborative filtering , 2012, KDD.

[24]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[25]  Yong Yu,et al.  SVDFeature: a toolkit for feature-based collaborative filtering , 2012, J. Mach. Learn. Res..

[26]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Yan Liu,et al.  Collaborative Topic Regression with Social Matrix Factorization for Recommendation Systems , 2012, ICML.

[29]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[30]  Ebru Arisoy,et al.  Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  Guandong Xu,et al.  Personalized recommendation via cross-domain triadic factorization , 2013, WWW.

[32]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[33]  Dit-Yan Yeung,et al.  Learning a Deep Compact Image Representation for Visual Tracking , 2013, NIPS.

[34]  Pierre Baldi,et al.  Understanding Dropout , 2013, NIPS.

[35]  Sida I. Wang,et al.  Dropout Training as Adaptive Regularization , 2013, NIPS.

[36]  Pascal Vincent,et al.  Generalized Denoising Auto-Encoders as Generative Models , 2013, NIPS.

[37]  Preslav Nakov,et al.  A non-IID Framework for Collaborative Filtering with Restricted Boltzmann Machines , 2013, ICML.

[38]  Wu-Jun Li,et al.  Collaborative Topic Regression with Social Regularization for Tag Recommendation , 2013, IJCAI.

[39]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[40]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[41]  Hailong Sun,et al.  Temporal QoS-aware web service recommendation via non-negative tensor factorization , 2014, WWW.

[42]  Ye Wang,et al.  Improving Content-based and Hybrid Music Recommendation using Deep Learning , 2014, ACM Multimedia.

[43]  Wu-Jun Li,et al.  Relational Collaborative Topic Regression for Recommender Systems , 2015, IEEE Transactions on Knowledge and Data Engineering.

[44]  Dit-Yan Yeung,et al.  Relational Stacked Denoising Autoencoder for Tag Recommendation , 2015, AAAI.