Two-step hybrid collaborative filtering using deep variational Bayesian autoencoders

Abstract Existing recommender systems rely on user and item representations in a fixed continuous low-dimensional latent space. To predict ratings, they use only an implicit feedback matrix , whereas user and item side information is ignored. Furthermore, they use the same arbitrary priors for the user and item latent vectors, reducing the ability of the model to identify the actual latent vectors. Currently, the latent parameters should be learned directly for every user and movie. This is problematic, as it would require both model retraining and learning the latent vector representations when users or items are added to the underlying dataset. To address these issues, we propose a two-step hybrid variational Bayesian autoencoder to characterize the uncertainty of predicted ratings. An encoder is first trained to map data vectors to the latent space so that the latent representations can be dynamically computed. Subsequently, we use the generative process of users and items with their priors as side-specific information to handle matrix sparsity and better learn their latent vectors. Finally, we consider stochastic variational inference to approximate the posterior density of intractable user-item latent vectors. Experiments conducted on two real-world datasets demonstrate the effectiveness of the proposed method compared with state-of-the-art methods.

[1]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[2]  Shujian Huang,et al.  Deep Matrix Factorization Models for Recommender Systems , 2017, IJCAI.

[3]  Yogesh Kumar Meena,et al.  A two-step hybrid unsupervised model with attention mechanism for aspect extraction , 2020, Expert Syst. Appl..

[4]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[5]  Hwanjo Yu,et al.  Deep hybrid recommender systems via exploiting document context and statistics of items , 2017, Inf. Sci..

[6]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[7]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[8]  Samy Bengio,et al.  Generating Sentences from a Continuous Space , 2015, CoNLL.

[9]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[10]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[11]  Matthew D. Hoffman,et al.  On the challenges of learning with inference networks on sparse, high-dimensional data , 2017, AISTATS.

[12]  Florian Strub,et al.  Collaborative Filtering with Stacked Denoising AutoEncoders and Sparse Inputs , 2015, NIPS 2015.

[13]  Matthew D. Hoffman,et al.  Variational Autoencoders for Collaborative Filtering , 2018, WWW.

[14]  Idris Rabiu,et al.  Recommendation system exploiting aspect-based opinion mining with deep learning method , 2020, Inf. Sci..

[15]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[16]  Chong Wang,et al.  Collaborative topic modeling for recommending scientific articles , 2011, KDD.

[17]  Lei Zheng,et al.  Joint Deep Modeling of Users and Items Using Reviews for Recommendation , 2017, WSDM.

[18]  Arun K. Pujari,et al.  Conformal matrix factorization based recommender system , 2018, Inf. Sci..

[19]  Michael L. Nelson,et al.  Comparing the Archival Rate of Arabic, English, Danish, and Korean Language Web Pages , 2017, ACM Trans. Inf. Syst..

[20]  Yee Whye Teh,et al.  Variational Bayesian Approach to Movie Rating Prediction , 2007, KDD 2007.

[21]  Sheng Li,et al.  Deep Collaborative Filtering via Marginalized Denoising Auto-encoder , 2015, CIKM.

[22]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[23]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[24]  Lei Yu,et al.  A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems , 2017, AAAI.

[25]  Martin Ester,et al.  Collaborative Denoising Auto-Encoders for Top-N Recommender Systems , 2016, WSDM.

[26]  Sotirios Chatzis,et al.  Recurrent Latent Variable Networks for Session-Based Recommendation , 2017, DLRS@RecSys.

[27]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[28]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[29]  Mykel J. Kochenderfer,et al.  Amortized Inference Regularization , 2018, NeurIPS.

[30]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Lei Shi,et al.  Local Representative-Based Matrix Factorization for Cold-Start Recommendation , 2017, ACM Trans. Inf. Syst..

[33]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[34]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[35]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[36]  Andrew McCallum,et al.  Ask the GRU: Multi-task Learning for Deep Text Recommendations , 2016, RecSys.

[37]  Huan Liu,et al.  What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation , 2017, WWW.

[38]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[39]  Ju Ren,et al.  A Survey on End-Edge-Cloud Orchestrated Network Computing Paradigms , 2019, ACM Comput. Surv..

[40]  Xiaomei Yu,et al.  Attention-based context-aware sequential recommendation model , 2020, Inf. Sci..

[41]  James She,et al.  Collaborative Variational Autoencoder for Recommender Systems , 2017, KDD.

[42]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[43]  George Karypis,et al.  FISM: factored item similarity models for top-N recommender systems , 2013, KDD.

[44]  Oren Barkan,et al.  Bayesian Neural Word Embedding , 2016, AAAI.

[45]  Kyungwoo Song,et al.  Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information , 2017, CIKM.

[46]  Xiangnan He,et al.  A Generic Coordinate Descent Framework for Learning from Implicit Feedback , 2016, WWW.

[47]  Xiangliang Zhang,et al.  Co-Embedding Attributed Networks , 2019, WSDM.

[48]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.