TDR: Two-stage deep recommendation model based on mSDA and DNN

Abstract Recently, deep learning techniques have been widely used in recommendation tasks and have attained record performance. However, the input quality of the deep learning model has a great influence on the recommendation performance. In this work, an efficient and effective input optimization method is proposed. Specifically, we propose an integrated recommendation framework based on two-stage deep learning. In the first stage, with user and item features as the original input, a low-cost marginalized stacked denoising auto-encoder (mSDA) model is used to learn the latent factors of users and items. In the second stage, the resulting latent factors are combined and used as input vector to the DNN model for fast and accurate prediction. Using the latent factor vector as the input to the deep learning-based recommendation model not only captures the high-order feature interaction, but also reduces the burden of the hidden layer, and also avoids the model training falling into local optimum. Extensive experiments with real-world datasets show that the proposed model shows much better performance than the state-of-the-art recommendation methods in terms of prediction accuracy, parameter space and training speed.

[1]  Jungang Lou,et al.  A novel matrix factorization model for recommendation with LOD-based semantic similarity measure , 2019, Expert Syst. Appl..

[2]  Martin Ester,et al.  Collaborative Denoising Auto-Encoders for Top-N Recommender Systems , 2016, WSDM.

[3]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[4]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[5]  Chih-Jen Lin,et al.  Field-aware Factorization Machines for CTR Prediction , 2016, RecSys.

[6]  Sheng Li,et al.  Deep Collaborative Filtering via Marginalized Denoising Auto-encoder , 2015, CIKM.

[7]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[8]  Liang Lu,et al.  Segmental Recurrent Neural Networks for End-to-End Speech Recognition , 2016, INTERSPEECH.

[9]  Lina Yao,et al.  AutoSVD++: An Efficient Hybrid Collaborative Filtering Model via Contractive Auto-encoders , 2017, SIGIR.

[10]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[11]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[12]  Mao Ye,et al.  Location recommendation for location-based social networks , 2010, GIS '10.

[13]  Martin Wattenberg,et al.  Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation , 2016, TACL.

[14]  Jing Cai,et al.  Enlister: baidu's recommender system for the biggest chinese Q&A website , 2012, RecSys.

[15]  Ying Zhang,et al.  Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks , 2016, INTERSPEECH.

[16]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[17]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[18]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[19]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[20]  Olivier Chapelle,et al.  Field-aware Factorization Machines in a Real-world Online Advertising System , 2017, WWW.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Xing Xie,et al.  xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems , 2018, KDD.

[23]  Shengcai Liao,et al.  Which photo groups should I choose? A comparative study of recommendation algorithms in Flickr , 2010, J. Inf. Sci..

[24]  Lovekesh Vig,et al.  Deep Convolutional Neural Networks for Pairwise Causality , 2017, ArXiv.

[25]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.