Deep Feature Fusion over Multi-field Categorical Data for Rating Prediction

Many predictive tasks in recommender systems model from categorical variables. Different from continuous features extracted from images and videos, categorical data is discrete and of multi-field while their dependencies are little known, which brings the problem of heavy computation on a large-scale sparse feature space. Deep learning methods have strong feature extraction capabilities and now have been more and more widely applied to recommender systems, but they do not perform well on discrete data. To tackle these two problems, in this paper we propose Deep Feature Fusion Model(DFFM) over sparse multi-field categorical data. DFFM utilizes categorical features as inputs and applies the Stacked Denoising AutoEncoder to obtain a dense representation. We construct a full feature connection layer and adopt a multi-layer convolution neural network to further extract deeper features and convert rating prediction to a classification problem. The extensive experiments on real world datasets show that our proposed method outperforms other state-of-the-art approaches.

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Michael R. Lyu,et al.  Learning to recommend with social trust ensemble , 2009, SIGIR.

[3]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[4]  Alexander J. Smola,et al.  DiFacto: Distributed Factorization Machines , 2016, WSDM.

[5]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Jun Wang,et al.  Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction , 2016, ECIR.

[8]  Chao Liu,et al.  Recommender systems with social regularization , 2011, WSDM '11.

[9]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[10]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[11]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[12]  Neil Yorke-Smith,et al.  TrustSVD: Collaborative Filtering with Both the Explicit and Implicit Influence of User Trust and of Item Ratings , 2015, AAAI.

[13]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[14]  Karol J. Piczak.  Environmental sound classification with convolutional neural networks , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[15]  Alexandros Karatzoglou,et al.  Gaussian process factorization machines for context-aware recommendations , 2014, SIGIR.

[16]  Gang Chen,et al.  Personal recommendation using deep recurrent neural networks in NetEase , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[17]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[18]  Lei Yu,et al.  A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems , 2017, AAAI.

[19]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[20]  Tao Mei,et al.  Personalized Recommendation Combining User Interest and Social Circle , 2014, IEEE Transactions on Knowledge and Data Engineering.

[21]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[24]  Fei Wang,et al.  Social contextual recommendation , 2012, CIKM.

[25]  Donghyun Kim,et al.  Convolutional Matrix Factorization for Document Context-Aware Recommendation , 2016, RecSys.

[26]  Tieniu Tan,et al.  Personalized ranking with pairwise Factorization Machines , 2016, Neurocomputing.

[27]  Mandar Mitra,et al.  Word Embedding based Generalized Language Model for Information Retrieval , 2015, SIGIR.

[28]  Sheng Li,et al.  Deep Collaborative Filtering via Marginalized Denoising Auto-encoder , 2015, CIKM.

[29]  Tong Zhang,et al.  Gradient boosting factorization machines , 2014, RecSys '14.

[30]  Michael R. Lyu,et al.  SoRec: social recommendation using probabilistic matrix factorization , 2008, CIKM '08.

[31]  Lei Zheng,et al.  Joint Deep Modeling of Users and Items Using Reviews for Recommendation , 2017, WSDM.

[32]  Shubhra Kanti Karmaker Santu,et al.  Generative Feature Language Models for Mining Implicit Features from Customer Reviews , 2016, CIKM.

[33]  Dit-Yan Yeung,et al.  Relational Stacked Denoising Autoencoder for Tag Recommendation , 2015, AAAI.

[34]  John Riedl,et al.  Application of Dimensionality Reduction in Recommender System - A Case Study , 2000 .

[35]  Paul Covington,et al.  Deep Neural Networks for YouTube Recommendations , 2016, RecSys.

[36]  Steffen Rendle.  Scaling Factorization Machines to Relational Data , 2013, Proc. VLDB Endow..

[37]  Christoph Freudenthaler,et al.  Bayesian Factorization Machines , 2011 .

[38]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[39]  Bing Liu,et al.  Recommending Items with Conditions Enhancing User Experiences Based on Sentiment Analysis of Reviews , 2016, CBRecSys@RecSys.

[40]  Harald Steck,et al.  Circle-based recommendation in online social networks , 2012, KDD.

[41]  Paul Resnick,et al.  Recommender systems , 1997, CACM.