A Hierarchical Attention Model for Social Contextual Image Recommendation

Image based social networks are among the most popular social networking services in recent years. With a tremendous amount of images uploaded everyday, understanding users’ preferences on user-generated images and making recommendations have become an urgent need. In fact, many hybrid models have been proposed to fuse various kinds of side information (e.g., image visual representation, social network) and user-item historical behavior for enhancing recommendation performance. However, due to the unique characteristics of the user generated images in social image platforms, the previous studies failed to capture the complex aspects that influence users’ preferences in a unified framework. Moreover, most of these hybrid models relied on predefined weights in combining different kinds of information, which usually resulted in sub-optimal recommendation performance. To this end, in this paper, we develop a hierarchical attention model for social contextual image recommendation. In addition to basic latent user interest modeling in the popular matrix factorization based recommendation, we identify three key aspects (i.e., upload history, social influence, and owner admiration) that affect each user's latent preferences, where each aspect summarizes a contextual factor from the complex relationships between users and images. After that, we design a hierarchical attention network that naturally mirrors the hierarchical relationship (elements in each aspects level, and the aspect level) of users’ latent interests with the identified key aspects. Specifically, by taking embeddings from state-of-the-art deep learning models that are tailored for each kind of data, the hierarchical attention network could learn to attend differently to more or less content. Finally, extensive experimental results on real-world datasets clearly show the superiority of our proposed model.

[1]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[2]  Le Wu,et al.  Collaborative Neural Social Recommendation , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[3]  Yueting Zhuang,et al.  User Preference Learning for Online Social Recommendation , 2016, IEEE Transactions on Knowledge and Data Engineering.

[4]  Chen Fang,et al.  Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation , 2016, RecSys.

[5]  Chen Liu,et al.  Modeling User Session and Intent with an Attention-based Encoder-Decoder Architecture , 2017, RecSys.

[6]  Lina Yao,et al.  AutoSVD++: An Efficient Hybrid Collaborative Filtering Model via Contractive Auto-encoders , 2017, SIGIR.

[7]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[8]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[9]  Xuanjing Huang,et al.  Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network , 2017, IJCAI.

[10]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[11]  Neil Yorke-Smith,et al.  A Novel Recommendation Model Regularized with User Trust and Item Ratings , 2016, IEEE Transactions on Knowledge and Data Engineering.

[12]  Junping Du,et al.  Modeling the Evolution of Users’ Preferences and Social Links in Social Networking Services , 2017, IEEE Transactions on Knowledge and Data Engineering.

[13]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[14]  Jing Huang,et al.  Interpretable Convolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction , 2017, RecSys.

[15]  Zhaohui Wu,et al.  On Deep Learning for Trust-Aware Recommendations in Social Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Yi Tay,et al.  Deep Learning based Recommender System: A Survey and New Perspectives , 2018 .

[17]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[18]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[19]  Le Wu,et al.  Attentive Recurrent Social Recommendation , 2018, SIGIR.

[20]  Ravi Kumar,et al.  Influence and correlation in social networks , 2008, KDD.

[21]  Philip S. Yu,et al.  Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model , 2018, KDD.

[22]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[23]  Qi Zhang,et al.  Hashtag Recommendation Using Attention-Based Convolutional Neural Network , 2016, IJCAI.

[24]  Wei Niu,et al.  Neural Personalized Ranking for Image Recommendation , 2018, WSDM.

[25]  N. Latha,et al.  Personalized Recommendation Combining User Interest and Social Circle , 2015 .

[26]  Qiao Liu,et al.  STAMP: Short-Term Attention/Memory Priority Model for Session-based Recommendation , 2018, KDD.

[27]  Zhong Ming,et al.  Collaborative Recommendation with Multiclass Preference Context , 2017, IEEE Intelligent Systems.

[28]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[29]  Meng Wang,et al.  Tri-Clustered Tensor Completion for Social-Aware Image Tag Refinement , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xiangnan He,et al.  NAIS: Neural Attentive Item Similarity Model for Recommendation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[32]  Julian J. McAuley,et al.  VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback , 2015, AAAI.

[33]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[34]  Alberto Del Bimbo,et al.  Beyond the Product: Discovering Image Posts for Brands in Social Media , 2018, ACM Multimedia.

[35]  Stephanie Rogers,et al.  Related Pins at Pinterest: The Evolution of a Real-World Recommender System , 2017, WWW.

[36]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[37]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[38]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[39]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[40]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[41]  Leon A. Gatys,et al.  Controlling Perceptual Factors in Neural Style Transfer , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Daniel Jurafsky,et al.  A Hierarchical Neural Autoencoder for Paragraphs and Documents , 2015, ACL.

[43]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[44]  Xiangnan He,et al.  Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention , 2017, SIGIR.

[45]  Tao Chen,et al.  How Personality Affects our Likes: Towards a Better Understanding of Actionable Images , 2017, ACM Multimedia.

[46]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[47]  Huan Liu,et al.  What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation , 2017, WWW.

[48]  Tao Chen,et al.  Context-aware Image Tweet Modelling and Recommendation , 2016, ACM Multimedia.

[49]  Chao Liu,et al.  Recommender systems with social regularization , 2011, WSDM '11.

[50]  Dong Liu,et al.  Comparative Deep Learning of Hybrid Representations for Image Recommendations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[52]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[53]  Siu Cheung Hui,et al.  Multi-Pointer Co-Attention Networks for Recommendation , 2018, KDD.

[54]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[55]  Simon Hessner,et al.  Image Style Transfer using Convolutional Neural Networks , 2018 .

[56]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[57]  Le Wu,et al.  A Neural Influence Diffusion Model for Social Recommendation , 2019, SIGIR.

[58]  Fei Wang,et al.  Scalable Recommendation with Social Contextual Information , 2014, IEEE Transactions on Knowledge and Data Engineering.

[59]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[60]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.