Exploiting Visual Contents in Posters and Still Frames for Movie Recommendation

Recommender systems, e.g., movie recommendation, play an important role in our life. However, few movie recommendation methods have considered the rich visual content information in posters and still frames, which can be used to alleviate the data sparsity and cold start problems in recommendation. Moreover, no existing paper has taken visual feature learning and recommendation into a unified optimization process. To this end, in this paper, we focus on how to use visual contents to improve the performance of movie recommendation and propose a novel movie recommendation model named unified visual contents matrix factorization (UVMF) that integrates visual feature extraction and recommendation into a unified framework. Specifically, we integrate convolutional neural network into probabilistic matrix factorization, and the model can be trained end-to-end. Moreover, we unfix weights in the last few layers of VGG16 to learn features and adapt them for the movie recommendation task. Finally, the experimental results on real-world data show that UVMF outperforms other benchmark methods in terms of recommendation accuracy.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[3]  Min Zhao,et al.  Social temporal collaborative ranking for context aware movie recommendation , 2013, TIST.

[4]  Michael R. Lyu,et al.  Improving Recommender Systems by Incorporating Social Contextual Information , 2011, TOIS.

[5]  Baoxin Li,et al.  PPP: Joint Pointwise and Pairwise Image Label Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Julian J. McAuley,et al.  Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering , 2016, WWW.

[7]  Tiffany Ya Tang,et al.  The role of user mood in movie recommendations , 2010, Expert Syst. Appl..

[8]  Baoxin Li,et al.  CLARE: A Joint Approach to Label Classification and Tag Recommendation , 2017, AAAI.

[9]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Xiaoli Li,et al.  Where you Instagram?: Associating Your Instagram Photos with Points of Interest , 2015, CIKM.

[11]  Haifeng Zhao,et al.  Bi-Group Bayesian Personalized Ranking from Implicit Feedback , 2019, CSSE.

[12]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[13]  Jianguo Wang,et al.  Sherlock: Sparse Hierarchical Embeddings for Visually-Aware One-Class Collaborative Filtering , 2016, IJCAI.

[14]  Nick Antonopoulos,et al.  CinemaScreen recommender agent: combining collaborative and content-based filtering , 2006, IEEE Intelligent Systems.

[15]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[16]  Qiang Yang,et al.  Matrix Factorization+ for Movie Recommendation , 2016, IJCAI.

[17]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[18]  Rafael Valencia-García,et al.  Solving the cold-start problem in recommender systems with social tags , 2010, Expert Syst. Appl..

[19]  Doo-Soon Park,et al.  Performance Improvement of a Movie Recommendation System based on Personal Propensity and Secure Collaborative Filtering , 2013, J. Inf. Process. Syst..

[20]  Donghyun Kim,et al.  Convolutional Matrix Factorization for Document Context-Aware Recommendation , 2016, RecSys.

[21]  Hui Xiong,et al.  Photo2Trip: Exploiting Visual Contents in Geo-Tagged Photos for Personalized Tour Recommendation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[22]  Xing Xie,et al.  Content-Based Collaborative Filtering for News Topic Recommendation , 2015, AAAI.

[23]  Baoxin Li,et al.  Unsupervised Sentiment Analysis for Social Media Images , 2015, IJCAI.

[24]  Michael R. Lyu,et al.  SoRec: social recommendation using probabilistic matrix factorization , 2008, CIKM '08.

[25]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[26]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Yannis Kalantidis,et al.  Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos , 2013, ICMR.

[28]  Andreas Stafylopatis,et al.  A hybrid movie recommender system based on neural networks , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[29]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[30]  Yunni Xia,et al.  Applying the learning rate adaptation to the matrix factorization based collaborative filtering , 2013, Knowl. Based Syst..