Photo2Trip: Exploiting Visual Contents in Geo-Tagged Photos for Personalized Tour Recommendation

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized tour recommendation. However, no existing work has considered the visual content information in these photos for tour recommendation. We believe the visual features of photos provide valuable information on measuring user / Point-of-Interest (POI) similarities, which is challenging due to data sparsity. To this end, in this paper, we propose a visual feature enhanced tour recommender system, named ‘Photo2Trip’, to utilize the visual contents and collaborative filtering models for recommendation. Specifically, we propose a Visual-enhanced Probabilistic Matrix Factorization model (VPMF), which integrates visual features into the collaborative filtering model, to learn user interests by leveraging the historical travel records. We then extend VPMF to End-to-End training framework to incorporate users (POIs) latent factors into the learning process of the visual content of photos, which generalizes the applicability of the proposed VPMF framework in tour recommendation. Extensive empirical studies verify that our proposed visual-enhanced personalized tour recommendation method outperforms other benchmark methods in terms of recommendation accuracy. The results also show that visual features are effective in alleviating the data sparsity and cold start problems on personalized tour recommendation.

[1]  Ulf Brefeld,et al.  MDP-based Itinerary Recommendation using Geo-Tagged Social Media , 2018, IDA.

[2]  Bingbing Ni,et al.  First-Person Daily Activity Recognition With Manipulated Object Proposals and Non-Linear Feature Fusion , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Liang Chang,et al.  Personalized POIs Travel Route Recommendation System Based on Tourism Big Data , 2018, PRICAI.

[4]  Kotagiri Ramamohanarao,et al.  A Jointly Learned Context-Aware Place of Interest Embedding for Trip Recommendations , 2018, ArXiv.

[5]  Bo Shen,et al.  Fused matrix factorization with multi-tag, social and geographical influences for POI recommendation , 2018, World Wide Web.

[6]  Ickjai Lee,et al.  Itinerary recommender system with semantic trajectory pattern mining from geo-tagged photos , 2018, Expert Syst. Appl..

[7]  Christopher Leckie,et al.  Personalized trip recommendation for tourists based on user interests, points of interest visit durations and visit recency , 2018, Knowledge and Information Systems.

[8]  Tao Li,et al.  Attention-based recurrent neural network for location recommendation , 2017, 2017 12th International Conference on Intelligent Systems and Knowledge Engineering (ISKE).

[9]  Christopher Leckie,et al.  Personalized Itinerary Recommendation with Queuing Time Awareness , 2017, SIGIR.

[10]  Huan Liu,et al.  What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation , 2017, WWW.

[11]  Charu C. Aggarwal,et al.  Linked Document Embedding for Classification , 2016, CIKM.

[12]  Cheng Soon Ong,et al.  Learning Points and Routes to Recommend Trajectories , 2016, CIKM.

[13]  Hui Xiong,et al.  Unified Point-of-Interest Recommendation with Temporal Interval Assessment , 2016, KDD.

[14]  Christopher Leckie,et al.  Personalized Tour Recommendation Based on User Interests and Points of Interest Visit Durations , 2015, IJCAI.

[15]  Kwan Hui Lim.  Recommending Tours and Places-of-Interest based on User Interests from Geo-tagged Photos , 2015, SIGMOD PhD Symposium.

[16]  Tao Mei,et al.  Author Topic Model-Based Collaborative Filtering for Personalized POI Recommendations , 2015, IEEE Transactions on Multimedia.

[17]  David A. Shamma,et al.  The New Data and New Challenges in Multimedia Research , 2015, ArXiv.

[18]  Raffaele Perego,et al.  On planning sightseeing tours with TripBuilder , 2015, Inf. Process. Manag..

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Hui Xiong,et al.  Exploiting Heterogeneous Human Mobility Patterns for Intelligent Bus Routing , 2014, 2014 IEEE International Conference on Data Mining.

[21]  Ye Wang,et al.  Improving Content-based and Hybrid Music Recommendation using Deep Learning , 2014, ACM Multimedia.

[22]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[23]  Charalampos Konstantopoulos,et al.  A survey on algorithmic approaches for solving tourist trip design problems , 2014, Journal of Heuristics.

[24]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[25]  Raffaele Perego,et al.  Where shall we go today?: planning touristic tours with tripbuilder , 2013, CIKM.

[26]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[27]  Huan Liu,et al.  Exploring temporal effects for location recommendation on location-based social networks , 2013, RecSys.

[28]  Nadia Magnenat-Thalmann,et al.  Time-aware point-of-interest recommendation , 2013, SIGIR.

[29]  Hui Xiong,et al.  Point-of-Interest Recommendation in Location Based Social Networks with Topic and Location Awareness , 2013, SDM.

[30]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[31]  Le Wu,et al.  Leveraging tagging for neighborhood-aware probabilistic matrix factorization , 2012, CIKM.

[32]  Michael R. Lyu,et al.  Fused Matrix Factorization with Geographical and Social Influence in Location-Based Social Networks , 2012, AAAI.

[33]  Hong-Yuan Mark Liao,et al.  Personalized travel recommendation by mining people attributes from community-contributed photos , 2011, ACM Multimedia.

[34]  Hui Xiong,et al.  Cost-aware travel tour recommendation , 2011, KDD.

[35]  Mao Ye,et al.  Exploiting geographical influence for collaborative point-of-interest recommendation , 2011, SIGIR.

[36]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[37]  Mao Ye,et al.  Location recommendation for location-based social networks , 2010, GIS '10.

[38]  Takahiro Hara,et al.  Mining people's trips from large scale geo-tagged photos , 2010, ACM Multimedia.

[39]  Changhu Wang,et al.  Photo2Trip: generating travel routes from geo-tagged photos for trip planning , 2010, ACM Multimedia.

[40]  Ryan P. Adams,et al.  Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Processes , 2010, UAI.

[41]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[42]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[43]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[44]  Michel Gendreau,et al.  Traveling Salesman Problems with Profits , 2005, Transp. Sci..

[45]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[46]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[47]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.