Unsupervised Learning of Parsimonious General-Purpose Embeddings for User and Location Modeling

Many social network applications depend on robust representations of spatio-temporal data. In this work, we present an embedding model based on feed-forward neural networks which transforms social media check-ins into dense feature vectors encoding geographic, temporal, and functional aspects for modeling places, neighborhoods, and users. We employ the embedding model in a variety of applications including location recommendation, urban functional zone study, and crime prediction. For location recommendation, we propose a Spatio-Temporal Embedding Similarity algorithm (STES) based on the embedding model. In a range of experiments on real life data collected from Foursquare, we demonstrate our model’s effectiveness at characterizing places and people and its applicability in aforementioned problem domains. Finally, we select eight major cities around the globe and verify the robustness and generality of our model by porting pre-trained models from one city to another, thereby alleviating the need for costly local training.

[1]  Q. Mcnemar Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[2]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[3]  Donald E. Knuth,et al.  Dynamic Huffman Coding , 1985, J. Algorithms.

[4]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[5]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[6]  Peter J. van Koppen,et al.  The Time to Rob: Variations in Time of Number of Commercial Robberies , 1999 .

[7]  Johan Bollen,et al.  Hebbian algorithms for a digital library recommendation system , 2002, Proceedings. International Conference on Parallel Processing Workshop.

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  S. Bushway,et al.  Trajectories of Crime at Places: A Longitudinal Study of Street Segments in the City of Seattle , 2004 .

[10]  Paolo Avesani,et al.  Trust-aware recommender systems , 2007, RecSys '07.

[11]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[12]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[13]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[14]  Jianguo Wu,et al.  Spatial pattern of urban functions in the Beijing metropolitan region , 2010 .

[15]  Mao Ye,et al.  Location recommendation for location-based social networks , 2010, GIS '10.

[16]  Cecilia Mascolo,et al.  Exploiting place features in link prediction on location-based social networks , 2011, KDD.

[17]  Cecilia Mascolo,et al.  Exploiting Semantic Annotations for Clustering Geographic Areas and Users in Location-based Social Networks , 2011, The Social Mobile Web.

[18]  Kyumin Lee,et al.  Exploring Millions of Footprints in Location Sharing Services , 2011, ICWSM.

[19]  Krzysztof Janowicz,et al.  On the semantic annotation of places in location-based social networks , 2011, KDD.

[20]  Martha Larson,et al.  The where in the tweet , 2011, CIKM '11.

[21]  Norman M. Sadeh,et al.  The Livehoods Project: Utilizing Social Media to Understand the Dynamics of a City , 2012, ICWSM.

[22]  Michael R. Lyu,et al.  Fused Matrix Factorization with Geographical and Social Influence in Location-Based Social Networks , 2012, AAAI.

[23]  Xiaofeng Wang,et al.  Automatic Crime Prediction Using Events Extracted from Twitter Posts , 2012, SBP.

[24]  Kevin C. Almeroth,et al.  Social computing: an intersection of recommender systems, trust/reputation systems, and social networks , 2012, IEEE Network.

[25]  Alexander J. Smola,et al.  Discovering geographical topics in the twitter stream , 2012, WWW.

[26]  Imad Aad,et al.  The Mobile Data Challenge: Big Data for Mobile Computing Research , 2012 .

[27]  Frank Dürr,et al.  PShare: Position sharing for location privacy based on multi-secret sharing , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications.

[28]  Bruno Martins,et al.  Predicting future locations with hidden Markov models , 2012, UbiComp.

[29]  Mohamed F. Mokbel,et al.  Location-based and preference-aware recommendation using sparse geo-social networking data , 2012, SIGSPATIAL/GIS.

[30]  Quoc V. Le,et al.  Exploiting Similarities among Languages for Machine Translation , 2013, ArXiv.

[31]  Bo Hu,et al.  Spatio-Temporal Topic Modeling in Mobile Social Media for Location Recommendation , 2013, 2013 IEEE 13th International Conference on Data Mining.

[32]  Huan Liu,et al.  Exploring temporal effects for location recommendation on location-based social networks , 2013, RecSys.

[33]  Satish V. Ukkusuri,et al.  Understanding urban human activity and mobility patterns using large-scale location-based data from online social media , 2013, UrbComp '13.

[34]  Michael R. Lyu,et al.  Where You Like to Go Next: Successive Point-of-Interest Recommendation , 2013, IJCAI.

[35]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[36]  Tomoharu Iwata,et al.  Geo topic model: joint modeling of user's activity area and interests for location recommendation , 2013, WSDM.

[37]  Hui Xiong,et al.  Learning geographical preferences for point-of-interest recommendation , 2013, KDD.

[38]  John Krumm,et al.  Placer: semantic place labels from diary data , 2013, UbiComp.

[39]  Claudio Bettini,et al.  A Platform for Privacy-Preserving Geo-social Recommendation of Points of Interest , 2013, 2013 IEEE 14th International Conference on Mobile Data Management.

[40]  Aida Mustapha,et al.  An experimental study of classification algorithms for crime prediction. , 2013 .

[41]  Martin Ester,et al.  Spatial topic modeling in online social media for location recommendation , 2013, RecSys.

[42]  Satish V. Ukkusuri,et al.  Urban activity pattern classification using topic models from online geo-location data , 2014 .

[43]  Chi-Yin Chow,et al.  LORE: exploiting sequential influence for location recommendations , 2014, SIGSPATIAL/GIS.

[44]  Wenhao Huang,et al.  Traffic zone division using mobile billing data , 2014, 2014 11th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[45]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[46]  Matthew S. Gerber,et al.  Predicting crime using Twitter and kernel density estimation , 2014, Decis. Support Syst..

[47]  Ling Chen,et al.  LCARS , 2014, ACM Trans. Inf. Syst..

[48]  Xing Xie,et al.  GeoMF: joint geographical modeling and matrix factorization for point-of-interest recommendation , 2014, KDD.

[49]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[50]  Ting Liu,et al.  User Modeling with Neural Network for Review Rating Prediction , 2015, IJCAI.

[51]  Shazia Wasim Sadiq,et al.  Joint Modeling of User Check-in Behaviors for Point-of-Interest Recommendation , 2015, CIKM.

[52]  Ting Liu,et al.  Learning Semantic Representations of Users and Products for Document Level Sentiment Classification , 2015, ACL.

[53]  Hui Xiong,et al.  Discovering Urban Functional Zones Using Latent Activity Trajectories , 2015, IEEE Transactions on Knowledge and Data Engineering.

[54]  Xiaoli Li,et al.  Rank-GeoFM: A Ranking based Geographical Factorization Method for Point of Interest Recommendation , 2015, SIGIR.

[55]  Ling Chen,et al.  Geo-SAGE: A Geographical Sparse Additive Generative Model for Spatial Item Recommendation , 2015, KDD.

[56]  Jing Yang,et al.  Learning functional compositions of urban spaces with crowd-augmented travel survey data , 2015, SIGSPATIAL/GIS.

[57]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Makbule Gulcin Ozsoy,et al.  From Word Embeddings to Item Recommendation , 2016, ArXiv.

[59]  Jia Li,et al.  Tweet modeling with LSTM recurrent neural networks for hashtag recommendation , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[60]  Hao Wang,et al.  Adapting to User Interest Drift for POI Recommendation , 2016, IEEE Transactions on Knowledge and Data Engineering.

[61]  Weitong Chen,et al.  Learning Graph-based POI Embedding for Location-based Recommendation , 2016, CIKM.

[62]  Wen Li,et al.  Probabilistic Local Expert Retrieval , 2016, ECIR.

[63]  Ling Chen,et al.  SPORE: A sequential personalized spatial item recommender system , 2016, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[64]  Xin Liu,et al.  Exploring the Context of Locations for Personalized Location Recommendations , 2016, IJCAI.

[65]  Yang Song,et al.  Multi-Rate Deep Learning for Temporal Recommendation , 2016, SIGIR.

[66]  Zi Huang,et al.  Joint Modeling of User Check-in Behaviors for Real-time Point-of-Interest Recommendation , 2016, ACM Trans. Inf. Syst..

[67]  Yu Hirate,et al.  Distributed Representation-based Recommender Systems in E-commerce , 2016 .

[68]  Amit P. Sheth,et al.  Word Embeddings to Enhance Twitter Gang Member Profile Identification , 2016, ArXiv.

[69]  Tieniu Tan,et al.  Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts , 2016, AAAI.

[70]  Shazia Wasim Sadiq,et al.  A Spatial-Temporal Topic Model for the Semantic Annotation of POIs in LBSNs , 2016, ACM Trans. Intell. Syst. Technol..

[71]  Thomas Hofmann,et al.  Semantic Place Descriptors for Classification and Map Discovery , 2016, ArXiv.

[72]  Bowen Zhou,et al.  A Structured Self-attentive Sentence Embedding , 2017, ICLR.

[73]  Camille Roth,et al.  Natural Scales in Geographical Patterns , 2017, Scientific Reports.

[74]  Gao Cong,et al.  An Experimental Evaluation of Point-of-interest Recommendation in Location-based Social Networks , 2017, Proc. VLDB Endow..

[75]  Michael R. Lyu,et al.  Geo-Teaser: Geo-Temporal Sequential Embedding Rank for Point-of-interest Recommendation , 2016, WWW.

[76]  Ling Chen,et al.  Spatial-Aware Hierarchical Collaborative Deep Learning for POI Recommendation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[77]  A. Churcha,et al.  Transport and social exclusion in London , 2022 .