GPS Estimation for Places of Interest From Social Users' Uploaded Photos

Social media has become a very popular way for people to share their photos with friends. Because most of the social images are attached with GPS (geo-tags), a photo's GPS information can be estimated with the help of the large geo-tagged image set while using a visual searching based approach. This paper proposes an unsupervised image GPS location estimation approach with hierarchical global feature clustering and local feature refinement. It consists of two parts: an offline system and an online system. In the offline system, a hierarchical structure is constructed for a large-scale offline social image set with GPS information. Representative images are selected for each GPS location refined cluster, and an inverted file structure is proposed. In the online system, when given an input image, its GPS information can be estimated by hierarchical global clusters selection and local feature refinement in the online system. Both the computational cost and GPS estimation performance demonstrates the effectiveness of the proposed hierarchical structure and inverted file structure in our approach.

[1]  B. S. Manjunath,et al.  Global annotation on georeferenced photographs , 2009, CIVR '09.

[2]  Jon M. Kleinberg,et al.  Mapping the world's photos , 2009, WWW '09.

[3]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[4]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Steven Schockaert,et al.  Ghent University at the 2010 placing task , 2010 .

[6]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Adrian Popescu,et al.  MonuAnno: automatic annotation of georeferenced landmarks images , 2009, CIVR '09.

[8]  Xing Xie,et al.  Mining city landmarks from blogs by graph modeling , 2009, ACM Multimedia.

[9]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[10]  Martha Larson,et al.  Preliminary Exploration of the Use of Geographical Information for Content-based Geo-tagging of Social Video , 2012, MediaEval.

[11]  Daniel P. Huttenlocher,et al.  Landmark classification in large-scale image collections , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Marc Pollefeys,et al.  Leveraging 3D City Models for Rotation Invariant Place-of-Interest Recognition , 2011, International Journal of Computer Vision.

[13]  Alexei A. Efros,et al.  Image sequence geolocation with human travel priors , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Xueming Qian,et al.  Visual summarization of landmarks via viewpoint modeling , 2012, 2012 19th IEEE International Conference on Image Processing.

[15]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[16]  Thomas Sikora,et al.  How Spatial Segmentation improves the Multimodal Geo-Tagging , 2012, MediaEval.

[17]  Gang Hua,et al.  Building contextual visual vocabulary for large-scale image applications , 2010, ACM Multimedia.

[18]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Tao Mei,et al.  Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing , 2012, ACM Multimedia.

[20]  Xueming Qian,et al.  Object Categorization Using Hierarchical Wavelet Packet Texture Descriptors , 2009, 2009 11th IEEE International Symposium on Multimedia.

[21]  Xueming Qian,et al.  HWVP: hierarchical wavelet packet descriptors and their applications in scene categorization and semantic concept retrieval , 2012, Multimedia Tools and Applications.

[22]  Mor Naaman,et al.  Generating diverse and representative image search results for landmarks , 2008, WWW.

[23]  Lei Zhang,et al.  Image retrieval based on micro-structure descriptor , 2011, Pattern Recognit..

[24]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yuan Yan Tang,et al.  GPS Estimation from Users' Photos , 2013, MMM.

[26]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[27]  B. S. Manjunath,et al.  Cortina: a system for large-scale, content-based web image retrieval , 2004, MULTIMEDIA '04.

[28]  Yang Song,et al.  Tour the world: Building a web-scale landmark recognition engine , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Qi Tian,et al.  Mining flickr landmarks by modeling reconstruction sparsity , 2011, TOMCCAP.

[30]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Xueming Qian,et al.  Tagging photos using users' vocabularies , 2013, Neurocomputing.

[32]  T. Sikora,et al.  Video2GPS: Geotagging using collaborative systems, textual and visual features , 2010 .

[33]  Qi Tian,et al.  Spatial coding for large scale partial-duplicate web image search , 2010, ACM Multimedia.

[34]  Steven Schockaert,et al.  Ghent University at the 2011 Placing Task , 2011, MediaEval.

[35]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  William B. Thompson,et al.  Geometric Reasoning for Map-Based Localization , 1996 .

[37]  Jurandy Almeida,et al.  A Multimodal Approach for Video Geocoding , 2012, MediaEval.

[38]  Bart Thomee,et al.  Working Notes for the Placing Task at MediaEval 2013 , 2013, MediaEval.

[39]  G. Gravier,et al.  How INRIA/IRISA identifies Geographic Location of Videos , 2012 .

[40]  Jiebo Luo,et al.  Beyond GPS: determining the camera viewing direction of a geotagged image , 2010, ACM Multimedia.

[41]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[42]  Jan-Michael Frahm,et al.  3D model search and pose estimation from single images using VIP features , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[43]  Luc Van Gool,et al.  World-scale mining of objects and events from community photo collections , 2008, CIVR '08.

[44]  Meng Wang,et al.  Social Image Search with Diverse Relevance Ranking , 2010, MMM.

[45]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.