Introduction to Large-Scale Visual Geo-localization

Despite recent advances in computer vision and large-scale indexing techniques, automatic geo-localization of images and videos remains a challenging task. The majority of existing computer vision solutions for geo-localization are limited to highly-visited urban regions for which a significant amount of geo-tagged imagery is available, and therefore, do not scale well to large and ordinary geo-spatial regions. In this chapter, we provide an overview of the major research themes in visual geo-localization, investigate the challenges, and point to problem areas that will benefit from common synthesis of perspectives from these research themes. In particular, we discuss how the availability of web-scale geo-referenced data affects visual geo-localization, what role semantic information plays in this problem, and how precise localization can be achieved using large-scale textured (RGB) and untextured (non-RGB) 3D models. We also introduce a few real-world applications which became feasible as a result of the capability of estimating an image’s geo-location. We conclude this chapter by providing an overview of the emerging trends in visual geo-localization and a summary of the rest of the chapters of the book.

[1]  Harpreet S. Sawhney,et al.  Registration of video to geo-referenced imagery , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[2]  Serge J. Belongie,et al.  Cross-View Image Geolocalization , 2013, CVPR.

[3]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[4]  Afshin Dehghan,et al.  Visual business recognition: a multimodal approach , 2013, MM '13.

[5]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[6]  Tomás Pajdla,et al.  Avoiding Confusing Features in Place Recognition , 2010, ECCV.

[7]  Mubarak Shah,et al.  GIS-Assisted Object Detection and Geospatial Localization , 2014, ECCV.

[8]  Mubarak Shah,et al.  Image Geo-Localization Based on MultipleNearest Neighbor Feature Matching UsingGeneralized Graphs , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[10]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[12]  Michael Goesele,et al.  Scene Reconstruction and Visualization From Community Photo Collections , 2010, Proceedings of the IEEE.

[13]  Yong Jae Lee,et al.  Style-Aware Mid-level Representation for Discovering Visual Connections in Space and Time , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Amir Roshan Zamir,et al.  City scale geo-spatial trajectory estimation of a moving camera , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Hans-Peter Seidel,et al.  Automatic photo-to-terrain alignment for the annotation of mountain pictures , 2011, CVPR 2011.

[16]  Masatoshi Okutomi,et al.  Visual Place Recognition with Repetitive Structures , 2013, CVPR.

[17]  Mayank Bansal,et al.  Ultra-wide Baseline Facade Matching for Geo-localization , 2012, ECCV Workshops.

[18]  Anthony Stefanidis,et al.  Image-to-X Registration using Linear Features , 2007, 2007 IEEE International Fuzzy Systems Conference.

[19]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[20]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[21]  Anthony Stefanidis,et al.  Relaxation Matching for Georegistration of Aerial and Satellite Imagery , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Robert Pless,et al.  Webcam geo-localization using aggregate light levels , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[23]  Pascal Fua,et al.  Worldwide Pose Estimation Using 3D Point Clouds , 2012, ECCV.

[24]  Jana Kosecka,et al.  Creating compact architectural models by geo-registering image collections , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[25]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR.

[26]  Avideh Zakhor,et al.  User-Driven Geolocation of Untagged Desert Imagery Using Digital Elevation Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[27]  Frank Dellaert,et al.  4D Cities: Analyzing, Visualizing, and Interacting with Historical Urban Photo Collections , 2012, J. Multim..

[28]  Scott Workman,et al.  Wide-Area Image Geolocalization with Aerial Reference Imagery , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Tomás Pajdla,et al.  Learning and Calibrating Per-Location Classifiers for Visual Place Recognition , 2013, CVPR.

[30]  Xin Chen,et al.  City-scale landmark identification on mobile devices , 2011, CVPR 2011.

[31]  Robert Pless,et al.  Geolocating Static Cameras , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[32]  Hui Cheng,et al.  Geo-localization of street views with aerial image databases , 2011, ACM Multimedia.

[33]  Mubarak Shah,et al.  Accurate Image Localization Based on Google Maps Street View , 2010, ECCV.

[34]  Richard Szeliski,et al.  City-Scale Location Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Alexei A. Efros,et al.  What makes Paris look like Paris? , 2015, Commun. ACM.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Masatoshi Okutomi,et al.  24/7 Place Recognition by View Synthesis , 2015, CVPR.

[38]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[39]  Serge J. Belongie,et al.  Learning deep representations for ground-to-aerial geolocalization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Yaser Sheikh,et al.  Feature-based Georegistration of Aerial Images , 2004 .

[41]  Marc Pollefeys,et al.  Large Scale Visual Geo-Localization of Images in Mountainous Terrain , 2012, ECCV.