Inferring photographic location using geotagged web images

Geotagging has become a recent phenomenon that allows users to visualize and manage photo collections in many new and interesting ways. Unfortunately, manual geotagging of a large collection of pictures on the globe is still a time-consuming and laborious task even though geotagging devices are gradually being adopted. At the same time, there exist billions of legacy pictures taken before the onset of geotagging. In recent times, large collections of Web images have been found to facilitate a number of image understanding tasks including geolocation estimation. In this paper, we leverage user tags along with image content to infer the geolocation of images. Our model builds upon the fact that the visual content and user tags of pictures can together provide significant hints about their geolocations. Using a collection of over a million geotagged pictures, we build location probability maps for commonly used image tags over the entire globe. These maps reflect the collective picture-taking and tagging behaviors of thousands of users from all over the world. We further study the geographic entropy and frequency of user tags as geo-inference features and investigate the usefulness of using these features for selecting geographically meaningful annotations. On the other hand, visual content matching is performed using multiple feature descriptors including tiny images, color histograms, GIST features, and bags of textons. Finally, visual KNN matching based geographic mapping scheme is integrated with tag location probability maps to form a strong geo-inference engine. Experiments have shown improvements over geolocation inference performed using either modality alone.