Spirittagger: a geo-aware tag suggestion tool mined from flickr

Geographically referenced, or "geo-tagged," photo data sets offer tantalizing potential for automated knowledge discovery in the world. By combining tag reranking based on geographic context with content-based image analysis we are able to suggest geographically relevant tags for photos newly tagged with GPS coordinates. These tag suggestions could be used to help users organize their photo collections or improve retrieval systems. Our algorithm weights labels that correspond to pertinent objects, events, neighborhoods, and activities in a region. While previous work with geo-tagged images has focused on representative views of landmarks or estimating location, our tag suggestion tool, SpiritTagger, suggests tags that reveal an insight into the spirit, or genius loci, of a city or region. Experiments on a data set consisting of over 100,000 Flickr photos in Los Angeles and Southern California show that our geographically relevant tag suggestion tool provides a significant improvement in precision-recall performance over baseline image-based similarity suggestion.

[1]  B. S. Manjunath,et al.  Automatic video annotation through search and mining , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[2]  Gang Wang,et al.  Using Dependent Regions for Object Categorization in a Generative Framework , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  A. Smeaton,et al.  Combination of content analysis and context features for digital photograph retrieval. , 2005 .

[4]  Wei-Ying Ma,et al.  AnnoSearch: Image Auto-Annotation by Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Mor Naaman,et al.  How flickr helps us make sense of the world: context and content in community-contributed media collections , 2007, ACM Multimedia.

[6]  Jitendra Malik,et al.  SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Mor Naaman,et al.  World explorer: visualizing aggregate data from unstructured text in geo-referenced collections , 2007, JCDL '07.

[8]  Akio Yamada,et al.  The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[9]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[11]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[12]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Mor Naaman,et al.  From Where to What: Metadata Sharing for Digital Photographs with Geographic Coordinates , 2003, OTM.