Tag recommendation for georeferenced photos

This paper presents methods for annotating georeferenced photos with descriptive tags, exploring the annotations for other georeferenced photos which are available at online repositories like Flickr. Specifically, by using the geospatial coordinates associated to the photo which we want to annotate, we start by collecting the photos from an online repository which were taken from nearby locations. Next, and for each tag associated to the collected photos, we compute a set of relevance estimators with basis on factors such as the tag frequency, the geospatial proximity of the photo, the image content similarity, and the number of different users employing the tag. The multiple estimators can then be combined through supervised learning to rank methods such as Rank-Boost or AdaRank, or through unsupervised rank aggregation methods well-known in the information retrieval literature, namely the CombSUM or the CombMNZ approaches. The most relevant tags are finally suggested. Experimental results with a collection of photos collected from Flickr attest for the adequacy of the proposed approaches.

[1]  Stefanie N. Lindstaedt,et al.  Automatic image annotation using visual content and folksonomies , 2009, Multimedia Tools and Applications.

[2]  Ingmar Weber,et al.  Personalized, interactive tag recommendation for flickr , 2008, RecSys '08.

[3]  Vincent Conitzer,et al.  Computational aspects of preference aggregation , 2006 .

[4]  Stefanie N. Lindstaedt,et al.  Recommending Tags for Pictures Based on Text, Visual Content and User Context , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[5]  C. Burges,et al.  Learning to Rank Using Classification and Gradient Boosting , 2008 .

[6]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[7]  Cristina Videira Lopes,et al.  Tree ensembles for learning to rank , 2011 .

[8]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  R. Manmatha,et al.  Multiple Bernoulli relevance models for image and video annotation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[10]  Ralf Krestel,et al.  Latent dirichlet allocation for tag recommendation , 2009, RecSys '09.

[11]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[12]  Rich Caruana,et al.  Additive Groves of Regression Trees , 2007, ECML.

[13]  Julien Ah-Pine,et al.  On data fusion in information retrieval using different aggregation operators , 2011, Web Intell. Agent Syst..

[14]  Shih-Fu Chang,et al.  To search or to label?: predicting the performance of search-based automatic image classifiers , 2006, MIR '06.

[15]  Jon M. Kleinberg,et al.  Mapping the world's photos , 2009, WWW '09.

[16]  B. S. Manjunath,et al.  Global annotation on georeferenced photographs , 2009, CIVR '09.

[17]  Hang Li.  Learning to Rank for Information Retrieval and Natural Language Processing , 2011, Synthesis Lectures on Human Language Technologies.

[18]  Tao Qin,et al.  Learning to rank relational objects and its application to web search , 2008, WWW.

[19]  Nenghai Yu,et al.  Flickr distance , 2008, ACM Multimedia.

[20]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[21]  Edward A. Fox,et al.  Combination of Multiple Searches , 1993, TREC.

[22]  Andreas Hotho,et al.  Tag Recommendations in Folksonomies , 2007, LWA.

[23]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[24]  Hector Garcia-Molina,et al.  Social tag prediction , 2008, SIGIR '08.

[25]  Qiang Wu,et al.  McRank: Learning to Rank Using Multiple Classification and Gradient Boosting , 2007, NIPS.

[26]  Remco C. Veltkamp,et al.  Content-based image retrieval systems: A survey , 2000 .

[27]  W. Bruce Croft,et al.  Linear feature-based models for information retrieval , 2007, Information Retrieval.

[28]  Raimondo Schettini,et al.  Image annotation using SVM , 2003, IS&T/SPIE Electronic Imaging.

[29]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[30]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[31]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[32]  Pinar Duygulu Sahin,et al.  Automatic tag expansion using visual similarity for photo sharing websites , 2010, Multimedia Tools and Applications.

[33]  Mor Naaman,et al.  ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone? , 2008, IEEE MultiMedia.

[34]  Pinar Duygulu Sahin,et al.  Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites , 2008, SAMT.

[35]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[36]  Hang Li,et al.  AdaRank: a boosting algorithm for information retrieval , 2007, SIGIR.

[37]  Bin Wang,et al.  A graph-based image annotation framework , 2008, Pattern Recognit. Lett..

[38]  Jing Hua,et al.  Region-based Image Annotation using Asymmetrical Support Vector Machine-based Multiple-Instance Learning , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Mohamed Farah,et al.  An outranking approach for rank aggregation in information retrieval , 2007, SIGIR.

[40]  B. S. Manjunath,et al.  Spirittagger: a geo-aware tag suggestion tool mined from flickr , 2008, MIR '08.