Quest for relevant tags using local interaction networks and visual content

Typical tag recommendation systems for photos shared on social networks such as Flickr, use visual content analysis, collaborative filtering or personalization strategies to produce annotations. However, the dependence on manual intervention and the knowledge of sufficient personal preferences coupled with the folksonomic issues limit the scope of these strategies. In this paper, we present a fully automatic and folksonomically scalable tag recommendation model that can recommend tags for a user's photos without an explicit knowledge of the user's personal tagging preferences. The model is learned using the collective tagging behavior of other users in the user's local interaction network, which we believe approximates the user's preferences, at least partially. The tag recommendation model generates content-based annotations and then uses a Naïve Bayes formulation to translate these annotations to a set of folksonomic tags selected from the tags used by the users in the local interaction network. Quantitative and qualitative comparisons with 890 Flickr networks show that this approach is highly useful for tag recommendation in the presence of insufficient information of a user's own preferences.

[1]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Pinar Duygulu Sahin,et al.  Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites , 2008, SAMT.

[3]  Bernardo A. Huberman,et al.  Usage patterns of collaborative tagging systems , 2006, J. Inf. Sci..

[4]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[6]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[7]  James Ze Wang,et al.  Real-Time Computerized Annotation of Pictures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  James Ze Wang,et al.  Tagging over time: real-world image annotation by lightweight meta-learning , 2007, ACM Multimedia.

[9]  Stefanie N. Lindstaedt,et al.  Recommending Tags for Pictures Based on Text, Visual Content and User Context , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[10]  John Riedl,et al.  tagging, communities, vocabulary, evolution , 2006, CSCW '06.

[11]  Stefanie N. Lindstaedt,et al.  Automatic image annotation using visual content and folksonomies , 2009, Multimedia Tools and Applications.

[12]  Wei-Ying Ma,et al.  AnnoSearch: Image Auto-Annotation by Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  Wei-Ying Ma,et al.  Image annotation by large-scale content-based image retrieval , 2006, MM '06.

[14]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[15]  Ingmar Weber,et al.  Personalized, interactive tag recommendation for flickr , 2008, RecSys '08.

[16]  B. S. Manjunath,et al.  Spirittagger: a geo-aware tag suggestion tool mined from flickr , 2008, MIR '08.

[17]  Mor Naaman,et al.  Leveraging context to resolve identity in photo albums , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[18]  Rong Yan,et al.  An efficient manual image annotation approach based on tagging and browsing , 2007, MS '07.

[19]  Bamshad Mobasher,et al.  Personalized recommendation in social tagging systems using hierarchical clustering , 2008, RecSys '08.

[20]  Mor Naaman,et al.  ZoneTag's Collaborative Tag Suggestions: What is This Person Doing in My Phone? , 2008, IEEE MultiMedia.

[21]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[22]  Daniel Gatica-Perez,et al.  Analyzing Flickr groups , 2008, CIVR '08.