Visually weighted neighbor voting for image tag relevance learning

The presence of non-relevant tags in image folksonomies hampers the effective organization and retrieval of user-contributed images. In this paper, we propose to learn the relevance of user-supplied tags by means of visually weighted neighbor voting, a variant of the popular baseline neighbor voting algorithm proposed by Li et al. (IEEE Trans Multimedia 11(7):1310–1322, 2009). To gain insight into the effectiveness of baseline and visually weighted neighbor voting, we qualitatively analyze the difference in tag relevance when using a different number of neighbors, for both tags relevant and tags not relevant to the content of a seed image. Our qualitative analysis shows that tag relevance values computed by means of visually weighted neighbor voting are more stable and representative than tag relevance values computed by means of baseline neighbor voting. This is quantitatively confirmed through extensive experimentation with MIRFLICKR-25000, studying the variation of tag relevance values as a function of the number of neighbors used (for both tags relevant and tags not relevant with respect to the content of a seed image), as well as the influence of tag relevance learning on the effectiveness of image tag refinement, tag-based image retrieval, and image tag recommendation.

[1]  Marcel Worring,et al.  Unsupervised multi-feature tag relevance learning for social image retrieval , 2010, CIVR '10.

[2]  Ming Yang,et al.  Contextual weighting for vocabulary tree based image retrieval , 2011, 2011 International Conference on Computer Vision.

[3]  Wesley De Neve,et al.  Tag refinement in an image folksonomy using visual similarity and tag co-occurrence statistics , 2010, Signal Process. Image Commun..

[4]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[5]  Latifur Khan,et al.  Image annotations by combining multiple evidence & wordNet , 2005, ACM Multimedia.

[6]  Alberto Rodrigues da Silva,et al.  How to Improve Retrieval Effectiveness on the Web , 2004 .

[7]  Dong Liu,et al.  Tag quality improvement for social images , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[8]  Sourav S. Bhowmick,et al.  Quantifying tag representativeness of visual content of social images , 2010, ACM Multimedia.

[9]  De Xu,et al.  Combining visual attention model with multi-instance learning for tag ranking , 2011, Neurocomputing.

[10]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[11]  Syungog An,et al.  Image Indexing Based On Mpeg-7 Scalable Color Descriptor , 2004 .

[12]  Rashmi Chaudhary,et al.  Relevancy tag ranking , 2011, 2011 2nd International Conference on Computer and Communication Technology (ICCCT-2011).

[13]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[14]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[15]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[16]  Steven C. H. Hoi,et al.  A two-view learning approach for image tag ranking , 2011, WSDM '11.

[17]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[19]  Marcel Worring,et al.  Learning Social Tag Relevance by Neighbor Voting , 2009, IEEE Transactions on Multimedia.

[20]  Stefanie N. Lindstaedt,et al.  Automatic image annotation using visual content and folksonomies , 2009, Multimedia Tools and Applications.

[21]  Kilian Q. Weinberger,et al.  Reliable tags using image similarity: mining specificity and expertise from large-scale multimedia databases , 2009, WSMC '09.