Tag ranking based on salient region graph propagation

AbstractIn this paper, the problem of tag ranking by propagating relevance over community-contributed images and their associated tags is explored. To rank the tags more accurately, we propose a novel tag ranking approach based on the salient region. Firstly, we extract the salient region sub-image based on Itti model, and then construct two graphs with the whole image and the salient region sub-image. Secondly, we use the graph-based model to propagate the relevance of labels. Finally, we calculate the relevance score according to the results of the relevance propagation on the two sparse graphs. And then the new order of tags is determined by the relevance of the tags from high to low. Compared to existing methods, the proposed method considers not only the relationship between the whole images, but also the relationship between the salient regions. Therefore, it enhances the accuracy of the tag ranking. Experimental results conducted on a real dataset demonstrate that the ranking result of the proposed approach is closer to the manual rank.

[1]  Haojie Li,et al.  Tag ranking by propagating relevance over tag and image graphs , 2012, ICIMCS '12.

[2]  De Xu,et al.  Beyond tag relevance: integrating visual attention model and multi-instance learning for tag saliency ranking , 2010, CIVR '10.

[3]  Marcel Worring,et al.  Learning tag relevance by neighbor voting for social image retrieval , 2008, MIR '08.

[4]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[5]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[6]  Tao Mei,et al.  Graph-based semi-supervised learning with multiple labels , 2009, J. Vis. Commun. Image Represent..

[7]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[8]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[9]  Zi Huang,et al.  Mining multi-tag association for image tagging , 2011, World Wide Web.

[10]  Steven C. H. Hoi,et al.  A two-view learning approach for image tag ranking , 2011, WSDM '11.

[11]  Zi Huang,et al.  Tag localization with spatial correlations and joint group sparsity , 2011, CVPR 2011.

[12]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Linda G. Shapiro,et al.  Computer Vision , 2001 .

[14]  Tao Mei,et al.  Joint multi-label multi-instance learning for image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[16]  Jeff Shrager,et al.  Observation of Phase Transitions in Spreading Activation Networks , 1987, Science.

[17]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[18]  Meng Wang,et al.  Visual query suggestion , 2009, ACM Multimedia.

[19]  Benjamin Naumann.  The Architecture Of Cognition , 2016 .

[20]  Jing Liu,et al.  Image annotation using multi-correlation probabilistic matrix factorization , 2010, ACM Multimedia.

[21]  Yi Yang,et al.  Effective transfer tagging from image to video , 2013, TOMCCAP.

[22]  Jing Liu,et al.  Unsupervised Feature Selection Using Nonnegative Spectral Analysis , 2012, AAAI.

[23]  Gang Hua,et al.  Descriptive visual words and visual phrases for image applications , 2009, ACM Multimedia.

[24]  Ramesh C. Jain,et al.  Image annotation by kNN-sparse graph-based label propagation over noisily tagged web images , 2011, TIST.

[25]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Sunil Arya,et al.  ANN: library for approximate nearest neighbor searching , 1998 .

[27]  Chong-Wah Ngo,et al.  Semantic context modeling with maximal margin Conditional Random Fields for automatic image annotation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[29]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .

[30]  D. Donoho.  For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[31]  Shuicheng Yan,et al.  Inferring semantic concepts from community-contributed images and noisy tags , 2009, ACM Multimedia.

[32]  Yi Yang,et al.  Interactive Video Indexing With Statistical Active Learning , 2012, IEEE Transactions on Multimedia.

[33]  Chong-Wah Ngo,et al.  A revisit of Generative Model for Automatic Image Annotation using Markov Random Fields , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Chee Sun Won,et al.  Efficient use of local edge histogram descriptor , 2000, MULTIMEDIA '00.