Automatic Abstract Tag Detection for Social Image Tag Refinement and Enrichment

Collaborative image tagging systems, such as Flickr, are very attractive for supporting keyword-based image retrieval, but some user-provided tags of collaboratively-tagged social images might be imprecise. Some people may use general or high-level words (i.e., abstract tags) to tag their images for saving time and effort, but such general or high-level tags are too abstract to describe the visual content of social images precisely. As a result, users may not be able to find what they need when they use the specific keywords for query specification. To tackle the problem of abstract tags, an ontology with three-level semantics is constructed for detecting the candidates of abstract tags from large-scale social images. Then the image context (nearest neighbors) and tag context (most relevant tags) of social images with abstract tags are used to ultimately confirm whether these candidates are abstract or not and identify the specific tags which can further depict the images with abstract tags. In addition, all the relevant tags, which correspond with intermediate nodes between the abstract tags and specific tags on our concept ontology, are added to enrich the tags of social images so that users can have more choices to select various keywords for query specification. We have tested our proposed algorithms on two types of data sets (revised standard datasets and self-constructed dataset) and compared our approach with other approaches.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Qi Tian,et al.  What are the high-level concepts with small semantic gaps? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[4]  Alberto Del Bimbo,et al.  Enriching and localizing semantic tags in internet videos , 2011, ACM Multimedia.

[5]  Dong Liu,et al.  Image retagging , 2010, ACM Multimedia.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  B. S. Manjunath,et al.  Texture features and learning similarity , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[9]  Mor Naaman,et al.  How flickr helps us make sense of the world: context and content in community-contributed media collections , 2007, ACM Multimedia.

[10]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[11]  John R. Smith,et al.  Large-scale concept ontology for multimedia , 2006, IEEE MultiMedia.

[12]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[13]  Marcel Worring,et al.  Learning tag relevance by neighbor voting for social image retrieval , 2008, MIR '08.

[14]  B. S. Manjunath,et al.  An efficient color representation for image retrieval , 2001, IEEE Trans. Image Process..

[15]  Jianping Fan,et al.  Harvesting large-scale weakly-tagged image databases from the web , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[17]  Rong Jin,et al.  Multi-label learning with incomplete class assignments , 2011, CVPR 2011.

[18]  Azhar Rauf,et al.  Semantics in social tagging systems: A review , 2011, International Conference on Computer Networks and Information Technology.

[19]  Isabella Peters,et al.  Folksonomies - Indexing and Retrieval in Web 2.0 , 2009, Knowledge and Information.

[20]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[21]  Abebe Rorissa,et al.  User-generated descriptions of individual images versus labels of groups of images: A comparison using basic level theory , 2008, Inf. Process. Manag..

[22]  Dong Liu,et al.  Content-based tag processing for Internet social images , 2010, Multimedia Tools and Applications.

[23]  Shuicheng Yan,et al.  Inferring semantic concepts from community-contributed images and noisy tags , 2009, ACM Multimedia.

[24]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[25]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[26]  C. Fellbaum.  An Electronic Lexical Database , 1998 .

[27]  Tat-Seng Chua,et al.  NUS-WIDE: a real-world web image database from National University of Singapore , 2009, CIVR '09.

[28]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Meng Wang,et al.  Tag Tagging: Towards More Descriptive Keywords of Image Content , 2011, IEEE Transactions on Multimedia.

[30]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.

[31]  Shuicheng Yan,et al.  Image tag refinement towards low-rank, content-tag prior and error sparsity , 2010, ACM Multimedia.