CEA LIST's Participation to Visual Concept Detection Task of ImageCLEF 2011

This paper describes the CEA LIST participation in the ImageCLEF 2011 Photo Annotation challenge. This year, our motivation was to investigate the annotation performance by using provided Flickr-tags as additionnal infor- mation. First, we present an overview of our local and global visual features used in this work. Second, we present a new method, that we call "Fuzzy-tfidf", which takes into account the uncertainty of user tags. Our textual descriptor is based on semantic similarity between tags and visual concepts. To compute this similarity, we used two distances: the first one is based on Wordnet ontology and the second is based on social networks. We perform a late fusion to combine scores from visual and textual modalities. Our best model, a late fusion trained on global visual features and user tags, obtains 38.3 % MAP, almost a 8 % MAP absolute improvement compared to our best visual-only system. The results show that the combination of Flickr-tags with visual features improves the results of the run using only visual features. It corroborates the importance of taking into account the uncertainty of user tags and the complementarity between visual and textual modalities.