Semantic Annotation of Image Groups with Self-organizing Maps

Automatic image annotation has attracted a lot of attention recently as a method for facilitating semantic indexing and text-based retrieval of visual content. In this paper, we propose the use of multiple Self-Organizing Maps in modeling various semantic concepts and annotating new input images automatically. The effect of the semantic gap is compensated by annotating multiple images concurrently, thus enabling more accurate estimation of the semantic concepts’ distributions. The presented method is applied to annotating images from a freely-available database consisting of images of different semantic categories.

[1]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[2]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[3]  Theo Gevers,et al.  Integrating Visual and Textual Cues for Image Classification , 2000, VISUAL.

[4]  Erkki Oja,et al.  Class distributions on SOM surfaces for feature extraction and object retrieval , 2004, Neural Networks.

[5]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[6]  Wei-Ying Ma,et al.  Image and Video Retrieval , 2003, Lecture Notes in Computer Science.

[7]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[8]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[10]  Clement H. C. Leung,et al.  Advances in Visual Information Systems, 9th International Conference, VISUAL 2007, Shanghai, China, June 28-29, 2007 Revised Selected Papers , 2007, VISUAL.

[11]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[12]  Erkki Oja,et al.  PicSOM-self-organizing image retrieval with MPEG-7 content descriptors , 2002, IEEE Trans. Neural Networks.

[13]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[14]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[15]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[16]  Erkki Oja,et al.  Use of Image Subset Features in Image Retrieval with Self-Organizing Maps , 2004, CIVR.