Supervised Semantic Image Annotation Using Region Relevance

Abstract Automatic image annotation is the key to semantic-based image retrieval. In this paper we formulate image annotation as a multi-class classification problem, which deals with the weak annotation problem and works with image-level ground truth training data. The relationship between low-level visual features and semantic concepts is found by supervised Bayesian learning. For each region in the test image, a posterior probability for each concept is calculated from class densities estimated from the training set and then the probability is modified using relevance with the other regions in the image. The image-level posterior probabilities are obtained by combining the regional posterior probabilities and keywords are selected according to their ranks. The proposed algorithm achieves good annotation performance on the Corel5K benchmark dataset.

[1]  Hichem Frigui,et al.  Unsupervised learning of prototypes and attribute weights , 2004, Pattern Recognit..

[2]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[3]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[4]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[6]  Farshad Fotouhi,et al.  Image content annotation using Bayesian framework and complement components analysis , 2005, IEEE International Conference on Image Processing 2005.

[7]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[8]  Luc Van Gool,et al.  Modeling scenes with local descriptors and latent aspects , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Raimondo Schettini,et al.  Image annotation using SVM , 2003, IS&T/SPIE Electronic Imaging.

[11]  Zheru Chi,et al.  Annotating Image Regions Using Spatial Context , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[12]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[13]  Jianping Fan,et al.  Multi-level annotation of natural scenes using dominant image components and semantic concepts , 2004, MULTIMEDIA '04.

[14]  Keiji Yanai,et al.  Probabilistic web image gathering , 2005, MIR '05.

[15]  Guojun Lu,et al.  Content-based Image Retrieval Using Gabor Texture Features , 2000 .

[16]  David G. Stork,et al.  Pattern Classification , 1973 .

[17]  Ying Liu,et al.  Region-based image retrieval with high-level semantics using decision tree learning , 2008, Pattern Recognit..

[18]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.