Automatic Image Annotation Using a Semi-supervised Ensemble of Classifiers

Automatic image annotation consists on automatically labeling images, or image regions, with a pre-defined set of keywords, which are regarded as descriptors of the high-level semantics of the image. In supervised learning, a set of previously annotated images is required to train a classifier. Annotating a large quantity of images by hand is a tedious and time consuming process; so an alternative approach is to label manually a small subset of images, using the other ones under a semi-supervised approach. In this paper, a new semi-supervised ensemble of classifiers, called WSA, for automatic image annotation is proposed. WSA uses naive Bayes as its base classifier. A set of these is combined in a cascade based on the AdaBoost technique. However, when training the ensemble of Bayesian classifiers, it also considers the unlabeled images on each stage. These are annotated based on the classifier from the previous stage, and then used to train the next classifier. The unlabeled instances are weighted according to a confidence measure based on their predicted probability value; while the labeled instances are weighted according to the classifier error, as in standard AdaBoost. WSA has been evaluated with benchmark data sets, and 2 sets of images, with promising results.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[3]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Robert M. Haralick,et al.  Textural features for image database retrieval , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[5]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  Guojun Lu,et al.  Content-based Image Retrieval Using Gabor Texture Features , 2000 .

[8]  Ian Witten,et al.  Data Mining , 2000 .

[9]  Ludmila I. Kuncheva,et al.  Using measures of similarity and inclusion for multiple classifier fusion by decision templates , 2001, Fuzzy Sets Syst..

[10]  Ayhan Demiriz,et al.  Exploiting unlabeled data in ensemble methods , 2002, KDD.

[11]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[12]  Maosong Sun,et al.  Automatic Image Annotation Based on WordNet and Hierarchical Ensembles , 2006, CICLing.

[13]  Jiayu Tang,et al.  Image Auto-Annotation using a Statistical Model with Salient Regions , 2006, 2006 IEEE International Conference on Multimedia and Expo.