An empirical study of automatic image annotation through Multi-Instance Multi-Label Learning

Although many region based models for image auto-annotation have been proposed recently, their performances are not satisfactory due to the sensitivity to segmentation errors. In this paper, by evaluating two image partition methods and four visual features, we propose a new ensemble method under Multi-Instance Multi-Label (MIML) learning framework which has been proposed recently. The ensemble method combines all the outputs of these separate learning machines trained on different features. The experimental results over Corel images show that the ensemble method is efficient for image auto-annotation and comparable with other methods. In addition, the results show that the region-based image segmentation approach significantly improves the performance of the proposed model.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Y. Mori,et al.  Image-to-word transformation based on dividing and vector quantizing images with words , 1999 .

[3]  Yang Yu,et al.  Automatic image annotation using group sparsity , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[5]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  Farshad Fotouhi,et al.  Region based image annotation through multiple-instance learning , 2005, MULTIMEDIA '05.

[8]  Zhi-Hua Zhou,et al.  Multi-Instance Multi-Label Learning with Application to Scene Classification , 2006, NIPS.

[9]  Jitendra Malik,et al.  From contours to regions: An empirical evaluation , 2009, CVPR.

[10]  Chong Wang,et al.  Simultaneous image classification and annotation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Thomas Gärtner,et al.  Multi-Instance Kernels , 2002, ICML.

[12]  Thomas Hofmann,et al.  Multi-Instance Multi-Label Learning with Application to Scene Classification , 2007 .

[13]  Gertjan J. Burghouts,et al.  Performance evaluation of local colour invariants , 2009, Comput. Vis. Image Underst..