Automatic image annotation with weakly labeled dataset

It is very attractive to exploit weakly-labeled image dataset for multi-label annotation applications. In our paper the meaning of the terminology weakly labeled is threefold: i) only a small subset of the available images are labeled; ii) even for the labeled image, the given labels may be uncorrect or incomplete; iii) the given labels do not provide the exact object locations in the images. A novel method is developed to predict the multiple labels for images and to provide region-level labels for the objects. We cluster the image regions to learn several region-exemplars and predict the label vector for each image region as a locally weighted average of the label vectors on exemplars. By investigating the label confidence matrix for the region-exemplars from different perspectives (column picture and row picture), we sufficiently leverage the visual contexts, the semantic contexts, and the consistency between similarities in the visual feature space and semantic label space. Experimental results on real web images demonstrate the effectiveness of the proposed method.