Word Co-occurrence and Markov Random Fields for Improving Automatic Image Annotation

In this paper a novel approach for improving automatic image annotation methods is proposed. The approach is based on the fact that accuracy of current image annotation methods is low if we look at the most confident label only. Instead, accuracy is improved if we look for the correct label within the set of the top k candidate labels. We take advantage of this fact and propose a Markov random field ( MRF) based on word co-occurrence information for the improvement of annotation systems. Through the MRF structure we take into account spatial dependencies between connected regions. As a result, we are considering semantic relationships between labels. We performed experiments with iterated conditional modes and simulated annealing as optimization strategies in a subset of the Corel benchmark collection. Experimental results of the proposed method together with a k nearest neighbors classifier as our annotation method show important error reductions.