Markov Random Fields and Spatial Information to Improve Automatic Image Annotation

Content-based image retrieval (CBIR) is currently limited because of the lack of representational power of the low-level image features, which fail to properly represent the actual contents of an image, and consequently poor results are achieved with the use of this sole information. Spatial relations represent a class of high-level image features which can improve image annotation. We apply spatial relations to automatic image annotation, a task which is usually a first step towards CBIR. We follow a probabilistic approach to represent different types of spatial relations to improve the automatic annotations which are obtained based on low-level features. Different configurations and subsets of the computed spatial relations were used to perform experiments on a database of landscape images. Results show a noticeable improvement of almost 9% compared to the base results obtained using the k-Nearest Neighbor classifier.

[1]  Peter Carbonetto.  Unsupervised Statistical Models for General Object Recognition , 2003 .

[2]  Maneesha Singh,et al.  IMAGE RETRIEVAL USING SPATIAL CONTEXT , 2002 .

[3]  C. Preston.  Gibbs States on Countable Sets , 1974 .

[4]  Jayant Sharma,et al.  A Critical Comparison of the 4-Intersection and 9-Intersection Models for Spatial Relations: Formal Analysis* , 2003 .

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[7]  Stephen S.-T. Yau,et al.  On Interactability of Spatial Relationships in Content-Based Image Database Systems , 2004, Commun. Inf. Syst..

[8]  C. Gold,et al.  DESCRIBING TOPOLOGICAL RELATIONS WITH VORONOI-BASED 9-INTERSECTION MODEL , 2003 .

[9]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[10]  Marcel Worring,et al.  Adding Spatial Semantics to Image Annotations , 2004, LSTKM@EKAW.

[11]  Arun K. Majumdar,et al.  Content Based Image Search over the World Wide Web , 2002, ICVGIP.

[12]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Hugo Jair Escalante,et al.  Word Co-occurrence and Markov Random Fields for Improving Automatic Image Annotation , 2007, BMVC.

[14]  Aaron C. Courville,et al.  Interacting Markov Random Fields for Simultaneous Terrain Modeling and Obstacle Detection , 2005, Robotics: Science and Systems.

[15]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[16]  Clement T. Yu,et al.  Reasoning About Spatial Relationships in Picture Retrieval Systems , 1994, VLDB.

[17]  Qing-Long Zhang,et al.  On consistency checking of spatial relationships in content-based image database systems , 2005, Commun. Inf. Syst..

[18]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .