Hierarchical Markov Random Fields with Irregular Pyramids for Improving Image Annotation

Image segmentation and Automatic Image Annotation (AIA) are two important areas that still impose challenging problems. Addressing both problems simultaneously may improve their results since they are interdependent. In this paper we give a step ahead in that direction considering different segmentation levels simultaneously and possible contextual relations among segments in order to improve the automatic image annotation. We propose to include hierarchical relations among regions of an image in a Markov Random Field (MRF) model for annotation. This relations are obtained from irregular pyramids, which keep parent-child relations among regions through all the levels. Our main contribution is therefore the combination of the irregular pyramid approach with context modeling by means of hierarchical MRFs. Experiments run in a subset of the Corel image collection showed a relevant improvement in the annotation accuracy.

[1]  Peter Carbonetto.  Unsupervised Statistical Models for General Object Recognition , 2003 .

[2]  Hugo Jair Escalante,et al.  Word Co-occurrence and Markov Random Fields for Improving Automatic Image Annotation , 2007, BMVC.

[3]  Yll Haxhimusa,et al.  Segmentation Graph Hierarchies , 2004, SSPR/SPR.

[4]  Luis Rueda,et al.  Advances in Image and Video Technology, Second Pacific Rim Symposium, PSIVT 2007, Santiago, Chile, December 17-19, 2007, Proceedings , 2007, PSIVT.

[5]  R. Manmatha,et al.  Image retrieval using Markov Random Fields and global image features , 2010, CIVR '10.

[6]  Annette Morales-González,et al.  Simple object recognition based on spatial relations and visual features represented using irregular pyramids , 2013, Multimedia Tools and Applications.

[7]  Edwin R. Hancock,et al.  Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshop, SSPR&SPR 2010, Cesme, Izmir, Turkey, August 18-20, 2010. Proceedings , 2010, SSPR/SPR.

[8]  Sang Uk Lee,et al.  New MRF Parameter Estimation Technique for Texture Image Segmentation using Hierarchical GMRF Model Based on Random Spatial Interaction and Mean Field Theory , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[9]  Yang Cao,et al.  Image denoising based on hierarchical Markov random field , 2011, Pattern Recognit. Lett..

[10]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Atsushi Imiya,et al.  Digital and Image Geometry: Advanced Lectures , 2002 .

[12]  Thomas Brox,et al.  Hierarchical Markov Random Fields for mast cell segmentation in electron microscopic recordings , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[13]  Carlos Arturo Hernández-Gracidas,et al.  Markov Random Fields and Spatial Information to Improve Automatic Image Annotation , 2007, PSIVT.

[14]  Luc Brun,et al.  Introduction to Combinatorial Pyramids , 2000, Digital and Image Geometry.

[15]  Luc Brun,et al.  Contains and inside relationships within combinatorial pyramids , 2006, Pattern Recognit..

[16]  Chong-Wah Ngo,et al.  A revisit of Generative Model for Automatic Image Annotation using Markov Random Fields , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Atsushi Imiya,et al.  Digital and Image Geometry , 2002, Lecture Notes in Computer Science.

[18]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.