Graph of visual words for semantic annotation of remote sensing images

Nowadays semantic image annotation is becoming more than ever a very challenging issue since it helps improving image interpretation and retrieval. Currently, most semantic annotation methods represent images as lists of keywords or histogram of visual words, and do not consider the spatial distribution of regions, nor any prior knowledge concerning objects in a scene. This obviously leads to weak and limited representation of image content. In this paper, we propose a new method for semantic image annotation that simultaneously handles all the available information of the image (contextual, spatial, and spectral). We use a remote sensing ontology as semantic resource and develop an annotation process producing a graph representing objects of a scene as well as their spatial relations.

[1]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[2]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[3]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[4]  Christos Faloutsos,et al.  GCap: Graph-based Automatic Image Captioning , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Nicolas Mellado,et al.  MCGraph: multi-criterion representation for scene understanding , 2014, SIGGRAPH ASIA Indoor Scene Understanding Where Graphics Meets Vision.

[6]  Arnold W. M. Smeulders,et al.  Visual synonyms for landmark image retrieval , 2012, Comput. Vis. Image Underst..

[7]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[8]  Anthony G. Cohn,et al.  Qualitative Spatial Representation and Reasoning with the Region Connection Calculus , 1997, GeoInformatica.

[9]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[10]  Marc Sebban,et al.  Supervised learning of Gaussian mixture models for visual vocabulary generation , 2012, Pattern Recognit..

[11]  I. Bloch,et al.  On the interest of spatial relations and fuzzy representations for ontology-based image interpretation , 2006 .

[12]  Chong Wang,et al.  Simultaneous image classification and annotation , 2009, CVPR.

[13]  G CohnAnthony,et al.  Qualitative Spatial Representation and Reasoning with the Region Connection Calculus , 1997 .

[14]  Zhiwu Lu,et al.  Image categorization with spatial mismatch kernels , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  R. Manmatha,et al.  Multiple Bernoulli relevance models for image and video annotation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Yong Jae Lee,et al.  Object-Graphs for Context-Aware Visual Category Discovery , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jianping Fan,et al.  Automatic image annotation by incorporating feature hierarchy and boosting to scale up SVM classifiers , 2006, MM '06.

[18]  Xiao Bai,et al.  Discriminative Features for Image Classification and Retrieval , 2011, 2011 Sixth International Conference on Image and Graphics.

[19]  Chong Wang,et al.  Simultaneous image classification and annotation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Max J. Egenhofer,et al.  Determining Semantic Similarity among Entity Classes from Different Ontologies , 2003, IEEE Trans. Knowl. Data Eng..

[21]  Chih-Fong Tsai,et al.  Bag-of-Words Representation in Image Annotation: A Review , 2012 .

[22]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  James Ze Wang,et al.  Real-Time Computerized Annotation of Pictures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  B. Luo,et al.  Image Auto-annotation with Graph Learning , 2010, 2010 International Conference on Artificial Intelligence and Computational Intelligence.

[25]  Vasant G Honavar,et al.  Annotating images and image objects using a hierarchical dirichlet process model , 2008, MDM '08.

[26]  Jonathon S. Hare,et al.  Efficient clustering and quantisation of SIFT features: exploiting characteristics of the SIFT descriptor and interest region detectors under image inversion , 2011, ICMR '11.

[27]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Daniel Gatica-Perez,et al.  PLSA-based image auto-annotation: constraining the latent space , 2004, MULTIMEDIA '04.

[29]  Fahad Shahbaz Khan,et al.  Discriminative compact pyramids for object and scene recognition , 2012, Pattern Recognition.

[30]  Z WangJames,et al.  Real-Time Computerized Annotation of Pictures , 2008 .

[31]  Nelson H. C. Yung,et al.  Feature fusion within local region using localized maximum-margin learning for scene categorization , 2012, Pattern Recognit..

[32]  Germain Forestier,et al.  Ontology-Based Object Recognition for Remote Sensing Image Interpretation , 2007, 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007).

[33]  Van Rijsbergen,et al.  A theoretical basis for the use of co-occurence data in information retrieval , 1977 .

[34]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[35]  Chin-Hui Lee,et al.  Bayesian Learning of Hierarchical Multinomial Mixture Models of Concepts for Automatic Image Annotation , 2006, CIVR.

[36]  Germain Forestier,et al.  Ontology-Based Object Recognition for Remote Sensing Image Interpretation , 2007 .

[37]  Gabriela Csurka,et al.  Images as sets of locally weighted features , 2012, Comput. Vis. Image Underst..

[38]  Angela Schwering,et al.  Measuring Semantic Similarity Between Geospatial Conceptual Regions , 2005, GeoS.

[39]  Alan F. Smeaton,et al.  Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval , 2003, SIGIR 2003.