Natural Scene Retrieval Based on a Semantic Modeling Step

In this paper, we present an approach for the retrieval of natural scenes based on a semantic modeling step. Semantic modeling stands for the classification of local image regions into semantic classes such as grass, rocks or foliage and the subsequent summary of this information in so-called concept-occurrence vectors. Using this semantic representation, images from the scene categories coasts, rivers/lakes, forests, plains, mountains and sky/clouds are retrieved. We compare two implementations of the method quantitatively on a visually diverse database of natural scenes. In addition, the semantic modeling approach is compared to retrieval based on low-level features computed directly on the image. The experiments show that semantic modeling leads in fact to better retrieval performance.

[1]  B. Tversky,et al.  Categories of environmental scenes , 1983, Cognitive Psychology.

[2]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[3]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[4]  Oded Maron,et al.  Multiple-Instance Learning for Natural Scene Classification , 1998, ICML.

[5]  Charles A. Bouman,et al.  Perceptual image similarity experiments , 1998, Electronic Imaging.

[6]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[8]  Milind R. Naphade,et al.  A probabilistic framework for semantic video indexing, filtering, and retrieval , 2001, IEEE Trans. Multim..

[9]  Nando de Freitas,et al.  Object Recognition as Machine Translation – Part 2: Exploiting Image Database Clustering Models , 2001 .

[10]  Nicu Sebe,et al.  The State of the Art in Image and Video Retrieval , 2003, CIVR.

[11]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[12]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.