Perceptual Feature Selection for Semantic Image Classification

Content-based image retrieval has become an indispensable tool for managing the rapidly growing collections of digital images. The goal is to organize the contents semantically, according to meaningful categories. In recent papers we introduced a new approach for semantic image classification that relies on the adaptive perceptual color-texture segmentation algorithm proposed by Chen et al. This algorithm combines knowledge of human perception and signal characteristics to segment natural scenes into perceptually uniform regions. The resulting segments can be classified into semantic categories using region-wide features as medium level descriptors. Such descriptors are the key to bridging the gap between low-level image primitives and high-level image semantics. The segment classification is based on linear discriminant analysis techniques. In this paper, we examine the classification performance (precision and recall rates) when different sets of region-wide features are used. These include different color composition features, spatial texture, and segment location. We demonstrate the effectiveness of the proposed techniques on a database that includes 9000 segments from approximately 2500 photographs of natural scenes.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Aleksandra Mojsilovic,et al.  A computational model for color naming and describing color composition of images , 2005, IEEE Transactions on Image Processing.

[3]  Aleksandra Mojsilovic,et al.  Semantic metric for image library exploration , 2004, IEEE Transactions on Multimedia.

[4]  Aleksandra Mojsilovic,et al.  Adaptive perceptual color-texture image segmentation , 2005, IEEE Transactions on Image Processing.

[5]  Thrasyvoulos N. Pappas,et al.  Experimental determination of visual color and texture statistics for image segmentation , 2005, IS&T/SPIE Electronic Imaging.

[6]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Christos Faloutsos,et al.  Automatic image captioning , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[9]  Aidong Zhang,et al.  Semantics Retrieval by Content and Context of Image Regions , 2002 .

[10]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[11]  Robert M. Boynton,et al.  Eleven Colors That Are Almost Never Confused , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[12]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[13]  Thrasyvoulos N. Pappas,et al.  Perceptually based techniques for semantic image classification and retrieval , 2006, Electronic Imaging.

[14]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Thrasyvoulos N. Pappas,et al.  A PERCEPTUAL APPROACH FOR SEMANTIC IMAGE RETRIEVAL , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[17]  Nuggehally Sampath Jayant,et al.  An adaptive clustering algorithm for image segmentation , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[18]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[19]  Lei Zhu,et al.  Keyblock: an approach for content-based image retrieval , 2000, ACM Multimedia.