Image Classification and Querying Using Composite Region Templates

The tremendous growth in digital imagery is driving the need for more sophisticated methods for automatic image analysis, cataloging, and searching. We present a method for classifying and querying images based on the spatial orderings of regions or objects using composite region templates (CRTs). The CRTs capture the spatial information statistically and provide a robust way to measure similarity in the presence of region insertions, deletions, substitutions, replications, and relocations. The CRTs can be used for classifying and annotating images by assigning symbols to the regions or objects and by extracting symbol strings from spatial scans of the images. The symbol strings can be decoded using a library of annotated CRTs to automatically label and classify the images. The CRTs can also be used for searching by sketch or example by measuring image similarity based on relative counts of the CRTs.

[1]  Shih-Fu Chang,et al.  Local color and texture extraction and spatial query , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[2]  A. Del Bimbo,et al.  Using weighted spatial relationships in retrieval by visual contents , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[3]  Shih-Fu Chang,et al.  Transform features for texture classification and discrimination in large image databases , 1994, Proceedings of 1st International Conference on Image Processing.

[4]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[5]  Yiqing Liang,et al.  A digital video library on the World Wide Web , 1997, MULTIMEDIA '96.

[6]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[7]  Su-Shing Chen.  Content-Based Indexing of Spatial Objects in Digital Libraries , 1996, J. Vis. Commun. Image Represent..

[8]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[9]  E. Vicario,et al.  Using weighted spatial relationships in retrieval by visual contents , 1998 .

[10]  J.R. Smith,et al.  Decoding image semantics using composite region templates , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[11]  S. Sclaroff,et al.  ImageRover: a content-based image browser for the World Wide Web , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[12]  Terry Caelli,et al.  On the classification of image regions by colour, texture and shape , 1993, Pattern Recognit..

[13]  Serge J. Belongie,et al.  Region-based image querying , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[14]  Gérard G. Medioni,et al.  Finding Waldo, or focus of attention using local color information , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[16]  Shih-Fu Chang,et al.  Visually Searching the Web for Content , 1997, IEEE Multim..

[17]  Suh-Yin Lee,et al.  Retrieval of similar pictures on pictorial databases , 1991, Pattern Recognit..

[18]  Tat-Seng Chua,et al.  Content-based retrieval of segmented images , 1994, MULTIMEDIA '94.

[19]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Electronic Imaging.

[20]  Hayit Greenspan,et al.  Finding Pictures of Objects in Large Collections of Images , 1996, Object Representation in Computer Vision.

[21]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[22]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[23]  Shi-Kuo Chang,et al.  Iconic Indexing by 2-D Strings , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Vijay V. Raghavan,et al.  Design and evaluation of algorithms for image retrieval by spatial similarity , 1995, TOIS.

[25]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[26]  S. Tominaga.  Color classification of natural color images , 1992 .

[27]  Markus A. Stricker,et al.  Color indexing with weak spatial constraints , 1996, Electronic Imaging.

[28]  John R. Smith,et al.  Content-based transcoding of images in the Internet , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[29]  John R. Smith,et al.  Transcoding Internet content for heterogeneous client devices , 1998, ISCAS '98. Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (Cat. No.98CH36187).

[30]  Shih-Fu Chang,et al.  Integrated spatial and feature image query , 1999, Multimedia Systems.

[31]  C. Frankel,et al.  Distinguishing photographs and graphics on the World Wide Web , 1997, 1997 Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries.

[32]  Joshua R. Smith,et al.  Digital video libraries and the Internet , 1999, IEEE Commun. Mag..