QBIC project: querying images by content, using color, texture, and shape

In the query by image content (QBIC) project we are studying methods to query large on-line image databases using the images' content as the basis of the queries. Examples of the content we use include color, texture, and shape of image objects and regions. Potential applications include medical (`Give me other images that contain a tumor with a texture like this one'), photo-journalism (`Give me images that have blue at the top and red at the bottom'), and many others in art, fashion, cataloging, retailing, and industry. Key issues include derivation and computation of attributes of images and objects that provide useful query functionality, retrieval methods based on similarity as opposed to exact match, query by image example or user drawn image, the user interfaces, query refinement and navigation, high dimensional database indexing, and automatic and semi-automatic database population. We currently have a prototype system written in X/Motif and C running on an RS/6000 that allows a variety of queries, and a test database of over 1000 images and 1000 objects populated from commercially available photo clip art images. In this paper we present the main algorithms for color texture, shape and sketch query that we use, show example query results, and discuss future directions.

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  Kenneth Steiglitz,et al.  Operations on Images Using Quad Trees , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jürg Nievergelt,et al.  The Grid File: An Adaptable, Symmetric Multikey File Structure , 1984, TODS.

[5]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[6]  Makoto Miyahara,et al.  Mathematical Transform Of (R, G, B) Color Data To Munsell (H, V, C) Color Data , 1988, Other Conferences.

[7]  Hanan Samet,et al.  The Design and Analysis of Spatial Data Structures , 1989 .

[8]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[9]  H. V. Jagadish.  Spatial search with polyhedra , 1990, [1990] Proceedings. Sixth International Conference on Data Engineering.

[10]  Toshikazu Kato,et al.  Intelligent visual interaction with image database systems-toward the multimedia personal interface , 1991 .

[11]  David Mumford,et al.  Mathematical theories of shape: do they model perception? , 1991, Optics & Photonics.

[12]  Hamid Pirahesh,et al.  Extensions to Starburst: objects, types, functions, and rules , 1991, CACM.

[13]  David B. Cooper,et al.  Recognition and positioning of rigid objects using algebraic moment invariants , 1991, Optics & Photonics.

[14]  .. Gevers,et al.  Nigma: an Image Retrieval System , 1992 .

[15]  Mary C. Dyson.  How do you describe a symbol?: The problems involved in retrieving symbols from a database , 1992 .

[16]  Peiya Liu,et al.  Content-based indexing technique using relative geometry features , 1992, Electronic Imaging.

[17]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[18]  A.W.M. Smeulders,et al.  Enigma: an image retrieval system , 1992 .

[19]  Freddy Fierens,et al.  Interactive outlining: an improved approach using active contours , 1993, Electronic Imaging.