Color- and texture-based image segmentation using EM and its application to content-based image retrieval

Retrieving images from large and varied collections using image content as a key is a challenging and important problem. In this paper we present a new image representation which provides a transformation from the raw pixel data to a small set of image regions which are coherent in color and texture space. This so-called "blobworld" representation is based on segmentation using the expectation-maximization algorithm on combined color and texture features. The texture features we use for the segmentation arise from a new approach to texture description and scale selection. We describe a system that uses the blobworld representation to retrieve images. An important and unique aspect of the system is that, in the context of similarity-based querying, the user is allowed to view the internal representation of the submitted image and the query results. Similar systems do not offer the user this view into the workings of the system; consequently, the outcome of many queries on these systems can be quite inexplicable, despite the availability of knobs for adjusting the similarity metric.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[3]  Josef Bigün Local symmetry features in image processing , 1988 .

[4]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms , 1990 .

[5]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[7]  David Yarowsky,et al.  Word-Sense Disambiguation Using Statistical Models of Roget’s Categories Trained on Large Corpora , 2010, COLING.

[8]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[9]  Wolfgang Förstner,et al.  A Framework for Low Level Feature Extraction , 1994, ECCV.

[10]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[11]  Martial Hebert,et al.  Object Representation in Computer Vision , 1994, Lecture Notes in Computer Science.

[12]  Don R. Hush,et al.  Query by image example: The CANDID approach , 1995 .

[13]  T. M. Cannon,et al.  Query by image example: the comparison algorithm for navigating digital image databases (CANDID) approach , 1995, Electronic imaging.

[14]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[15]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  W. Eric L. Grimson,et al.  Adaptive Segmentation of MRI Data , 1995, CVRMed.

[17]  Raghu Ramakrishnan,et al.  Data Modeling and Querying in the PIQ Image DBMS. , 1996 .

[18]  Jitendra Malik,et al.  Detecting, localizing and grouping repeated scene elements from an image , 1996, ECCV.

[19]  Edward H. Adelson,et al.  A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Martial Hebert,et al.  Object Representation in Computer Vision II , 1996, Lecture Notes in Computer Science.

[21]  D. Forsyth,et al.  Searching for Digital Pictures , 1997 .

[22]  Amarnath Gupta,et al.  Visual information retrieval , 1997, CACM.

[23]  W. Eric L. Grimson,et al.  Configuration based scene classification and image indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Joachim M. Buhmann,et al.  Deterministic Annealing for Unsupervised Texture Segmentation , 1997, EMMCVPR.