Segmentation-based image retrieval

We have been developing an image retrieval system, called MIPS (multiscalar image processing and retrieval system), for use in uncontrolled environments. On insertion into the image database, the images are automatically segmented into homogeneous regions. Generic features are computed and stored for each segment. Specifically, we maintain not only geometric and photometric attributes but also simple spatial information for each extracted region. This approach asks the user to construct queries in terms of the given primitives i.e. regions and their spatial relations. Preliminary results show that the success of the system depends on how well the images can be modeled by homogeneous regions, on how useful the generic features are for the given application, and on the knowledge that the user puts into the formulation of the queries. A fully automatic segmentation algorithm is of paramount importance. We have designed an algorithm called perceptual region growing that combines region growing, edge detection, and perceptual organization principles, without resorting to any kind of high level knowledge or interactive user intervention. Decision thresholds and quality measures are directly derived from the image data, based on image statistics. Search through critical parameter spaces is the key idea to cope with noise in uncontrolled environments. The dynamics of the region growing process is constantly monitored and exploited.