High-Level Features for Image Indexing and Retrieval

Image indexing for content-based image retrieval is the process of automatically computing a compact representation (numerical or alphanumerical) of some attribute of digital images, to be used to derive information about the image contents. A feature, or attribute, can be related to a visual characteristic, but it may also be related to an interpretative response to an image or to a spatial, symbolic, semantic, or emotional characteristic. A feature may relate to a single attribute or be a composite representation of different attributes. Features can be classified as general purpose or domain-dependent. The general purpose features can be used in any context, while the domain-dependent features are designed specifically for a given application. Every feature is intimately tied with the kind of information that it captures. The choice of a particular feature over another depends on the given application, and the kind (level) of information required. While more complex and sophisticated general purpose and domain dependent features were being developed, two important issues became evident: the sensory gap and the semantic gap. The sensory gap is the gap between the information of the real world, and the information in a computational description derived from a digital recording of a scene of the world (i.e. low-level features or visual features such as color, texture, etc.). The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given application. As stated by Smeulders et al. (2000), the gap between the pictorial features and the image’s semantics makes it difficult for purely low-level content-based retrieval systems to obtain satisfactory results. High-level features try to bridge the semantic gap by embedding in their representation information about the image content. This goal can be pursued by different means from manually annotating the images with texts to exploiting pattern recognition, computer vision and machine learning algorithms.