Psychophysical Evaluation for a Qualitative Semantic Image Categorisation and Retrieval Approach

This paper details the behavioral evaluation of a qualitative image categorisation and retrieval approach using semantic features of images. Content based image retrieval and classification systems are highly active research areas and a cognitively plausible image description can improve effectiveness of such systems. While most approaches focus on low level image feature in order to classify images, humans, while certainly relying on some aspects of low level features, also apply high-level classifications. These high-level classification are often qualitative in nature and we have implemented a qualitative image categorisation and retrieval framework to account for human cognitive principles. While the dataset, i.e. the image database that was used for classification and retrieval purposes contained images that where annotated and therefore provided some ground truth for assessing the validity of the algorithm, we decided to add an additional behavioral evaluation step: Participants performed similarity ratings on a carefully chosen subset of picture implemented as a grouping task. Instead of using a predefined number of categories, participants could make their own choice on a) how many groups they thought were appropriate and b) which icons/images belong into these groups. The results show that the overall underlying conceptual structure created by the participants corresponds well to the classification provided through the algorithm.