Assessment of Effectiveness of Content Based Image Retrieval Systems

Performance evaluation is an often neglected topic in content based image retrieval research. Commonly used evaluation measures for effectiveness assessment are reviewed and their shortcomings are discussed. A new evaluation measure is proposed which overcomes some of the shortcomings of existing evaluation measures. The presented performance measure is especially suited for rank based retrieval methods. The measure is obtained fully automatic and can be exploit to investigate different criteria of effectiveness in detail. For better comparison of performance differences, the measure provides a confidence and credibility measure, too. Experiments were carried out to study the properties of the evaluation method.