A new performance measure for image retrieval algorithms

A new performance measure for evaluating image retrieval algorithms, called maximum ratio measure, is introduced to calculate the matching rate from a brand-new view. It is better than the existing ones, as it is more consistent with the retrieval abilities of the considered algorithms. It judges whether a query image belongs to a category according to the ratio of the total number of images reported in the matching result from the same class to Q, which is the size of the considered category. If the output of the algorithm is the same as the category, which the query image comes from, the query is successful and produces a mark of 1; otherwise, it produces a mark of 0. The average matching rate is equal to the ratio of total marks to the times of query. A novel strategy is also suggested to improve the overall performance of a retrieval algorithm by adjusting the parameters of the algorithm.

[1]  Atul Sajjanhar,et al.  An Experimental Study of Moment Invariants and Fourier Descriptors for Shape Based Image Retrieval , 1997 .

[2]  Ulrich Eckhardt,et al.  Shape descriptors for non-rigid shapes with a single closed contour , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Guojun Lu,et al.  On Performance Measurement of Multimedia Information Retrieval Systems , 1998 .

[4]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[5]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[6]  Thierry Pun,et al.  A Framework for Benchmarking in CBIR , 2003, Multimedia Tools and Applications.

[7]  Brian Scassellati,et al.  Retrieving images by 2D shape: a comparison of computation methods with human perceptual judgments , 1994, Electronic Imaging.

[8]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, CVPR.

[9]  Ellen M. Voorhees,et al.  Retrieval evaluation with incomplete information , 2004, SIGIR '04.

[10]  Alberto Del Bimbo,et al.  Visual Image Retrieval by Elastic Matching of User Sketches , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Gerard Salton,et al.  The State of Retrieval System Evaluation , 1992, Inf. Process. Manag..

[12]  Francoise J. Preteux,et al.  3D-shape-based retrieval within the MPEG-7 framework , 2001, IS&T/SPIE Electronic Imaging.

[13]  Erkki Oja,et al.  Evaluating the Performance of Content-Based Image Retrieval Systems , 2000, VISUAL.

[14]  George Hripcsak,et al.  Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval , 2005, J. Am. Medical Informatics Assoc..

[15]  Neil J. Gunther,et al.  Benchmark for image retrieval using distributed systems over the Iinternet: BIRDS-I , 2000, IS&T/SPIE Electronic Imaging.

[16]  Wenhe Liao,et al.  Direct Curvature Scale Space: Theory and Corner Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  C. J. van Rijsbergen,et al.  FOUNDATION OF EVALUATION , 1974 .

[18]  Miroslaw Bober,et al.  Curvature Scale Space Representation: Theory, Applications, and MPEG-7 Standardization , 2011, Computational Imaging and Vision.