Quantitative evaluation of rank-order similarity of images

Region importance maps from image understanding algorithms and human observer studies are ordered rankings of the pixel locations. Kemeny and Snell's distance (d/sub KS/), an existing measure from ordinal ranking theory, can thus be used as a similarity measure between images. We address three problems with d/sub KS/: its high computational cost, its bias in favor of images with sparse histograms, and its image-size dependent range of values. We present a novel computationally efficient algorithm for computing d/sub KS/ between two images, and we derive a normalized form d/sub KS/ with no bias whose range is independent of image size. For evaluating an algorithm where the reference data and algorithm output are ordered rankings of pixels, d/sub KS/ is subjectively superior to the correlation coefficient as a figure of merit.