Perceptual image similarity experiments

In this paper, we study how human observers judge image similarity. To do so, we have conducted two psychophysical scaling experiments and have compared the results to two algorithmic image similarity metrics. For these experiments, we selected a set of 97 digitized photographic images which represent a range of semantic categories, viewing distances, and colors. We then used the two perceptual and the two algorithmic methods to measure the similarity of each image to every other image in the data set, producing four similarity matrices. These matrices were analyzed using multidimensional scaling techniques to gain insight into the dimensions human observers use for judging image similarity, and how these dimensions differ from the results of algorithmic methods. This paper also describes and validates a new technique for collecting similarity judgments which can provide meaningful results with a factor of four fewer judgments, as compared with the paired comparisons method.

[1]  Allen McPherson,et al.  Interactive layout mechanisms for image database retrieval , 1996, Electronic Imaging.

[2]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[3]  Dragutin Petkovic,et al.  Content-Based Representation and Retrieval of Visual Media: A State-of-the-Art Review , 1996 .

[4]  Jan P. Allebach,et al.  Methodology for designing image similarity metrics based on human visual system models , 1997, Electronic Imaging.

[5]  C. Krumhansl.  The psychological representation of musical pitch in a tonal context , 1979, Cognitive Psychology.

[6]  Jordi Vitrià,et al.  A multidimensional scaling approach to explore the behavior of a texture perception algorithm , 1997, Machine Vision and Applications.

[7]  A. Ravishankar Rao,et al.  Identifying High Level Features of Texture Perception , 1993, CVGIP Graph. Model. Image Process..

[8]  C. Tomasi.  The Earth Mover's Distance, Multi-Dimensional Scaling, and Color-Based Image Retrieval , 1997 .

[9]  John D. Mollon,et al.  Difficulties faced by color-anomalous observers in interpreting color displays , 1990, Electronic Imaging.

[10]  Jan P. Allebach,et al.  Multiscale branch-and-bound image database search , 1997, Electronic Imaging.

[11]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[12]  Brian Scassellati,et al.  Retrieving images by 2D shape: a comparison of computation methods with human perceptual judgments , 1994, Electronic Imaging.