A Groundtruth Database for Testing Objective Metrics for Image Difference

An objective metric that can predict the perceived difference of a processed image from an original would be very useful in optimizing image-processing algorithms (e.g., JPEG compression). Ideally, such a metric must be validated against human data before it is used in real imaging applications. There have been numerous efforts to develop such metrics and a few attempts to validate these metrics, but none of the previous validation work intended to cover a wide range of image differences. As a result, the metric developed may not be useful for general purposes. In the present study, we developed a comprehensive database of images and psychophysical data and used it as a tool to test various models of image difference. Several image manipulations were performed to introduce image differences of different types (such as density shift, JPEG compression, and image blur). A psychophysical study was performed to obtain subjective evaluations of image differences from ten observers. As a first step, we tested CIE 2000 and S-CIELAB models against the database. Our results indicate that simple models, such as the CIE 2000 color difference model, can predict density shift and image blur well, but models that incorporate spatial components (such as S-CIELAB) are better in predicting the results of JPEG compression.