Image Quality Assessment: From Error Measurement to Structural Similarity

Objective methods for assessing perceptual im- age quality traditionally attempt to quantify the visibility of errors (dierences) between a distorted image and a ref- erence image using a variety of known properties of the hu- man visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative com- plementary framework for quality assessment based on the degradation of structural information. As a specific exam- ple of this concept, we develop a Structural Similarity Index and demonstrate its promise through a set of intuitive ex- amples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MatLab imple- mentation of the proposed algorithm is available online at http://www.cns.nyu.edu/~lcv/ssim/.

[1]  Andrew P. Bradley,et al.  Perceptual quality metrics applied to still image compression , 1998, Signal Process..

[2]  Zhou Wang,et al.  Embedded foveation image coding , 2001, IEEE Trans. Image Process..

[3]  Olivier Verscheure,et al.  Perceptual quality measure using a spatiotemporal model of the human visual system , 1996, Electronic Imaging.

[4]  Alan C. Bovik,et al.  41 OBJECTIVE VIDEO QUALITY ASSESSMENT , 2003 .

[5]  Patrick C. Teo,et al.  A model of perceptual image fidelity , 1995, Proceedings., International Conference on Image Processing.

[6]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[7]  A. Bovik,et al.  OBJECTIVE VIDEO QUALITY ASSESSMENT , 2003 .

[8]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[9]  Pascual Capilla,et al.  Image quality metric based on multidimensional contrast perception models , 1999 .

[10]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[11]  Jesús Malo,et al.  Linear transform for simultaneous diagonalization of covariance and perceptual metric matrix in image coding , 2003, Pattern Recognit..

[12]  Jean-Bernard Martens,et al.  Image dissimilarity , 1998, Signal Process..

[13]  Jing Xing,et al.  P‐14: An Image Processing Model of Contrast Perception and Discrimination of the Human Visual System , 2002 .

[14]  Jerome R. Cox,et al.  Experimental evaluation of psychophysical distortion metrics for JPEG-encoded images , 1995, J. Electronic Imaging.

[15]  Francesc J. Ferri,et al.  Non-linear Invertible Representation for Joint Statistical and Perceptual Feature Decorrelation , 2000, SSPR/SPR.

[16]  Geoffrey M. Boynton,et al.  New model of human luminance pattern vision mechanisms: analysis of the effects of pattern orientation, spatial phase, and temporal frequency , 1994, Other Conferences.

[17]  Jeffrey Lubin,et al.  The use of psychophysical data and models in the analysis of display system performance , 1993 .

[18]  Andrew B. Watson,et al.  DCT quantization matrices visually optimized for individual images , 1993, Electronic Imaging.

[19]  D. Amnon Silverstein,et al.  The relationship between image fidelity and image quality , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[20]  Stefan Winkler,et al.  Issues in vision modeling for perceptual video quality assessment , 1999, Signal Process..

[21]  B. Wandell,et al.  Appearance of colored patterns: pattern-color separability. , 1993, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  Thrasyvoulos N. Pappas,et al.  Perceptual criteria for image quality evaluation , 2005 .

[23]  James Hu,et al.  DVQ: A digital video quality metric based on human vision , 2001 .

[24]  Eero P. Simoncelli,et al.  Image compression via joint statistical characterization in the wavelet domain , 1999, IEEE Trans. Image Process..

[25]  Eero P. Simoncelli,et al.  Natural image statistics and divisive normalization: Modeling nonlinearity and adaptation in cortical neurons , 2002 .

[26]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.

[27]  Neil W. Bergmann,et al.  An automatic image quality assessment technique incorporating higher level perceptual factors , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[28]  Bernd Girod,et al.  What's wrong with mean-squared error? , 1993 .

[29]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[30]  C.-C. Jay Kuo,et al.  A Haar Wavelet Approach to Compressed Image Quality Measurement , 2000, J. Vis. Commun. Image Represent..

[31]  Pierre Moulin,et al.  Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients , 2001, IEEE Trans. Image Process..

[32]  Andrew P. Bradley,et al.  A wavelet visible difference predictor , 1999, IEEE Trans. Image Process..

[33]  A B Watson,et al.  Visual detection of spatial contrast patterns: evaluation of five simple models. , 2000, Optics express.

[34]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[35]  John D. Villasenor,et al.  Visibility of wavelet quantization noise , 1997, IEEE Transactions on Image Processing.

[36]  M. G. Ramos,et al.  Suprathreshold wavelet coefficient quantization in complex stimuli: psychophysical evaluation and analysis. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[37]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[38]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[39]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Proceedings of 1st International Conference on Image Processing.

[40]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[41]  Eero P. Simoncelli.  Statistical models for images: compression, restoration and synthesis , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[42]  Andrew B. Watson,et al.  Measurement of visual impairment scales for digital video , 2001, IS&T/SPIE Electronic Imaging.

[43]  Wen Xu,et al.  Picture quality evaluation based on error segmentation , 1994, Other Conferences.

[44]  Sheila S. Hemami,et al.  Additivity models for suprathreshold distortion in quantized wavelet-coded images , 2002, IS&T/SPIE Electronic Imaging.

[45]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Alan C. Bovik,et al.  Image features that draw fixations , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[47]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[48]  J A Solomon,et al.  Model of visual contrast gain control and pattern masking. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[49]  Andrew B. Watson,et al.  The cortex transform: rapid computation of simulated neural images , 1987 .

[50]  Jean-Bernard Martens,et al.  Quality asessment of coded images using numerical category scaling , 1995, Other Conferences.