Image quality assessment: from error visibility to structural similarity

Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a structural similarity index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MATLAB implementation of the proposed algorithm is available online at http://www.cns.nyu.edu//spl sim/lcv/ssim/.

[1]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[2]  Andrew B. Watson,et al.  The cortex transform: rapid computation of simulated neural images , 1987 .

[3]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[4]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[5]  Bernd Girod,et al.  What's wrong with mean-squared error? , 1993 .

[6]  B. Wandell,et al.  Appearance of colored patterns: pattern-color separability. , 1993, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[8]  Andrew B. Watson,et al.  DCT quantization matrices visually optimized for individual images , 1993, Electronic Imaging.

[9]  Jeffrey Lubin,et al.  The use of psychophysical data and models in the analysis of display system performance , 1993 .

[10]  Andrew B. Watson,et al.  Visually optimal DCT quantization matrices for individual images , 1993, [Proceedings] DCC `93: Data Compression Conference.

[11]  Geoffrey M. Boynton,et al.  New model of human luminance pattern vision mechanisms: analysis of the effects of pattern orientation, spatial phase, and temporal frequency , 1994, Other Conferences.

[12]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Proceedings of 1st International Conference on Image Processing.

[13]  Wen Xu,et al.  Picture quality evaluation based on error segmentation , 1994, Other Conferences.

[14]  Patrick C. Teo,et al.  A model of perceptual image fidelity , 1995, Proceedings., International Conference on Image Processing.

[15]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[16]  Jean-Bernard Martens,et al.  Quality asessment of coded images using numerical category scaling , 1995, Other Conferences.

[17]  Jerome R. Cox,et al.  Experimental evaluation of psychophysical distortion metrics for JPEG-encoded images , 1995, J. Electronic Imaging.

[18]  Jeffrey Lubin,et al.  A VISUAL DISCRIMINATION MODEL FOR IMAGING SYSTEM DESIGN AND EVALUATION , 1995 .

[19]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[20]  Olivier Verscheure,et al.  Perceptual quality measure using a spatiotemporal model of the human visual system , 1996, Electronic Imaging.

[21]  D. Amnon Silverstein,et al.  The relationship between image fidelity and image quality , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[22]  Eero P. Simoncelli.  Statistical models for images: compression, restoration and synthesis , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[23]  John D. Villasenor,et al.  Visibility of wavelet quantization noise , 1997, IEEE Transactions on Image Processing.

[24]  J A Solomon,et al.  Model of visual contrast gain control and pattern masking. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[25]  Andrew P. Bradley,et al.  Perceptual quality metrics applied to still image compression , 1998, Signal Process..

[26]  Neil W. Bergmann,et al.  An automatic image quality assessment technique incorporating higher level perceptual factors , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[27]  Jean-Bernard Martens,et al.  Image dissimilarity , 1998, Signal Process..

[28]  Andrew P. Bradley,et al.  A wavelet visible difference predictor , 1999, IEEE Trans. Image Process..

[29]  Eero P. Simoncelli,et al.  Image compression via joint statistical characterization in the wavelet domain , 1999, IEEE Trans. Image Process..

[30]  Stefan Winkler,et al.  Issues in vision modeling for perceptual video quality assessment , 1999, Signal Process..

[31]  Pascual Capilla,et al.  Image quality metric based on multidimensional contrast perception models , 1999 .

[32]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.

[33]  C.-C. Jay Kuo,et al.  A Haar Wavelet Approach to Compressed Image Quality Measurement , 2000, J. Vis. Commun. Image Represent..

[34]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  A B Watson,et al.  Visual detection of spatial contrast patterns: evaluation of five simple models. , 2000, Optics express.

[36]  Francesc J. Ferri,et al.  Non-linear Invertible Representation for Joint Statistical and Perceptual Feature Decorrelation , 2000, SSPR/SPR.

[37]  James Hu,et al.  DVQ: A digital video quality metric based on human vision , 2001 .

[38]  Zhou Wang,et al.  Embedded foveation image coding , 2001, IEEE Trans. Image Process..

[39]  M. G. Ramos,et al.  Suprathreshold wavelet coefficient quantization in complex stimuli: psychophysical evaluation and analysis. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[41]  Pierre Moulin,et al.  Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients , 2001, IEEE Trans. Image Process..

[42]  Andrew B. Watson,et al.  Measurement of visual impairment scales for digital video , 2001, IS&T/SPIE Electronic Imaging.

[43]  Jing Xing,et al.  P‐14: An Image Processing Model of Contrast Perception and Discrimination of the Human Visual System , 2002 .

[44]  Sheila S. Hemami,et al.  Additivity models for suprathreshold distortion in quantized wavelet-coded images , 2002, IS&T/SPIE Electronic Imaging.

[45]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[46]  Eero P. Simoncelli,et al.  Natural image statistics and divisive normalization: Modeling nonlinearity and adaptation in cortical neurons , 2002 .

[47]  Zhou Wang,et al.  Why is image quality assessment so difficult? , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[48]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[49]  A. Bovik,et al.  OBJECTIVE VIDEO QUALITY ASSESSMENT , 2003 .

[50]  Alan C. Bovik,et al.  41 OBJECTIVE VIDEO QUALITY ASSESSMENT , 2003 .

[51]  Alan C. Bovik,et al.  Image features that draw fixations , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[52]  Jesús Malo,et al.  Linear transform for simultaneous diagonalization of covariance and perceptual metric matrix in image coding , 2003, Pattern Recognit..

[53]  Thrasyvoulos N. Pappas,et al.  Perceptual criteria for image quality evaluation , 2005 .