Full-Reference Image Quality Metrics: Classification and Evaluation

The wide variety of distortions that images are subject to during acquisition, processing, storage, and reproduction can degrade their perceived quality. Since subjective evaluation is time-consuming, expensive, and resource-intensive, objective methods of evaluation have been proposed. One type of these methods, image quality (IQ) metrics, have become very popular and new metrics are proposed continuously. This paper aims to give a survey of one class of metrics, full-reference IQ metrics. First, these IQ metrics were classified into different groups. Second, further IQ metrics from each group were selected and evaluated against six state-of-the-art IQ databases.

[1]  Gloria Menegaz,et al.  A Discrete Model for Color Naming , 2007, EURASIP J. Adv. Signal Process..

[2]  M. R. Luo,et al.  Colour 0510 th Congress of the International Colour Association 1095 Image Quality Assessment for Photographic Images , 2005 .

[3]  Brian A. Wandell,et al.  A spatial extension of CIELAB for digital color‐image reproduction , 1997 .

[4]  Stephane Pefferkorn,et al.  Perceptual quality metric of color quantization errors on still images , 1998, Electronic Imaging.

[5]  Weisi Lin,et al.  Perceived visual quality metric based on error spread and contrast , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[6]  Brian A. Wandell,et al.  Color image quality metric S-CIELAB and its application on halftone texture visibility , 1997, Proceedings IEEE COMPCON 97. Digest of Papers.

[7]  Lei Zhang,et al.  Image quality assessment based on edge , 2011, Electronic Imaging.

[8]  Raimondo Schettini,et al.  Image quality assessment by preprocessing and full reference model combination , 2009, Electronic Imaging.

[9]  Weisi Lin,et al.  Image Quality Measure using Curvature Similarity , 2007, 2007 IEEE International Conference on Image Processing.

[10]  Tao Wang,et al.  A Content-Based Image Quality Metric , 2005, RSFDGrC.

[11]  Xiaoxia Wan,et al.  Quality evaluation of the halftone by halftoning algorithm-based methods and adaptive method , 2007, Electronic Imaging.

[12]  S. Nakauchi,et al.  Color gamut mapping based on a perceptual image difference measure , 1999 .

[13]  M R Luo,et al.  New algorithm for calculating perceived colour difference of images , 2006 .

[14]  Claudio Oleari,et al.  An Alternative Color Difference Formula for Computing Image Difference , .

[15]  R. J. Safranek,et al.  A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[16]  Kim-Han Thung,et al.  A survey of image quality measures , 2009, 2009 International Conference for Technical Postgraduates (TECHPOS).

[17]  Ming Ronnier Luo,et al.  Perceptually-based color difference for complex images , 2002, Other Conferences.

[18]  Patrick C. Teo,et al.  A model of perceptual image fidelity , 1995, Proceedings., International Conference on Image Processing.

[19]  Eric P. Lam,et al.  An image similarity measure using homogeneity regions and structure , 2008, Electronic Imaging.

[20]  P. Wong.  A Mixture Distortion Criterion for Halftones , 1996 .

[21]  Mong-Shu Lee,et al.  Image Similarity Comparison Using Dual-Tree Wavelet Transform , 2006, PSIVT.

[22]  V. Ralph Algazi,et al.  Objective picture quality scale (PQS) for image coding , 1998, IEEE Trans. Commun..

[23]  Yangli Hector Yee,et al.  A perceptual metric for production testing , 2004, SIGGRAPH '04.

[24]  S.A. Karunasekera,et al.  A distortion measure for image artifacts based on human visual sensitivity , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[25]  Alberto Del Bimbo,et al.  Encyclopedia of Multimedia , 2006 .

[26]  Jon Y. Hardeberg,et al.  An adaptive Bilateral Filter for Predicting Color Image Difference , 2009, CIC.

[27]  Nikolay N. Ponomarenko,et al.  Color image database for evaluation of image quality metrics , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[28]  David L. Neuhoff,et al.  Least-squares model-based halftoning , 1992, Electronic Imaging.

[29]  Rui Seara,et al.  Perceptual image quality assessment based on Bayesian networks , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[30]  Ahmet M. Eskicioglu,et al.  A full-reference color image quality measure in the DWT domain , 2005, 2005 13th European Signal Processing Conference.

[31]  D. Chandler,et al.  Supplement to “ VSNR : A Visual Signal-to-Noise Ratio for Natural Images Based on Near-Threshold and Suprathreshold Vision ” , 2007 .

[32]  Lai-Man Po,et al.  Edge-Based Structural Similarity for Image Quality Assessment , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[33]  Tubagus Maulana Kusuma,et al.  Reduced-reference metric design for objective perceptual quality assessment in wireless imaging , 2009, Signal Process. Image Commun..

[34]  Faouzi Alaya Cheikh,et al.  Evaluation of a Difference of Gaussians Based Image Difference Metric in Relation to Perceived Compression Artifacts , 2010, ISVC.

[35]  Jon Y. Hardeberg,et al.  Estimating Print Quality Attributes by Image Quality Metrics , 2010, Color Imaging Conference.

[36]  Risto Näsänen.  Visibility of halftone dot textures , 1984, IEEE Trans. Syst. Man Cybern..

[37]  Lei Zhang,et al.  RFSIM: A feature based image quality assessment metric using Riesz transforms , 2010, 2010 IEEE International Conference on Image Processing.

[38]  C.-C. Jay Kuo,et al.  New image compression artifact measure using wavelets , 1997, Electronic Imaging.

[39]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[40]  Alexander Toet,et al.  A new universal colour image fidelity metric , 2003 .

[41]  Zhou Wang,et al.  Translation insensitive image similarity in complex wavelet domain , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[42]  Nicolas Bonnier,et al.  Improvements in Spatial and Color Adaptive Gamut Mapping Algorithms , 2008, CGIV/MCS.

[43]  A. Beghdadi,et al.  Color Image Assessment Using Spatial Extension to CIE DE2000 , 2008, 2008 Digest of Technical Papers - International Conference on Consumer Electronics.

[44]  Rafael Huertas,et al.  Euclidean color-difference formula for small-medium color differences in log-compressed OSA-UCS space. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[45]  Rui Seara,et al.  A measure for perceptual image quality assessment , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[46]  Zofia Baranczuk,et al.  Saliency Models as Gamut-Mapping Artifact Detectors , 2010, CGIV/MCS.

[47]  Marius Pedersen,et al.  Importance of region-of-interest on image difference metrics , 2007 .

[48]  Fjj Frans Blommaert,et al.  Perceptual error measure and its application to sampled and interpolated single-edged images , 1997 .

[49]  M. Luo,et al.  The development of the CIE 2000 Colour Difference Formula , 2001 .

[50]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[51]  Patrick Le Callet,et al.  An image quality assessment method based on perception of structural information , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[52]  Pasi Fränti.  Blockwise distortion measure for statistical and structural errors in digital images , 1998, Signal Process. Image Commun..

[53]  Azeddine Beghdadi,et al.  A new image distortion measure based on wavelet decomposition , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[54]  Chun-Hsien Chou,et al.  A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[55]  Carlo Gatta,et al.  A Multiscale Framework for Spatial Gamut Mapping , 2007, IEEE Transactions on Image Processing.

[56]  Zhou Wang,et al.  Quantifying color image distortions based on adaptive spatio-chromatic signal decompositions , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[57]  Eli Peli,et al.  Vision Models for Target Detection and Recognition: In Memory of Arthur Menendez , 1995 .

[58]  Jon Y. Hardeberg,et al.  Using gaze information to improve image difference metrics , 2008, Electronic Imaging.

[59]  Jon Y. Hardeberg,et al.  Evaluation of Image Quality Metrics for Color Prints , 2011, SCIA.

[60]  Andrew B. Watson,et al.  DCT quantization matrices visually optimized for individual images , 1993, Electronic Imaging.

[61]  Ivar Farup,et al.  Efficient hue-preserving and edge-preserving spatial color gamut mapping , 2007, Color Imaging Conference.

[62]  C.-C. Jay Kuo,et al.  Perceptual fidelity measure of digital color images , 1998, Electronic Imaging.

[63]  Kevin J. Parker,et al.  A New Metric for Color Halftone Visibility , 1998, PICS.

[64]  Xiao-Fan Feng,et al.  Halftone Quality Evaluation Using Color Visual Models , 2002, PICS.

[65]  J. Movshon,et al.  Analysis of the development of spatial contrast sensitivity in monkey and human infants. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[66]  Fabio Pellacini,et al.  Functional difference predictors (FDPs): measuring meaningful image differences , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[67]  Patrick Le Callet,et al.  Subjective quality assessment IRCCyN/IVC database , 2004 .

[68]  Jon Yngve Hardeberg,et al.  Survey of full-reference image quality metrics , 2009 .

[69]  Gaurav Sharma.  Digital Color Imaging Handbook , 2002 .

[70]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[71]  Ravi Sankar,et al.  An algorithm for image quality assessment , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[72]  Norimichi Tsumura,et al.  Perceptual color difference metric for complex images based on Mahalanobis distance , 2001, J. Electronic Imaging.

[73]  Mark D. Fairchild,et al.  Darwinism of Color Image Difference Models , 2001, CIC.

[74]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[75]  Joyce E. Farrell,et al.  Perceptual quality metric for digitally coded color images , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[76]  Mark D. Fairchild.  Still Photography Throwdown: Silver Halide vs. Silicon , 2010, Color Imaging Conference.

[77]  Jon Yngve Hardeberg,et al.  SHAME: A new spatial hue angle metric for perceptual image difference , 2010 .

[78]  Gary W. Meyer,et al.  A perceptually based adaptive sampling algorithm , 1998, SIGGRAPH.

[79]  Nikolay N. Ponomarenko,et al.  A NEW FULL-REFERENCE QUALITY METRICS BASED ON HVS , 2006 .

[80]  Mirko Guarnera,et al.  A perceptual quality metric for color-interpolated images , 2005, IS&T/SPIE Electronic Imaging.

[81]  A. Beghdadi,et al.  Image quality assessment using a neural network approach , 2004, Proceedings of the Fourth IEEE International Symposium on Signal Processing and Information Technology, 2004..

[82]  B. Rigg,et al.  Modification to the JPC79 Colour–difference Formula , 2008 .

[83]  Christine D. Piatko,et al.  Comparing Real and Synthetic Images: Some Ideas about Metrics , 1995, Rendering Techniques.

[84]  Fan Zhang,et al.  Limitation and challenges of image quality measurement , 2010, Visual Communications and Image Processing.

[85]  Michael Elad,et al.  Space-dependent color gamut mapping: a variational approach , 2005, IEEE Transactions on Image Processing.

[86]  Andrew P. Bradley,et al.  A wavelet visible difference predictor , 1999, IEEE Trans. Image Process..

[87]  Mikko Nuutinen,et al.  Development and measurement of the goodness of test images for visual print quality evaluation , 2010, Electronic Imaging.

[88]  Mark D. Fairchild,et al.  Development and Testing of a Color Space (IPT) with Improved Hue Uniformity , 1998, CIC.

[89]  Lindsay W. MacDonald,et al.  Colour Difference Metrics and Image Sharpness , 2000, Color Imaging Conference.

[90]  Kevin J. Parker,et al.  Quality issues in blue noise halftoning , 1998, Electronic Imaging.

[91]  Jon Y. Hardeberg,et al.  A New Spatial Hue Angle Metric for Perceptual Image Difference , 2009, CCIW.

[92]  Florica Mindru.  Structure and color based comparison of halftone images , 2006 .

[93]  Jon Y. Hardeberg,et al.  Rank Order and Image Difference Metrics , 2008, CGIV/MCS.

[94]  D. Tolhurst,et al.  Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes , 2000, Vision Research.

[95]  J. Astola,et al.  ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS , 2007 .

[96]  M. Ronnier Luo,et al.  Testing Color-Difference Formulae on Complex Images Using a CRT Monitor , 2000, Color Imaging Conference.

[97]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[98]  Sophie Triantaphillidou,et al.  Image quality comparison between JPEG and JPEG2000. I. Psychophysical investigation , 2007 .

[99]  Jon Yngve Hardeberg,et al.  Perceptual evaluation of color gamut mapping algorithms , 2008 .

[100]  Olof Bryngdahl,et al.  Texture metric of halftone images , 1996 .

[101]  Nathan Moroney,et al.  A Radial Sampling of the OSA Uniform Color Scales , 2003, Color Imaging Conference.

[102]  Mark D. Fairchild,et al.  Meet iCAM: A Next-Generation Color Appearance Model , 2002, Color Imaging Conference.

[103]  Jon Yngve Hardeberg,et al.  Evaluating colour image difference metrics for gamut‐mapped images , 2008 .

[104]  V. Ralph Algazi,et al.  Evaluation of a human-vision-system-based image fidelity metric for image compression , 1999, Optics & Photonics.

[105]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Electronic Imaging.

[106]  Guoping Qiu,et al.  Social image quality , 2011, Electronic Imaging.

[107]  Bülent Sankur,et al.  Statistical evaluation of image quality measures , 2002, J. Electronic Imaging.

[108]  Hua Li,et al.  Image Quality Assessment Using Rough Fuzzy Integrals , 2007, 27th International Conference on Distributed Computing Systems Workshops - Supplements (ICDCSW'07).

[109]  Peter Zolliker,et al.  Adding Local Contrast to Global Gamut Mapping Algorithms , 2006, CGIV.

[110]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[111]  Takahiko Horiuchi,et al.  Digital Color Image Halftone : Hybrid Error Diffusion Using the Mask Perturbation and Quality Verification , 2007 .

[112]  Nick Holliman,et al.  Stereoscopic image quality metrics and compression , 2008, Electronic Imaging.

[113]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[114]  Eric C. Larson,et al.  Most apparent distortion: a dual strategy for full-reference image quality assessment , 2009, Electronic Imaging.

[115]  Mark D. Fairchild,et al.  Color Gamut Mapping in a Hue-Linearized CIELAB Color Space , 1998, Color Imaging Conference.

[116]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[117]  Azeddine Beghdadi,et al.  A Wigner-Ville distribution-based image dissimilarity measure , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[118]  Theophano Mitsa,et al.  Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[119]  Tomas Akenine-Möller,et al.  High dynamic range texture compression for graphics hardware , 2006, SIGGRAPH 2006.

[120]  Ahmet M. Eskicioglu,et al.  Multidimensional image quality measure using singular value decomposition , 2003, IS&T/SPIE Electronic Imaging.

[121]  A "Retained Image Appearance" Metric For Full Tonal Scale, Colorimetric Evaluation Of Photographic Image Stability , 2004 .

[122]  Anil C. Kokaram,et al.  Perceptual distortion measure for edgelike artifacts in image sequences , 1998, Electronic Imaging.

[123]  M. Ronnier Luo,et al.  Experimental Filters for Estimating Image Differences , 2008, CGIV/MCS.

[124]  Chun-Ling Yang,et al.  Gradient-Based Structural Similarity for Image Quality Assessment , 2006, 2006 International Conference on Image Processing.

[125]  Kuo-Cheng Liu,et al.  A Fidelity Metric for Assessing Visual Quality of Color Images , 2007, 2007 16th International Conference on Computer Communications and Networks.

[126]  Arne Valberg,et al.  Towards a uniform color space: A better formula to describe the munsell and OSA color scales , 1986 .

[127]  Eric C. Larson,et al.  Unveiling relationships between regions of interest and image fidelity metrics , 2008, Electronic Imaging.

[128]  Thrasyvoulos N. Pappas,et al.  Structural Similarity Quality Metrics in a Coding Context: Exploring the Space of Realistic Distortions , 2006, IEEE Transactions on Image Processing.

[129]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[130]  Peter G. J. Barten,et al.  Evaluation of Subjective Image Quality with the Square Root Integral Method , 1990, Applied Vision.

[131]  Hans-Peter Seidel,et al.  Visible difference predicator for high dynamic range images , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[132]  Patrick Le Callet,et al.  A robust quality metric for color image quality assessment , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[133]  D. Amnon Silverstein,et al.  DCT image fidelity metric and its application to a text-based scheme for image display , 1993, Electronic Imaging.

[134]  Olof Bryngdahl,et al.  SPATIALLY DEPENDENT TEXTURE ANALYSIS AND CONTROL IN DIGITAL HALFTONING , 1997 .

[135]  Marius Pedersen,et al.  Analysis of the Difference of Gaussians Model in Image Difference Metrics , 2010, CGIV/MCS.

[136]  S. Triantaphillidou,et al.  Predicting image quality using a modular image difference model , 2008, Electronic Imaging.

[137]  Kebin An,et al.  Homogeneity based image objective quality metric , 2005 .

[138]  Qian Lin.  Halftone image quality analysis based on a human vision model , 1993, Electronic Imaging.

[139]  Mark D. Fairchild,et al.  On Contrast Sensitivity in an Image Difference Model , 2002, PICS.

[140]  Zygmunt Pizlo,et al.  Color Image Fidelity Assessor , 2001, PICS.

[141]  Sos S. Agaian,et al.  Quantifying image similarity using measure of enhancement by entropy , 2007, SPIE Defense + Commercial Sensing.

[142]  Reginald L. Lagendijk,et al.  Perceptual image quality based on a multiple channel HVS model , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[143]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[144]  Tamar Shoham,et al.  A novel perceptual image quality measure for block based image compression , 2011, Electronic Imaging.

[145]  M. Ronnier Luo,et al.  Colour Differences for Complex Images , 2003, Color Imaging Conference.

[146]  Zygmunt Pizlo,et al.  The Image Fidelity Assessor , 1998, PICS.

[147]  Elaine Jin,et al.  A Groundtruth Database for Testing Objective Metrics for Image Difference , 2003, PICS.

[148]  M. Ronnier Luo,et al.  The Fundamentals of Gamut Mapping: A Survey , 2001, Journal of Imaging Science and Technology.

[149]  Jon Yngve Hardeberg,et al.  Full-Reference Image Quality Metrics , 2012 .

[150]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[151]  Maeng-Sub Cho,et al.  Experimental Approach for Human Perception Based Image Quality Assessment , 2006, ICEC.

[152]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[153]  Hans Brettel,et al.  Evaluation of Spatial Gamut Mapping Algorithms , 2006, Color Imaging Conference.

[154]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[155]  Jeff B. Pelz,et al.  Eye Tracking Observers During Rank Order, Paired Comparison, and Graphical Rating Tasks , 2003, PICS.

[156]  Brian A. Wandell,et al.  Color image fidelity metrics evaluated using image distortion maps , 1998, Signal Process..

[157]  Dirk W. Hertel.  Exploring s-CIELAB as a scanner metric for print uniformity , 2005, IS&T/SPIE Electronic Imaging.

[158]  Deng,et al.  Image Quality Evaluation Method Based on Human Visual System , 2010 .

[159]  B. Rigg,et al.  BFD COLOUR - DIFFERENCE FORMULA PART1-DEVELOPMENT OF THE FORMULA , 1987 .

[160]  Teresa H. Meng,et al.  Psycho-visual based distortion measures for monochrome image and video compression , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[161]  David Connah,et al.  Can gamut mapping quality be predicted by colour image difference formulae? , 2005, IS&T/SPIE Electronic Imaging.

[162]  Ján Morovic,et al.  Visual Differences In Colour Reproduction And Their Colorimetric Correlates , 2002, Color Imaging Conference.

[163]  Alain Fournier,et al.  Multiscale edge analysis of halftoned images , 1998, Electronic Imaging.

[164]  Hubert Konik,et al.  Full Reference Image Quality Assessment Based on Saliency Map Analysis , 2010 .

[165]  Maryellen L. Giger,et al.  Evaluating the EM algorithm for image processing using a human visual fidelity criterion , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[166]  Elaine W. Jin,et al.  The Development of A Color Visual Difference Model (CVDM) , 1998, PICS.

[167]  László Neumann,et al.  Perception Based Color Image Difference , 1998, Comput. Graph. Forum.

[168]  Fredrik Nilsson.  Objective quality measures for halftoned images , 1999 .

[169]  Robyn A. Owens,et al.  A New Metric for Grey-Scale Image Comparison , 1997, International Journal of Computer Vision.

[170]  Jan P. Allebach,et al.  Methodology for designing image similarity metrics based on human visual system models , 1997, Electronic Imaging.

[171]  Charles E. Hughes,et al.  High-dynamic-range still-image encoding in JPEG 2000 , 2005, IEEE Computer Graphics and Applications.

[172]  Norimichi Tsumura,et al.  Evaluation of Image Corrected by Retinex Method Based on S-CIELAB and Gazing Information , 2006, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..