Understanding SSIM

The use of the structural similarity index (SSIM) is widespread. For almost two decades, it has played a major role in image quality assessment in many different research disciplines. Clearly, its merits are indisputable in the research community. However, little deep scrutiny of this index has been performed. Contrary to popular belief, there are some interesting properties of SSIM that merit such scrutiny. In this paper, we analyze the mathematical factors of SSIM and show that it can generate results, in both synthetic and realistic use cases, that are unexpected, sometimes undefined, and nonintuitive. As a consequence, assessing image quality based on SSIM can lead to incorrect conclusions and using SSIM as a loss function for deep learning can guide neural network training in the wrong direction.

[1]  Hantao Liu,et al.  Perceptual Quality Assessment of Medical Images , 2019, Encyclopedia of Biomedical Engineering.

[2]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[3]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[4]  Dominique Brunet,et al.  A Study of the Structural Similarity Image Quality Measure with Applications to Image Processing , 2012 .

[5]  Hans-Peter Seidel,et al.  Learning to Predict Localized Distortions in Rendered Images , 2013, Comput. Graph. Forum.

[6]  Zhou Wang,et al.  On the Mathematical Properties of the Structural Similarity Index , 2012, IEEE Transactions on Image Processing.

[7]  J. M. Foley,et al.  Contrast masking in human vision. , 1980, Journal of the Optical Society of America.

[8]  Zhou Wang,et al.  Complex Wavelet Structural Similarity: A New Image Similarity Index , 2009, IEEE Transactions on Image Processing.

[9]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[10]  Richard Dosselmann,et al.  A comprehensive assessment of the structural similarity index , 2011, Signal Image Video Process..

[11]  R. Dosselmann,et al.  A Formal Assessment of the Structural Similarity Index , 2008 .

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Rafal Mantiuk,et al.  Analysis of reported error in Monte Carlo rendered images , 2017, The Visual Computer.

[14]  Scott Daly,et al.  Digital Images and Human Vision , 1993 .

[15]  Andrew B. Watson,et al.  Digital images and human vision , 1993 .

[16]  Zhou Wang,et al.  Spherical Structural Similarity Index for Objective Omnidirectional Video Quality Assessment , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[17]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[18]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[19]  Tomas Akenine-Möller,et al.  FLIP: A Difference Evaluator for Alternating Images , 2020, Proc. ACM Comput. Graph. Interact. Tech..

[20]  Zhou Wang,et al.  Video quality assessment based on structural distortion measurement , 2004, Signal Process. Image Commun..

[21]  Radoslaw Mantiuk,et al.  Calibration of Structural Similarity Index Metric to Detect Artefacts in Game Engines , 2016, ICCVG.

[22]  Kai Zeng,et al.  3D-SSIM for video quality assessment , 2012, 2012 19th IEEE International Conference on Image Processing.

[23]  Hans-Peter Seidel,et al.  Dataset and Metrics for Predicting Local Visible Differences , 2018, ACM Trans. Graph..

[24]  Hans-Peter Seidel,et al.  New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts , 2012, ACM Trans. Graph..

[25]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.