On the Mathematical Properties of the Structural Similarity Index

Since its introduction in 2004, the structural similarity (SSIM) index has gained widespread popularity as a tool to assess the quality of images and to evaluate the performance of image processing algorithms and systems. There has been also a growing interest of using SSIM as an objective function in optimization problems in a variety of image processing applications. One major issue that could strongly impede the progress of such efforts is the lack of understanding of the mathematical properties of the SSIM measure. For example, some highly desirable properties such as convexity and triangular inequality that are possessed by the mean squared error may not hold. In this paper, we first construct a series of normalized and generalized (vector-valued) metrics based on the important ingredients of SSIM. We then show that such modified measures are valid distance metrics and have many useful properties, among which the most significant ones include quasi-convexity, a region of convexity around the minimizer, and distance preservation under orthogonal or unitary transformations. The groundwork laid here extends the potentials of SSIM in both theoretical development and practical applications.

[1]  I. J. Schoenberg.  A remark on M. M. Day’s characterization of inner-product spaces and a conjecture of L. M. Blumenthal , 1952 .

[2]  Robert W. Heath,et al.  SSIM-optimal linear image restoration , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[4]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[6]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[7]  Hans Burkhardt,et al.  Multichannel image restoration based on optimization of the structural similarity index , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[8]  T. Apostol.  Ptolemy's Inequality and the Chordal Metric , 1967 .

[9]  Zhou Wang,et al.  Structural Similarity-Based Affine Approximation and Self-similarity of Images Revisited , 2011, ICIAR.

[10]  Xian Zhang,et al.  Cone metric spaces and fixed point theorems of contractive mappings , 2007 .

[11]  Robert W. Heath,et al.  Design of Linear Equalizers Optimized for the Structural Similarity Index , 2008, IEEE Transactions on Image Processing.

[12]  M. S. Klamkin,et al.  Ptolemy's inequality, chordal metric, multiplicative metric. , 1982 .

[13]  Javier Portilla,et al.  AND ITS APPLICATION TO IMAGE DENOISING , 2006 .

[14]  Thomas Richter,et al.  A MS-SSIM Optimal JPEG 2000 Encoder , 2009, 2009 Data Compression Conference.

[15]  Robert W. Heath,et al.  Rate Bounds on SSIM Index of Quantized Images , 2008, IEEE Transactions on Image Processing.

[16]  Peter A. Hasto.  A New Weighted Metric: the Relative Metric II , 2001 .

[17]  Sheila S. Hemami,et al.  Understanding and simplifying the structural similarity metric , 2008, 2008 15th IEEE International Conference on Image Processing.

[18]  Zhou Wang,et al.  Stimulus synthesis for efficient evaluation and refinement of perceptual image quality metrics , 2004, IS&T/SPIE Electronic Imaging.

[19]  Eero P. Simoncelli,et al.  Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities. , 2008, Journal of vision.

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Zhou Wang,et al.  Perceptual Image Coding Based on a Maximum of Minimal Structural Similarity Criterion , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Edward R. Vrscay,et al.  SSIM-inspired image denoising using sparse representations , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Jesús Malo,et al.  Linear transform for simultaneous diagonalization of covariance and perceptual metric matrix in image coding , 2003, Pattern Recognit..

[24]  N. Lu,et al.  Fractal imaging , 1997 .

[25]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[26]  Eero P. Simoncelli,et al.  Nonlinear image representation for efficient perceptual coding , 2006, IEEE Transactions on Image Processing.

[27]  Homer H. Chen,et al.  Perceptual Rate-Distortion Optimization Using Structural Similarity Index as Quality Metric , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Sumohana S. Channappayya,et al.  The High-Resolution Rate-Distortion Function under the Structural Similarity Index , 2011, EURASIP J. Adv. Signal Process..

[29]  Alireza Nasiri Avanaki,et al.  Exact global histogram specification optimized for structural similarity , 2008, 0901.0065.

[30]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[31]  Robert W. Heath,et al.  A Linear Estimator Optimized for the Structural Similarity Index and its Application to Image Denoising , 2006, 2006 International Conference on Image Processing.

[32]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[33]  Zhou Wang,et al.  Structural Similarity-Based Approximation of Signals and Images Using Orthogonal Bases , 2010, ICIAR.

[34]  Xin Li,et al.  Collective sensing: a fixed-point approach in the metric space , 2010, Visual Communications and Image Processing.

[35]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[36]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Proceedings of 1st International Conference on Image Processing.

[37]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[38]  Nikolay N. Ponomarenko,et al.  METRICS PERFORMANCE COMPARISON FOR COLOR IMAGE DATABASE , 2008 .

[39]  Peter Hästö,et al.  A new weighted metric , 2005 .

[40]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[41]  Valero Laparra,et al.  Divisive normalization image quality metric revisited. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[42]  Lai-Man Po,et al.  An SSIM-optimal H.264/AVC inter frame encoder , 2009, 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems.

[43]  Peter Yianilos,et al.  Normalized Forms for Two Common Metrics , 1991 .

[44]  Zhou Wang,et al.  A Class of Image Metrics Based on the Structural Similarity Quality Index , 2011, ICIAR.

[45]  Peter Alexander Hst,et al.  A New Weighted Metric: the Relative Metric I , 2001 .

[46]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[47]  Wen Gao,et al.  Rate-SSIM optimization for video coding , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.