Statistical evaluation of no-reference image visual quality metrics

A task of no-reference visual quality metric verification is considered. A test set that contains 500 JPEG format images having different distortions is created. The results of experiments carried out by 316 volunteer observers to evaluate visual quality of images are presented. The experiments allowed obtaining mean opinion scores (MOS) based on averaging the evaluations. Several non-reference image visual quality metrics have been analyzed. Spearman and Kendall correlations between MOS and metrics values are calculated. It is shown that all analyzed metrics have not enough correspondence to human perception.

[1]  Sanjit K. Mitra,et al.  No-reference video quality metric based on artifact measurements , 2005, IEEE International Conference on Image Processing 2005.

[2]  Alan C. Bovik,et al.  DCT-domain blind measurement of blocking artifacts in DCT-coded images , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[3]  Judith Redi,et al.  Hybrid Neural Systems for Reduced-Reference Image Quality Assessment , 2009, ICANN.

[4]  R. Venkatesh Babu,et al.  No-reference image quality assessment using modified extreme learning machine classifier , 2009, Appl. Soft Comput..

[5]  B. Vozel,et al.  Improved Noise Parametr Estimation and Filtering of MM-Band SLAR Images , 2007, 2007 International Kharkov Symposium Physics and Engrg. of Millimeter and Sub-Millimeter Waves (MSMW).

[6]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[7]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[8]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[9]  Lina J. Karam,et al.  Adaptive image coding with perceptual distortion control , 2002, IEEE Trans. Image Process..

[10]  Yuukou Horita,et al.  No-reference image quality assessment for JPEG/JPEG2000 coding , 2004, 2004 12th European Signal Processing Conference.

[11]  Maurice G. Kendall,et al.  The advanced theory of statistics , 1945 .

[12]  Stefan Winkler,et al.  Perceptual blur and ringing metrics: application to JPEG2000 , 2004, Signal Process. Image Commun..

[13]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[14]  Andrew Perkis,et al.  A Perceptual No-Reference Blockiness Metric for JPEG Images , 2004, ICVGIP.

[15]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[16]  Stefan Winkler,et al.  A no-reference perceptual blur metric , 2002, Proceedings. International Conference on Image Processing.

[17]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[18]  Jorge E. Caviedes,et al.  No-reference sharpness metric based on local edge kurtosis , 2002, Proceedings. International Conference on Image Processing.

[19]  Marc Gazalet,et al.  Univariant assessment of the quality of images , 2002, J. Electronic Imaging.

[20]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..