A new method for evaluating the subjective image quality of photographs: dynamic reference

The Dynamic Reference (DR) method has been developed for subjective image quality experiments in which original or undistorted images are unavailable. The DR method creates reference image series from test images. Reference images are presented to observers as a slide show prior to evaluating their quality. As the observers view the set of reference images, they determine the overall variation in quality within the set of test images. This study compared the performance of the DR method to that of the standardized absolute category rating (ACR) and paired comparison (PC) methods. We measured the performance of each method in terms of time effort and discriminability. The results showed that the DR method is faster than the PC method and more accurate than the ACR method. The DR method is especially suitable for experiments that require highly accurate results in a short time.

[1]  D. Heitjan,et al.  Mixed-effects models in psychophysiology. , 2000, Psychophysiology.

[2]  Elaine W. Jin,et al.  Slider-adjusted softcopy ruler for calibrated image quality assessment , 2010, J. Electronic Imaging.

[3]  Judith Redi,et al.  Comparing subjective image quality measurement methods for the creation of public databases , 2010, Electronic Imaging.

[4]  Kees Teunissen.  The Validity of CCIR Quality Indicators Along a Graphical Scale , 1996 .

[5]  Methodology for the subjective assessment of video quality in multimedia applications ( Question , 2007 .

[6]  Jarno Nikkanen,et al.  Subjective effects of white-balancing errors in digital photography , 2008 .

[7]  Cesson Sévigné.  NEW QUALITY EVALUATION METHOD SUITED TO MULTIMEDIA CONTEXT SAMVIQ , 2006 .

[8]  Göte Nyman,et al.  Subjective experience of image quality: attributes, definitions, and decision making of subjective image quality , 2009, Electronic Imaging.

[9]  Hiroshi Yamaguchi,et al.  Evaluating HDR rendering algorithms , 2007, TAP.

[10]  Glenys A. Hamilton,et al.  Image quality preferences among radiographers and radiologists. A conjoint analysis , 2005 .

[11]  Jukka Häkkinen,et al.  Evaluating the multivariate visual quality performance of image-processing components , 2008, TAP.

[12]  D. Amnon Silverstein,et al.  Efficient method for paired comparison , 2001, J. Electronic Imaging.

[13]  Pablo Martínez-Cañada,et al.  Real-time tone mapping on GPU and FPGA , 2012, EURASIP J. Image Video Process..

[14]  Stefan Winkler,et al.  Analysis of Public Image and Video Databases for Quality Assessment , 2012, IEEE Journal of Selected Topics in Signal Processing.

[15]  Stefan Winkler,et al.  On the properties of subjective ratings in video quality experiments , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[16]  Takanori Hayashi,et al.  Performance comparison of subjective assessment methods for 3D video quality , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[17]  V. Seagroatt.  An introduction to medical statistics (2nd ed.) , 1996 .

[18]  Rafal Mantiuk,et al.  Assessment of video tone-mapping: Are cameras' S-shaped tone-curves good enough? , 2013, J. Vis. Commun. Image Represent..

[19]  Jianping Zhou,et al.  Image Pipeline Tuning for Digital Cameras , 2007, 2007 IEEE International Symposium on Consumer Electronics.

[20]  W.E. Snyder,et al.  Color image processing pipeline , 2005, IEEE Signal Processing Magazine.

[21]  Anthony J. Maeder,et al.  Human observer confidence in image quality assessment , 2012, Signal Process. Image Commun..

[22]  Mikko Nuutinen,et al.  A framework for measuring sharpness in natural images captured by digital cameras based on reference image and local areas , 2012, EURASIP J. Image Video Process..

[23]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[24]  Mikko Nuutinen,et al.  Features for Predicting Quality of Images Captured by Digital Cameras , 2012, 2012 IEEE International Symposium on Multimedia.

[25]  Terry L King.  A Guide to Chi-Squared Testing , 1997 .

[26]  Rafal Mantiuk,et al.  Comparison of Four Subjective Methods for Image Quality Assessment , 2012, Comput. Graph. Forum.

[27]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[28]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.