Studying the effect of optimizing image quality in salient regions at the expense of background content

Abstract. Manufacturers of commercial display devices continuously try to improve the perceived image quality of their products. By applying postprocessing techniques on the incoming signal, they aim to enhance the quality level perceived by the viewer. These postprocessing techniques are usually applied globally over the whole image but may cause side effects, the visibility and annoyance of which differ with local content characteristics. To better understand and utilize this, a three-phase experiment was conducted where observers were asked to score images that had different levels of quality in their regions of interest and in the background areas. The results show that the region of interest has a greater effect on the overall quality of the image than the background. This effect increases with the increasing quality difference between the two regions. Based on the subjective data we propose a model to predict the overall quality of images with different quality levels in different regions. This model, which is constructed on empirical bases, can help craft weighted objective metrics that can better approximate subjective quality scores.

[1]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[2]  Ingrid Heynderickx,et al.  A Simplified Human Vision Model Applied to a Blocking Artifact Metric , 2007, CAIP.

[3]  Ingrid Heynderickx,et al.  A Perceptually Relevant No-Reference Blockiness Metric Based on Local Image Characteristics , 2009, EURASIP J. Adv. Signal Process..

[4]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2013, The Kluwer international series in engineering and computer science.

[5]  Patrick Le Callet,et al.  Does where you Gaze on an Image Affect your Perception of Quality? Applying Visual Attention to Image Quality Metric , 2007, 2007 IEEE International Conference on Image Processing.

[6]  Bernd Girod,et al.  The Information Theoretical Significance of Spatial and Temporal Masking in Video Signals , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[7]  David S. Taubman,et al.  High performance scalable image compression with EBCOT. , 2000, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[8]  Stefan Winkler,et al.  A no-reference perceptual blur metric , 2002, Proceedings. International Conference on Image Processing.

[9]  Liming Zhang,et al.  Image quality assessment with visual attention , 2008, 2008 19th International Conference on Pattern Recognition.

[10]  R. SheikhH.,et al.  No-reference quality assessment using natural scene statistics , 2005 .

[11]  Ingrid Heynderickx,et al.  Perceivable artifacts in compressed video and their relation to video quality , 2009, Signal Process. Image Commun..

[12]  H.R. Wu,et al.  A generalized block-edge impairment metric for video coding , 1997, IEEE Signal Processing Letters.

[13]  Alan C. Bovik,et al.  GAFFE: A Gaze-Attentive Fixation Finding Engine , 2008, IEEE Transactions on Image Processing.

[14]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[15]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[16]  Junle Wang,et al.  Quantifying the relationship between visual salience and visual importance , 2010, Electronic Imaging.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  C.-C. Jay Kuo,et al.  Review of Postprocessing Techniques for Compression Artifact Removal , 1998, J. Vis. Commun. Image Represent..

[19]  Michael Yuen,et al.  A survey of hybrid MC/DPCM/DCT video coding distortions , 1998, Signal Process..

[20]  L. Pratap Reddy,et al.  Image Quality Assessment Complemented with Visual Regions of Interest , 2007, 2007 International Conference on Computing: Theory and Applications (ICCTA'07).

[21]  Sanjit K. Mitra,et al.  Perceptual contributions of blocky, blurry, and fuzzy impairments to overall annoyance , 2004, IS&T/SPIE Electronic Imaging.

[22]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[23]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Judith Redi,et al.  Color Distribution Information for the Reduced-Reference Assessment of Perceived Image Quality , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Andrew Perkis,et al.  No-reference JPEG-image quality assessment using GAP-RBF , 2007, Signal Process..

[26]  Jeffrey Lubin,et al.  The use of psychophysical data and models in the analysis of display system performance , 1993 .

[27]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[28]  Heng-Ming Tai,et al.  Region of interest coding for image compression , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[29]  Anastasis A. Sofokleous,et al.  Review: H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia , 2005, Comput. J..

[30]  Abdelhakim Saadane,et al.  Blind Quality Metric using a Perceptual Importance Map for JPEG-20000 Compressed Images , 2006, 2006 International Conference on Image Processing.

[31]  Iain E. G. Richardson,et al.  H.264 and MPEG-4 Video Compression: Video Coding for Next-Generation Multimedia , 2003 .

[32]  Andrew Perkis,et al.  Attention modeling for video quality assessment: Balancing global quality and local quality , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[33]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[34]  R. J. Safranek,et al.  A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[35]  Nancy Millette,et al.  How People Look at Pictures , 1935 .

[36]  Sanjit K. Mitra,et al.  Annoyance of individual artifacts in MPEG-2 compressed video and their relation to overall annoyance , 2005, IS&T/SPIE Electronic Imaging.

[37]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Glen P. Abousleman,et al.  A no-reference perceptual image sharpness metric based on saliency-weighted foveal pooling , 2008, 2008 15th IEEE International Conference on Image Processing.

[39]  Jae Lim,et al.  Reduction Of Blocking Effects In Image Coding , 1984 .

[40]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[41]  K. van Zon,et al.  Automated video chain optimization , 2001, ICCE 2001.

[42]  L. Itti,et al.  Quantifying center bias of observers in free viewing of dynamic natural scenes. , 2009, Journal of vision.

[43]  Ingrid Heynderickx,et al.  How Does Image Content Affect the Added Value of Visual Attention in Objective Image Quality Assessment? , 2013, IEEE Signal Processing Letters.

[44]  Judith Redi,et al.  Image quality and visual attention interactions: Towards a more reliable analysis in the saliency space , 2011, 2011 Third International Workshop on Quality of Multimedia Experience.

[45]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[46]  Ingrid Heynderickx,et al.  A No-Reference Metric for Perceived Ringing Artifacts in Images , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Alan C. Bovik,et al.  Visual Importance Pooling for Image Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[48]  Mohammed Ghanbari,et al.  Standard Codecs: Image Compression to Advanced Video Coding , 2003 .

[49]  Touradj Ebrahimi,et al.  Balancing Attended and Global Stimuli in Perceived Video Quality Assessment , 2011, IEEE Transactions on Multimedia.

[50]  Jiebo Luo,et al.  Artifact reduction in low bit rate DCT-based image compression , 1996, IEEE Trans. Image Process..

[51]  James Hu,et al.  DVQ: A digital video quality metric based on human vision , 2001 .

[52]  Moncef Gabbouj,et al.  Improving image quality assessment with modeling visual attention , 2010, 2010 2nd European Workshop on Visual Information Processing (EUVIP).

[53]  M. Tinker How People Look at Pictures. , 1936 .

[54]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[55]  Zhou Wang,et al.  Blind measurement of blocking artifacts in images , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).