Learning picture quality from visual distraction: Psychophysical studies and computational models

Visual saliency has been increasingly studied in relation to image quality assessment. Incorporating saliency potentially leads to improved ability of image quality metrics to predict perceived quality. However, challenges to optimising the combination of saliency and image quality metrics remain. Previous psychophysical studies have shown that distortion occurring in an image causes visual distractions, and alters gaze patterns relative to that of the image without distortion. From this, it can be inferred that the measurable changes of gaze patterns driven by distortion may be used as a proxy for the likely variation in perceived quality of natural images. In this paper, rather than using saliency as an add-on to image quality metrics, we investigate the plausibility of approximating picture quality based on measuring the deviation of saliency induced by distortion. First, we designed and conducted a large-scale eye-tracking experiment to clarify the knowledge on the relationship between the deviation of saliency and the variability of image quality. We then used the results to devise an algorithm which predicts perceived image quality based on visual distraction. Experimental results demonstrate this can provide good results of image quality prediction.

[1]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[2]  Zhuo Chen,et al.  Natural image statistics based 3D reduced reference image quality assessment in contourlet domain , 2015, Neurocomputing.

[3]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[4]  E.C.L. Vu,et al.  Visual Fixation Patterns when Judging Image Quality: Effects of Distortion Type, Amount, and Subject Experience , 2008, 2008 IEEE Southwest Symposium on Image Analysis and Interpretation.

[5]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[6]  Judith Redi,et al.  Interactions of visual attention and quality perception , 2011, Electronic Imaging.

[7]  Ulrich Engelke,et al.  Visual Attention in Quality Assessment , 2011, IEEE Signal Processing Magazine.

[8]  Wen Lu,et al.  On combining visual perception and color structure based image quality assessment , 2016, Neurocomputing.

[9]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[10]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[11]  K. Guo,et al.  How does image noise affect actual and predicted human gaze allocation in assessing image quality? , 2015, Vision Research.

[12]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[13]  D. Chandler.  Seven Challenges in Image Quality Assessment: Past, Present, and Future Research , 2013 .

[14]  Nicolas Riche,et al.  Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Xiaoqiang Lu,et al.  Image quality assessment: A sparse learning way , 2015, Neurocomputing.

[16]  Thomas Martinetz,et al.  Variability of eye movements when viewing dynamic natural scenes. , 2010, Journal of vision.

[17]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[18]  Baojun Zhao,et al.  Gradient-based no-reference image blur assessment using extreme learning machine , 2016, Neurocomputing.

[19]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Stefan Winkler,et al.  Review of Existing Objective QoE Methodologies , 2015 .

[21]  Tao Liu,et al.  Saliency based objective quality assessment of decoded video affected by packet losses , 2008, 2008 15th IEEE International Conference on Image Processing.

[22]  A. Greenwald.  Within-subjects designs: To use or not to use? , 1976 .

[23]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[24]  Judith Redi,et al.  Circular-ELM for the reduced-reference assessment of perceived image quality , 2013, Neurocomputing.

[25]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  Christof Koch,et al.  Learning visual saliency by combining feature maps in a nonlinear manner using AdaBoost. , 2012, Journal of vision.

[27]  Peng Peng,et al.  General-purpose image quality assessment based on distortion-aware decision fusion , 2014, Neurocomputing.

[28]  E. Miller,et al.  Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices , 2007, Science.

[29]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jillian H. Fecteau,et al.  Salience, relevance, and firing: a priority map for target selection , 2006, Trends in Cognitive Sciences.

[33]  A. Torralba,et al.  Fixations on low-resolution images. , 2010, Journal of vision.

[34]  Qi Liu,et al.  Fast image quality assessment via supervised iterative quantization method , 2016, Neurocomputing.

[35]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[36]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[37]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[38]  Weisi Lin,et al.  BSD: Blind image quality assessment based on structural degradation , 2017, Neurocomputing.

[39]  David H Brainard,et al.  Surface color perception and equivalent illumination models. , 2011, Journal of vision.

[40]  Patrick Le Callet,et al.  Does where you Gaze on an Image Affect your Perception of Quality? Applying Visual Attention to Image Quality Metric , 2007, 2007 IEEE International Conference on Image Processing.

[41]  Peter G. J. Barten,et al.  Formula for the contrast sensitivity of the human eye , 2003, IS&T/SPIE Electronic Imaging.

[42]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[43]  Wei Zhang,et al.  The Application of Visual Saliency Models in Objective Image Quality Assessment: A Statistical Evaluation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Weisi Lin,et al.  No-reference quality assessment of deblocked images , 2016, Neurocomputing.

[45]  Za'er Salim Abo-Hammour,et al.  Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm , 2014, Inf. Sci..

[46]  Yong Gan,et al.  Perceptual image quality assessment by independent feature detector , 2015, Neurocomputing.

[47]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[48]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[49]  Xiongkuo Min,et al.  Influence of compression artifacts on visual attention , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[50]  Patrick Le Callet,et al.  Overt visual attention for free-viewing and quality assessment tasks Impact of the regions of interest on a video quality metric , 2010 .