Pre-Attention and Spatial Dependency Driven No-Reference Image Quality Assessment

The excessive emulation of the human visual system and the lack of connection between chromatic data and distortion have been the major bottlenecks in developing image quality assessment. To address this issue, we develop a new no-reference (NR) image quality assessment (IQA) metric that accounts for the impact of pre-attention and spatial dependency on the perceived quality of distorted images. The resulting model, dubbed the Pre-attention and Spatial-dependency driven Quality Assessment (PSQA) predictor, introduces the pre-attention theory to emulate early phase visual perception by refining luminance-channel data. Chromatic data are also processed concurrently by transforming images from RGB to the perceptually optimized SCIELAB color space. Considering that the gray-tone spatial dependency matrix conveys important texture properties that are closely related to visual quality, this matrix, as a mathematical solution for subsequent visual process emulation, is calculated along with its statistical features on both gray and color channels. To clarify the influence of different regression procedures on model output, support vector regression and AdaBoosting Back Propagation (BP) neural networks are adopted separately to train the prediction models. We thoroughly evaluated PSQA on four public image quality databases: LIVE, TID2013, CSIQ, and VCL. The experimental results show that PSQA delivers highly competitive performance compared with top-rank NR and full-reference IQA metrics.

[1]  David S. Doermann,et al.  No-Reference Image Quality Assessment Using Visual Codebooks , 2012, IEEE Transactions on Image Processing.

[2]  Qingjie Zhao,et al.  Blind image quality assessment by relative gradient statistics and adaboosting neural network , 2016, Signal Process. Image Commun..

[3]  Hocine Cherifi,et al.  A statistical reduced-reference method for color image quality assessment , 2014, Multimedia Tools and Applications.

[4]  Weisi Lin,et al.  A Psychovisual Quality Metric in Free-Energy Principle , 2012, IEEE Transactions on Image Processing.

[5]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[6]  Brian A. Wandell,et al.  A spatial extension of CIELAB for digital color‐image reproduction , 1997 .

[7]  Wei Zhang,et al.  A Saliency Dispersion Measure for Improving Saliency-Based Image Quality Metrics , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Weisi Lin,et al.  No-Reference Image Sharpness Assessment in Autoregressive Parameter Space , 2015, IEEE Transactions on Image Processing.

[9]  Yong Gan,et al.  Perceptual image quality assessment by independent feature detector , 2015, Neurocomputing.

[10]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[11]  Xuelong Li,et al.  Universal Blind Image Quality Assessment Metrics Via Natural Scene Statistics and Multiple Kernel Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Alan Conrad Bovik,et al.  Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment , 2017, Signal Process. Image Commun..

[13]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[14]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Qing Wang,et al.  A new blind image quality framework based on natural color statistic , 2016, Neurocomputing.

[16]  Ione Fine,et al.  Surface segmentation based on the luminance and color statistics of natural scenes , 2002 .

[17]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Weisi Lin,et al.  Analysis of Distortion Distribution for Pooling in Image Quality Prediction , 2016, IEEE Transactions on Broadcasting.

[19]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Konstantinos N. Plataniotis,et al.  Toward a No-Reference Image Quality Assessment Using Statistics of Perceptual Color Descriptors , 2016, IEEE Transactions on Image Processing.

[21]  Weisi Lin,et al.  Image Sharpness Assessment by Sparse Representation , 2016, IEEE Transactions on Multimedia.

[22]  Yuukou Horita,et al.  No reference image quality assessment for JPEG2000 based on spatial features , 2008, Signal Process. Image Commun..

[23]  Rubén Muñiz,et al.  Novel Techniques for Color Texture Classification , 2006, IPCV.

[24]  Alan C. Bovik,et al.  Automatic Prediction of Perceptual Image and Video Quality , 2013, Proceedings of the IEEE.

[25]  Lei Zhang,et al.  Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features , 2014, IEEE Transactions on Image Processing.

[26]  Weisi Lin,et al.  No-Reference Quality Assessment for Multiply-Distorted Images in Gradient Domain , 2016, IEEE Signal Processing Letters.

[27]  Piyush Joshi,et al.  Retina inspired no-reference image quality assessment for blur and noise , 2017, Multimedia Tools and Applications.

[28]  Wei Zhang,et al.  Learning picture quality from visual distraction: Psychophysical studies and computational models , 2017, Neurocomputing.

[29]  Wei Zhang,et al.  Study of Saliency in Objective Video Quality Assessment , 2017, IEEE Transactions on Image Processing.

[30]  Frederic Dufaux,et al.  A subjective study of the influence of color information on visual quality assessment of high resolution pictures , 2009 .

[31]  Xiongkuo Min,et al.  Blind Quality Assessment Based on Pseudo-Reference Image , 2018, IEEE Transactions on Multimedia.

[32]  Wei Zhang,et al.  Toward a Reliable Collection of Eye-Tracking Data for Image Quality Research: Challenges, Solutions, and Applications , 2017, IEEE Transactions on Image Processing.

[33]  Zhou Wang,et al.  A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images , 2015, IEEE Signal Processing Letters.

[34]  T. Lindeberg,et al.  Scale-Space Theory : A Basic Tool for Analysing Structures at Different Scales , 1994 .

[35]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[36]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[37]  Ingrid Heynderickx,et al.  A No-Reference Metric for Perceived Ringing Artifacts in Images , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Seungmin Rho,et al.  Structured entropy of primitive: big data-based stereoscopic image quality assessment , 2017, IET Image Process..

[39]  Weisi Lin,et al.  No-Reference and Robust Image Sharpness Evaluation Based on Multiscale Spatial and Spectral Features , 2017, IEEE Transactions on Multimedia.

[40]  M. Potter Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[41]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[42]  Wenjun Zhang,et al.  No-Reference Quality Metric of Contrast-Distorted Images Based on Information Maximization , 2017, IEEE Transactions on Cybernetics.

[43]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[44]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[45]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[46]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[47]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[48]  Sonja Grgic,et al.  VCL@FER image quality assessment database , 2011, Proceedings ELMAR-2011.

[49]  Jiandong Tian,et al.  A New Intrinsic-Lighting Color Space for Daytime Outdoor Images , 2017, IEEE Transactions on Image Processing.

[50]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[51]  U. Neisser VISUAL SEARCH. , 1964, Scientific American.

[52]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[53]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[54]  Zhou Wang,et al.  Reduced-Reference Image Quality Assessment Using Divisive Normalization-Based Image Representation , 2009, IEEE Journal of Selected Topics in Signal Processing.

[55]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[56]  Ingrid Heynderickx,et al.  The Relative Impact of Ghosting and Noise on the Perceived Quality of MR Images , 2016, IEEE Transactions on Image Processing.

[57]  Q. M. Jonathan Wu,et al.  Utilizing Image Scales Towards Totally Training Free Blind Image Quality Assessment , 2015, IEEE Transactions on Image Processing.

[58]  Tony Lindeberg,et al.  A computational theory of visual receptive fields , 2013, Biological Cybernetics.

[59]  Sir G. Archaeopteryx Object-based attention in the primary visual cortex of the macaque monkey , 1998 .

[60]  Ghassan Al-Regib,et al.  CSV: Image quality assessment based on color, structure, and visual system , 2016, Signal Process. Image Commun..

[61]  Wen Gao,et al.  Reduced-Reference Image Quality Assessment in Free-Energy Principle and Sparse Representation , 2017, IEEE Transactions on Multimedia.

[62]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[63]  Houbing Song,et al.  Image Quality Assessment Using Regularity of Color Distribution , 2016, IEEE Access.

[64]  Sinan Kalkan,et al.  Deep Hierarchies in the Primate Visual Cortex: What Can We Learn for Computer Vision? , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[66]  Hua Huang,et al.  Image Quality Assessment Using Directional Anisotropy Structure Measurement , 2017, IEEE Transactions on Image Processing.