No Reference Quality Assessment for Screen Content Images With Both Local and Global Feature Representation

In this paper, we propose a novel no reference quality assessment method by incorporating statistical luminance and texture features (NRLT) for screen content images (SCIs) with both local and global feature representation. The proposed method is designed inspired by the perceptual property of the human visual system (HVS) that the HVS is sensitive to luminance change and texture information for image perception. In the proposed method, we first calculate the luminance map through the local normalization, which is further used to extract the statistical luminance features in global scope. Second, inspired by existing studies from neuroscience that high-order derivatives can capture image texture, we adopt four filters with different directions to compute gradient maps from the luminance map. These gradient maps are then used to extract the second-order derivatives by local binary pattern. We further extract the texture feature by the histogram of high-order derivatives in global scope. Finally, support vector regression is applied to train the mapping function from quality-aware features to subjective ratings. Experimental results on the public large-scale SCI database show that the proposed NRLT can achieve better performance in predicting the visual quality of SCIs than relevant existing methods, even including some full reference visual quality assessment methods.

[1]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[2]  Guangming Shi,et al.  Visual Orientation Selectivity Based Structure Description , 2015, IEEE Transactions on Image Processing.

[3]  Kai-Kuang Ma,et al.  Gradient Direction for Screen Content Image Quality Assessment , 2016, IEEE Signal Processing Letters.

[4]  Zhou Wang,et al.  No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics , 2015, IEEE Signal Processing Letters.

[5]  Zhou Wang,et al.  Subjective quality assessment of Screen Content Images , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[6]  J A Solomon,et al.  Model of visual contrast gain control and pattern masking. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[8]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[9]  Liming Chen,et al.  HSOG: A Novel Local Image Descriptor Based on Histograms of the Second-Order Gradients , 2014, IEEE Transactions on Image Processing.

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Abdul Rehman,et al.  Reduced-Reference Image Quality Assessment by Structural Similarity Estimation , 2012, IEEE Transactions on Image Processing.

[12]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[13]  Kai-Kuang Ma,et al.  ESIM: Edge Similarity for Screen Content Image Quality Assessment , 2017, IEEE Transactions on Image Processing.

[14]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[15]  Weisi Lin,et al.  No-Reference Quality Assessment for Multiply-Distorted Images in Gradient Domain , 2016, IEEE Signal Processing Letters.

[16]  Weisi Lin,et al.  No-Reference Image Blur Assessment Based on Discrete Orthogonal Moments , 2016, IEEE Transactions on Cybernetics.

[17]  Guangming Shi,et al.  Orientation selectivity based visual pattern for reduced-reference image quality assessment , 2016, Inf. Sci..

[18]  Alexander Pollatsek,et al.  Eye Movements During Reading , 2008 .

[19]  Weisi Lin,et al.  Screen image quality assessment incorporating structural degradation measurement , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  Yonggang Wen,et al.  Toward Scalable Systems for Big Data Analytics: A Technology Tutorial , 2014, IEEE Access.

[21]  Min Zhang,et al.  Blind Image Quality Assessment Using the Joint Statistics of Generalized Local Binary Pattern , 2015, IEEE Signal Processing Letters.

[22]  King Ngi Ngan,et al.  Reduced-Reference Video Quality Assessment of Compressed Video Sequences , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Weisi Lin,et al.  Perceptual Quality Assessment of Screen Content Images , 2015, IEEE Transactions on Image Processing.

[24]  Weisi Lin,et al.  Learning a blind quality evaluation engine of screen content images , 2016, Neurocomputing.

[25]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[27]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[28]  Shuai Li,et al.  Depth Coding Based on Depth-Texture Motion and Structure Similarities , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[30]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[31]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[32]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Wen Gao,et al.  Subjective and Objective Quality Assessment of Compressed Screen Content Images , 2016, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[34]  Wen Gao,et al.  Nonlocal In-Loop Filter: The Way Toward Next-Generation Video Coding? , 2016, IEEE MultiMedia.

[35]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[36]  Guangming Shi,et al.  Blind image quality assessment with improved natural scene statistics model , 2016, Digit. Signal Process..

[37]  Kai Zeng,et al.  Objective Quality Assessment and Perceptual Compression of Screen Content Images , 2018, IEEE Computer Graphics and Applications.

[38]  Weisi Lin,et al.  BSD: Blind image quality assessment based on structural degradation , 2017, Neurocomputing.

[39]  Jizheng Xu,et al.  Overview of the Emerging HEVC Screen Content Coding Extension , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[41]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[42]  Weisi Lin,et al.  No-Reference Image Sharpness Assessment in Autoregressive Parameter Space , 2015, IEEE Transactions on Image Processing.

[43]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[44]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[45]  Jiaying Liu,et al.  Objective Quality Assessment of Screen Content Images by Uncertainty Weighting , 2017, IEEE Transactions on Image Processing.

[46]  Lei Zhang,et al.  Learning without Human Scores for Blind Image Quality Assessment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  M. Georgeson,et al.  Mach bands and multiscale models of spatial vision: the role of first, second, and third derivative operators in encoding bars and edges. , 2012, Journal of vision.

[48]  Shervin Minaee,et al.  Screen Content Image Segmentation Using Robust Regression and Sparse Decomposition , 2016, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[49]  Guangming Shi,et al.  Perceptual Quality Metric With Internal Generative Mechanism , 2013, IEEE Transactions on Image Processing.

[50]  Robert A. Frazor,et al.  Independence of luminance and contrast in natural scenes and in the early visual system , 2005, Nature Neuroscience.

[51]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[52]  H. Nothdurft Sensitivity for structure gradient in texture discrimination tasks , 1985, Vision Research.

[53]  Gangyi Jiang,et al.  Toward a Blind Quality Predictor for Screen Content Images , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[54]  Wen Gao,et al.  Reduced-Reference Quality Assessment of Screen Content Images , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  Mylène C. Q. Farias,et al.  No-reference image quality assessment based on statistics of Local Ternary Pattern , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[56]  Lei Zhang,et al.  Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features , 2014, IEEE Transactions on Image Processing.

[57]  Kai-Kuang Ma,et al.  Screen content image quality assessment using edge model , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[58]  Robert A. Frazor,et al.  Local luminance and contrast in natural images , 2006, Vision Research.

[59]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..