Naturalization Module in Neural Networks for Screen Content Image Quality Assessment

Deep learning approaches have demonstrated success in no-reference image quality assessment tasks. However, due to the specific properties of screen content images (SCIs), deep neural networks for SCI quality assessment are not as optimal as those designed for images depicting natural scenes. In order to tackle this discrepancy, a “naturalization” module composed of an upsampling layer and a convolutional layer is proposed to transform SCIs to have characteristics more similar to that of natural images. In addition, a new deep learning model architecture along with data augmentation techniques tailored to SCIs are implemented. The performance of the proposed approach is evaluated on the Screen Image Quality Assessment Database and Screen Content Image Database, and has shown to have superior performance to state-of-the-art methods in predicting the perceptual quality of SCIs.

[1]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[2]  Zhou Wang,et al.  Unified Blind Quality Assessment of Compressed Natural, Graphic, and Screen Content Images , 2017, IEEE Transactions on Image Processing.

[3]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[4]  Xuanqin Mou,et al.  Quality Assessment of Screen Content Images via Convolutional-Neural-Network-Based Synthetic/Natural Segmentation , 2018, IEEE Transactions on Image Processing.

[5]  Kai-Kuang Ma,et al.  A Gabor Feature-Based Quality Assessment Model for the Screen Content Images , 2018, IEEE Transactions on Image Processing.

[6]  Guorui Feng,et al.  Efficient Intra Mode Selection for Depth-Map Coding Utilizing Spatiotemporal, Inter-Component and Inter-View Correlations in 3D-HEVC , 2018, IEEE Transactions on Image Processing.

[7]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[8]  Ke Gu,et al.  Naturalization of Screen Content Images for Enhanced Quality Evaluation , 2017, IEICE Trans. Inf. Syst..

[9]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[10]  Sanghoon Lee,et al.  Fully Deep Blind Image Quality Predictor , 2017, IEEE Journal of Selected Topics in Signal Processing.

[11]  Weisi Lin,et al.  Perceptual Quality Assessment of Screen Content Images , 2015, IEEE Transactions on Image Processing.

[12]  Yong Li,et al.  Image quality assessment using deep convolutional networks , 2017 .

[13]  Benjamin Pfaff,et al.  Handbook Of Image And Video Processing , 2016 .

[14]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[16]  Kai-Kuang Ma,et al.  ESIM: Edge Similarity for Screen Content Image Quality Assessment , 2017, IEEE Transactions on Image Processing.

[17]  Leida Li,et al.  No Reference Quality Assessment for Screen Content Images With Both Local and Global Feature Representation , 2018, IEEE Transactions on Image Processing.

[18]  Joost van de Weijer,et al.  RankIQA: Learning from Rankings for No-Reference Image Quality Assessment , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Kai-Kuang Ma,et al.  Screen Content Image Quality Assessment Using Multi-Scale Difference of Gaussian , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[21]  Weisi Lin,et al.  Learning a blind quality evaluation engine of screen content images , 2016, Neurocomputing.

[22]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[23]  Weisi Lin,et al.  Saliency-Guided Quality Assessment of Screen Content Images , 2016, IEEE Transactions on Multimedia.

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  Jiaying Liu,et al.  Objective Quality Assessment of Screen Content Images by Uncertainty Weighting , 2017, IEEE Transactions on Image Processing.