Authentically Distorted Image Quality Assessment by Learning From Empirical Score Distributions

Most existing works on image quality assessment (IQA) focus on predicting a scalar quality score (SQS) based on the assumption that people can reach a consensus on the judgment of image quality. However, assigning a single scalar fails to reveal the subjective diversity that an image will probably receive divergent opinion scores from different subjects. This is particularly true for real-world authentically distorted images which usually involve composite mixtures of multiple distortions. To characterize such an property, this letter proposes to use a more informative vectorized label called empirical score distribution (ESD) to build an ESD-aided deep neural network (DNN) for authentically distorted image quality prediction. Our proposed network contains two streams: ESD prediction stream and SQS prediction stream. The whole DNN is optimized end-to-end with a combined loss so that both of the supervision information from ESD and SQS can be fully utilized in the training process. Experiments on two public authentically distorted image databases verify the superiority of our method.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Rongrong Ji,et al.  Image Quality Assessment for Color Correction Based on Color Contrast Similarity and Color Value Difference , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[4]  Zhou Wang,et al.  dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs , 2017, IEEE Transactions on Image Processing.

[5]  Lei Zhang,et al.  A Probabilistic Quality Representation Approach to Deep Blind Image Quality Prediction , 2017, ArXiv.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Dietmar Saupe,et al.  KonIQ-10k: Towards an ecologically valid and large-scale IQA database , 2018, ArXiv.

[8]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[9]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[10]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[11]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[12]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[13]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[14]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[15]  Sebastian Bosse,et al.  A deep neural network for image quality assessment , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[16]  Lei Zhang,et al.  Blind Image Quality Assessment with a Probabilistic Quality Representation , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[17]  Weisi Lin,et al.  A Fast Reliable Image Quality Predictor by Fusing Micro- and Macro-Structures , 2017, IEEE Transactions on Industrial Electronics.

[18]  Yong Liu,et al.  Blind Image Quality Assessment Based on High Order Statistics Aggregation , 2016, IEEE Transactions on Image Processing.

[19]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[20]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[21]  Zhuo Chen,et al.  Unified No-Reference Quality Assessment of Singly and Multiply Distorted Stereoscopic Images , 2019, IEEE Transactions on Image Processing.

[22]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Wujie Zhou,et al.  Local gradient patterns (LGP): An effective local-statistical-feature extraction scheme for no-reference image quality assessment , 2017, Inf. Sci..

[26]  Dietmar Saupe,et al.  Deeprn: A Content Preserving Deep Architecture for Blind Image Quality Assessment , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[27]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[29]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[30]  Gangyi Jiang,et al.  Optimizing Multistage Discriminative Dictionaries for Blind Image Quality Assessment , 2018, IEEE Transactions on Multimedia.