Encoding Distortions for Multi-task Full-Reference Image Quality Assessment

Most existing image quality assessment models focus on evaluating the image quality score, however, the quality score alone is not enough to characterize the degeneration. In this paper, we propose a full reference framework named Mask Gated Convolutional Network (MGCN) for evaluating the image quality score and identifying distortions simultaneously. Observing the fact that the reference images are distorted by various distortions in pixel space, we design an encoder module to capture the transformation between reference images and distorted images as low level features. Further higher level features are extracted from the low level features and shared by both the regression and the classification tasks. Instead of simply cropping patches to augment data, we mask the high level feature map in the spatial domain to randomly sample patches from the image and learn to assign the image quality score to the patch set. The proposed method achieves the state-of-the-art performance on LIVE2, TID2008 and TID2013 datasets.

[1]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[2]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[3]  Nikolay N. Ponomarenko,et al.  Color image database TID2013: Peculiarities and preliminary results , 2013, European Workshop on Visual Information Processing (EUVIP).

[4]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[5]  Sebastian Bosse,et al.  Neural network-based full-reference image quality assessment , 2016, 2016 Picture Coding Symposium (PCS).

[6]  Nanning Zheng,et al.  Image Quality Assessment Using Similar Scene as Reference , 2016, ECCV.

[7]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[9]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Roland Memisevic,et al.  Gradient-based learning of higher-order image features , 2011, 2011 International Conference on Computer Vision.

[12]  Afshin Dehghan,et al.  Who Do I Look Like? Determining Parent-Offspring Resemblance via Gated Autoencoders , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[14]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  Soo-Chang Pei,et al.  Image Quality Assessment Using Human Visual DOG Model Fused With Random Forest , 2015, IEEE Transactions on Image Processing.

[18]  Sanghoon Lee,et al.  Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yi Li,et al.  Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).