Blind Image Quality Assessment via Vector Regression and Object Oriented Pooling

This paper presents an effective method based on vector regression and object oriented pooling for blind image quality assessment. Unlike previous models that map the extracted features directly to a quality score, the proposed vector regression framework yields a vector of belief scores for the input image. We explore the uncertainty factors in quality assessment and design the belief scores to measure the confidences of an image to be assigned to the corresponding quality grades. Moreover, we propose an object oriented pooling strategy to further improve the performance by incorporating semantic information of image contents. According to this strategy, regions occupied by objects will be assigned more weights in the pooling phase, leading to a more accurate quality assessment. Extensive experiments on benchmark datasets demonstrate that our approach achieves state-of-the-art performance and shows a great generalization ability.

[1]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Judith Redi,et al.  Supporting visual quality assessment with machine learning , 2013, EURASIP Journal on Image and Video Processing.

[3]  Dit-Yan Yeung,et al.  Learning a Deep Compact Image Representation for Visual Tracking , 2013, NIPS.

[4]  Xuelong Li,et al.  Blind Image Quality Assessment via Deep Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[6]  Ashish Kapoor,et al.  Learning a blind measure of perceptual image quality , 2011, CVPR 2011.

[7]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[8]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[9]  Ulrich Engelke,et al.  Visual Attention in Quality Assessment , 2011, IEEE Signal Processing Magazine.

[10]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[11]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[14]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[15]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[16]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[18]  Xinbo Gao,et al.  Saliency-Guided Deep Framework for Image Quality Assessment , 2015, IEEE MultiMedia.

[19]  Thomas Deselaers,et al.  Measuring the Objectness of Image Windows , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Patrick Le Callet,et al.  Subjective quality assessment IRCCyN/IVC database , 2004 .

[22]  Fan Zhang,et al.  Image Quality Assessment by Separately Evaluating Detail Losses and Additive Impairments , 2011, IEEE Transactions on Multimedia.

[23]  Peter G. Engeldrum,et al.  Psychometric Scaling: A Toolkit for Imaging Systems Development , 2000 .

[24]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[25]  Judith Redi,et al.  Interactions of visual attention and quality perception , 2011, Electronic Imaging.

[26]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[27]  David Zhang,et al.  Robust Visual Knowledge Transfer via Extreme Learning Machine-Based Domain Adaptation , 2016, IEEE Transactions on Image Processing.

[28]  Radomír Mech,et al.  Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[30]  Lina J. Karam,et al.  Reduced-Reference Quality Assessment Based on the Entropy of DWT Coefficients of Locally Weighted Gradient Magnitudes , 2016, IEEE Transactions on Image Processing.

[31]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[34]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  David Zhang,et al.  LSDT: Latent Sparse Domain Transfer Learning for Visual Adaptation , 2016, IEEE Transactions on Image Processing.

[36]  Alan C. Bovik,et al.  Blind image quality assessment on real distorted images using deep belief nets , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[37]  Weisi Lin,et al.  Image Quality Assessment Based on Gradient Similarity , 2012, IEEE Transactions on Image Processing.

[38]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[39]  Xuelong Li,et al.  Sparse representation for blind image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Lei Zhang,et al.  Learning without Human Scores for Blind Image Quality Assessment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[42]  Alan C. Bovik,et al.  Automatic Prediction of Perceptual Image and Video Quality , 2013, Proceedings of the IEEE.

[43]  King Ngi Ngan,et al.  Blind proposal quality assessment via deep objectness representation and local linear regression , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[44]  David S. Doermann,et al.  Beyond Human Opinion Scores: Blind Image Quality Assessment Based on Synthetic Scores , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  King Ngi Ngan,et al.  No-Reference Retargeted Image Quality Assessment Based on Pairwise Rank Learning , 2016, IEEE Transactions on Multimedia.

[46]  Philip H. S. Torr,et al.  BING: Binarized normed gradients for objectness estimation at 300fps , 2019, Computational Visual Media.

[47]  한보형,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015 .

[48]  Vladlen Koltun,et al.  Geodesic Object Proposals , 2014, ECCV.

[49]  David S. Doermann,et al.  No-Reference Image Quality Assessment Using Visual Codebooks , 2012, IEEE Transactions on Image Processing.

[50]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[51]  Yuukou Horita,et al.  Impact of subjective dataset on the performance of image quality metrics , 2008, 2008 15th IEEE International Conference on Image Processing.

[52]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[53]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[54]  Abdul Rehman,et al.  Reduced-Reference Image Quality Assessment by Structural Similarity Estimation , 2012, IEEE Transactions on Image Processing.

[55]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[56]  Ashish Kapoor,et al.  Blind Image Quality Assessment Using Semi-supervised Rectifier Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[58]  Peng Zhang,et al.  SOM: Semantic obviousness metric for image quality assessment , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[60]  Mikko Nuutinen,et al.  CID2013: A Database for Evaluating No-Reference Image Quality Assessment Algorithms , 2015, IEEE Transactions on Image Processing.

[61]  Q. M. Jonathan Wu,et al.  Utilizing Image Scales Towards Totally Training Free Blind Image Quality Assessment , 2015, IEEE Transactions on Image Processing.

[62]  Fan Zhang,et al.  Reduced-Reference Image Quality Assessment Using Reorganized DCT-Based Image Representation , 2011, IEEE Transactions on Multimedia.

[63]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[64]  King Ngi Ngan,et al.  Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer , 2016, IEEE Transactions on Circuits and Systems for Video Technology.