Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild

Performance of blind image quality assessment (BIQA) models has been significantly boosted by end-to-end optimization of feature engineering and quality regression. Nevertheless, due to the distributional shifts between images simulated in the laboratory and captured in the wild, models trained on databases with synthetic distortions remain particularly weak at handling realistic distortions (and vice versa). To confront the cross-distortion-scenario challenge, we develop a unified BIQA model and an effective approach of training it for both synthetic and realistic distortions. We first sample pairs of images from the same IQA databases and compute a probability that one image of each pair is of higher quality as the supervisory signal. We then employ the fidelity loss to optimize a deep neural network for BIQA over a large number of such image pairs. We also explicitly enforce a hinge constraint to regularize uncertainty estimation during optimization. Extensive experiments on six IQA databases show the promise of the learned method in blindly assessing image quality in the laboratory and wild. In addition, we demonstrate the universality of the proposed training strategy by using it to improve existing BIQA models.

[1]  Rongxin Jiang,et al.  Convolutional neural network with uncertainty estimates for no-reference image quality assessment , 2019, International Conference on Graphic and Image Processing.

[2]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[3]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[4]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[5]  María Pérez-Ortiz,et al.  From Pairwise Comparisons and Rating to a Unified Quality Scale , 2020, IEEE Transactions on Image Processing.

[6]  Praful Gupta,et al.  From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Kede Ma,et al.  Perceptual Quality Assessment of Smartphone Photography , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lei Zhang,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models , 2017, IEEE Transactions on Image Processing.

[9]  Shiqi Wang,et al.  Image Quality Assessment: Unifying Structure and Texture Similarity , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Dietmar Saupe,et al.  KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment , 2019, IEEE Transactions on Image Processing.

[12]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[13]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[15]  Xiaokang Yang,et al.  Learning To Blindly Assess Image Quality In The Laboratory And Wild , 2019, 2020 IEEE International Conference on Image Processing (ICIP).

[16]  Zhou Wang,et al.  dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs , 2017, IEEE Transactions on Image Processing.

[17]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[18]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[19]  Valero Laparra,et al.  Perceptually Optimized Image Rendering , 2017, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[21]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Alexandre G. Ciancio,et al.  No-Reference Blur Assessment of Digital Pictures Based on Multifeature Classifiers , 2011, IEEE Transactions on Image Processing.

[25]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[26]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Lei Zhang,et al.  Blind Image Quality Assessment with a Probabilistic Quality Representation , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[28]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[29]  Yong Liu,et al.  Blind Image Quality Assessment Based on High Order Statistics Aggregation , 2016, IEEE Transactions on Image Processing.

[30]  Kede Ma,et al.  Active Fine-Tuning from gMAD Examples Improves Blind Image Quality Assessment , 2020, ArXiv.

[31]  Zhou Wang,et al.  Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Joost van de Weijer,et al.  RankIQA: Learning from Rankings for No-Reference Image Quality Assessment , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Qingbo Wu,et al.  Blind Image Quality Assessment Using Local Consistency Aware Retriever and Uncertainty Aware Evaluator , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Vlad Hosu,et al.  KADID-10k: A Large-scale Artificially Distorted IQA Database , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[36]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[37]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[38]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[39]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Zhengfang Duanmu,et al.  Group Maximum Differentiation Competition: Model Comparison with Few Samples , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  N. D. Birrell,et al.  Quantum fields in curved space , 2007 .

[42]  María Pérez-Ortiz,et al.  Psychometric scaling of TID2013 dataset , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[43]  Xuelong Li,et al.  Learning to Rank for Blind Image Quality Assessment , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[44]  L. Thurstone A law of comparative judgment. , 1994 .

[45]  Tao Qin,et al.  FRank: a ranking method with fidelity loss , 2007, SIGIR.

[46]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[47]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[48]  Eero P. Simoncelli,et al.  Blind Image Quality Assessment by Learning from Multiple Annotators , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[49]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[50]  Stefan Winkler,et al.  Perceptual blur and ringing metrics: application to JPEG2000 , 2004, Signal Process. Image Commun..