Distortion-Independent Pairwise Underwater Image Perceptual Quality Comparison

Ranking underwater images according to their quality is a key indicator comparing the performance of different methodologies and therefore is critical in the field of instrumentation and measurement. Perceiving differences in the quality of underwater images is a challenging task that has received relatively less attention, primarily due to mixed distortion, diversified image content, and the absence of high-quality reference images. Thus, based on the pairwise underwater image quality voting data, we develop a novel ternary classification transformer to predict a quality comparison of underwater images without reference. This is the first attempt to model the quality discrimination of an image pair. The proposed model combines the perception of convolutional neural networks and transformer encoder to explore local quality features and visual perceptual connections between different patch tokens. Experimental results reveal that the proposed pairwise underwater image quality comparison (PUIQC) scheme predicts noticeable quality differences correlating well with subjective perception. The quantification of complex distortions in underwater images compared to other learning-based methods is a compelling feature of this technology. It delivers competitive results in ranking the different enhancement outputs. In addition, we reveal the self-attention of local quality features within the two images and capture their responsive contribution to the quality decision, which explains the underlying subjective quality-sensitive mechanism during image quality comparison.

[1]  Q. Jiang,et al.  Underwater Image Enhancement Quality Evaluation: Benchmark Dataset and Objective Metric , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Bing Zheng,et al.  A Underwater Sequence Image Dataset for Sharpness and Color Analysis , 2022, Sensors.

[3]  Miao Yang,et al.  Multitopic underwater image quality assessment with visual attention factors , 2022, Journal of Electronic Imaging.

[4]  Xinjiao Chen,et al.  Saliency-Guided Transformer Network combined with Local Embedding for No-Reference Image Quality Assessment , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[5]  Miao Yang,et al.  Pair comparison based progressive subjective quality ranking for underwater images , 2021, Signal Process. Image Commun..

[6]  Kris M. Kitani,et al.  No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[7]  Peyman Milanfar,et al.  MUSIQ: Multi-scale Image Quality Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Peixian Zhuang,et al.  DewaterNet: A fusion adversarial real underwater image enhancement network , 2021, Signal Process. Image Commun..

[9]  Junyong You,et al.  Transformer For Image Quality Assessment , 2020, 2021 IEEE International Conference on Image Processing (ICIP).

[10]  Tingting Jiang,et al.  Unified Quality Assessment of in-the-Wild Videos with Mixed Datasets Training , 2020, Int. J. Comput. Vis..

[11]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[12]  Changli Li,et al.  Simple Estimation of Red Channel’s Transmittance and Balanced Color Correction for Underwater Image Enhancement , 2020, 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[13]  Junyu Dong,et al.  Detection perceptual underwater image enhancement with deep learning and physical priors , 2020, ArXiv.

[14]  Xiaokang Yang,et al.  Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild , 2020, IEEE Transactions on Image Processing.

[15]  Guangtao Zhai,et al.  Unsupervised Blind Image Quality Evaluation via Statistical Measurements of Structure, Naturalness, and Perception , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Bing Zheng,et al.  Offshore Underwater Image Restoration Using Reflection-Decomposition-Based Transmission Map Estimation , 2020, IEEE Journal of Oceanic Engineering.

[17]  Junaed Sattar,et al.  Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception , 2020, Robotics: Science and Systems.

[18]  Zhiqiang Wei,et al.  Underwater image enhancement based on conditional generative adversarial network , 2020, Signal Process. Image Commun..

[19]  Fatih Murat Porikli,et al.  Underwater scene prior inspired deep underwater image and video enhancement , 2020, Pattern Recognit..

[20]  Dietmar Saupe,et al.  KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment , 2019, IEEE Transactions on Image Processing.

[21]  Ke Hu,et al.  An In-Depth Survey of Underwater Image Enhancement and Restoration , 2019, IEEE Access.

[22]  Chunping Hou,et al.  Effective and Efficient Blind Quality Evaluator for Contrast Distorted Images , 2019, IEEE Transactions on Instrumentation and Measurement.

[23]  Zhou Wang,et al.  Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Yongdong Zhang,et al.  Context-Aware Visual Policy Network for Fine-Grained Image Captioning , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Vlad Hosu,et al.  KADID-10k: A Large-scale Artificially Distorted IQA Database , 2019, 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).

[26]  Junaed Sattar,et al.  Fast Underwater Image Enhancement for Improved Visual Perception , 2019, IEEE Robotics and Automation Letters.

[27]  Bo Yan,et al.  Naturalness-Aware Deep No-Reference Image Quality Assessment , 2019, IEEE Transactions on Multimedia.

[28]  Gangyi Jiang,et al.  Optimizing Multistage Discriminative Dictionaries for Blind Image Quality Assessment , 2018, IEEE Transactions on Multimedia.

[29]  Dan Mikami,et al.  Image quality assessment for inpainted images via learning to rank , 2018, Multimedia Tools and Applications.

[30]  Hong Cai,et al.  PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[32]  Joost van de Weijer,et al.  RankIQA: Learning from Rankings for No-Reference Image Quality Assessment , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Sanghoon Lee,et al.  Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Runmin Cong,et al.  A hybrid method for underwater image correction , 2017, Pattern Recognit. Lett..

[35]  Kin-Man Lam,et al.  The OUC-vision large-scale underwater image database , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[36]  Zhou Wang,et al.  dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs , 2017, IEEE Transactions on Image Processing.

[37]  Pamela C. Cosman,et al.  Underwater Image Restoration Based on Image Blurriness and Light Absorption , 2017, IEEE Transactions on Image Processing.

[38]  Sanghoon Lee,et al.  Fully Deep Blind Image Quality Predictor , 2017, IEEE Journal of Selected Topics in Signal Processing.

[39]  King Ngi Ngan,et al.  No-Reference Retargeted Image Quality Assessment Based on Pairwise Rank Learning , 2016, IEEE Transactions on Multimedia.

[40]  Lai-Man Po,et al.  No-reference image quality assessment with deep convolutional neural networks , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[41]  Chen Gao,et al.  Human-Visual-System-Inspired Underwater Image Quality Measures , 2016, IEEE Journal of Oceanic Engineering.

[42]  King Ngi Ngan,et al.  Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Arcot Sowmya,et al.  An Underwater Color Image Quality Evaluation Metric , 2015, IEEE Transactions on Image Processing.

[44]  Xiao-Ping Zhang,et al.  A retinex-based enhancing approach for single underwater image , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[45]  Xuelong Li,et al.  Learning to Rank for Blind Image Quality Assessment , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Nikolay N. Ponomarenko,et al.  Color image database TID2013: Peculiarities and preliminary results , 2013, European Workshop on Visual Information Processing (EUVIP).

[47]  Carlos Gershenson,et al.  Information and Computation , 2013, Handbook of Human Computation.

[48]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[49]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[50]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[52]  Christophe Charrier,et al.  A DCT Statistics-Based Blind Image Quality Index , 2010, IEEE Signal Processing Letters.

[53]  Tao Qin,et al.  FRank: a ranking method with fidelity loss , 2007, SIGIR.

[54]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[55]  Fabrizio Russo,et al.  Automatic enhancement of noisy images using objective evaluation of image quality , 2005, IEEE Transactions on Instrumentation and Measurement.

[56]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[57]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[58]  Zhi Zheng,et al.  HVS-Based Perception-Driven No-Reference Omnidirectional Image Quality Assessment , 2023, IEEE Transactions on Instrumentation and Measurement.

[59]  Leida Li,et al.  Study of Natural Scene Categories in Measurement of Perceived Image Quality , 2022, IEEE Transactions on Instrumentation and Measurement.

[60]  Zhaoqing Pan,et al.  Channel Recombination and Projection Network for Blind Image Quality Measurement , 2022, IEEE Transactions on Instrumentation and Measurement.

[61]  Xiaoming Chen,et al.  DeLFIQE—A Low-Complexity Deep Learning-Based Light Field Image Quality Evaluator , 2021, IEEE Transactions on Instrumentation and Measurement.

[62]  Guanghui Yue,et al.  Blind Omnidirectional Image Quality Assessment Based on Structure and Natural Features , 2021, IEEE Transactions on Instrumentation and Measurement.

[63]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[64]  A. Bovik,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[65]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..

[66]  Eric C. Larson,et al.  Most apparent distortion: full-reference image quality assessment and the role of strategy , 2010, J. Electronic Imaging.

[67]  Alessio De Angelis,et al.  A Vector Approach for Image Quality Assessment and Some Metrological Considerations , 2009, IEEE Transactions on Instrumentation and Measurement.