Contrastive distortion‐level learning‐based no‐reference image‐quality assessment

A contrastive distortion‐level learning‐based no‐reference image‐quality assessment (NR‐IQA) framework is proposed in this study to further effectively model various distortion types with the same or different distortion levels. The proposed method aims to improve the prediction accuracy of NR‐IQA. The proposed method consists of three parts: multiscale distortion‐level representation learning, single‐image NR‐IQA, and a representation affinity module, which can reduce NR‐IQA computational complexity while maintaining a low‐distortion representation of high‐distortion inputs. The proposed NR‐IQA method aims to extract distributional features of samples in real distorted images and predict ambiguity based on distortion‐level learning. Experimental results show that by comparing on many NR‐IQA data sets the proposed method can outperform state‐of‐the‐art methods.

[1]  Ximeng Liu,et al.  GATrust: A Multi-Aspect Graph Attention Network Model for Trust Assessment in OSNs , 2023, IEEE Transactions on Knowledge and Data Engineering.

[2]  X.Q. Yuan,et al.  Attacking Deep Reinforcement Learning With Decoupled Adversarial Policy , 2023, IEEE Transactions on Dependable and Secure Computing.

[3]  Tianqing Zhu,et al.  The Dynamic Privacy-Preserving Mechanisms for Online Dynamic Social Networks , 2022, IEEE Transactions on Knowledge and Data Engineering.

[4]  Changyu Dong,et al.  MAS-Encryption and its Applications in Privacy-Preserving Classifiers , 2022, IEEE Transactions on Knowledge and Data Engineering.

[5]  Zuoyong Li,et al.  Generative Adversarial Training for Supervised and Semi-supervised Learning , 2022, Frontiers in Neurorobotics.

[6]  Tong Li,et al.  Efficient and Secure Outsourcing of Differentially Private Data Publishing With Multiple Evaluators , 2022, IEEE Transactions on Dependable and Secure Computing.

[7]  Alan C. Bovik,et al.  Image Quality Assessment Using Contrastive Learning , 2021, IEEE Transactions on Image Processing.

[8]  Xinfeng Zhang,et al.  Quality Assessment of End-to-End Learned Image Compression: The Benchmark and Objective Measure , 2021, ACM Multimedia.

[9]  Junyong You,et al.  Long Short-term Convolutional Transformer for No-Reference Video Quality Assessment , 2021, ACM Multimedia.

[10]  Chixiao Chen,et al.  TSA-Net: Tube Self-Attention Network for Action Quality Assessment , 2021, ACM Multimedia.

[11]  King Ngi Ngan,et al.  Remember and Reuse: Cross-Task Blind Image Quality Assessment via Relevance-aware Incremental Learning , 2021, ACM Multimedia.

[12]  Yuming Fang,et al.  Image Quality Assessment in the Modern Age , 2021, ACM Multimedia.

[13]  Hongfei Fan,et al.  PUGCQ: A Large Scale Dataset for Quality Assessment of Professional User-Generated Content , 2021, ACM Multimedia.

[14]  Guangming Shi,et al.  Image Quality Caption with Attentive and Recurrent Semantic Attractor Network , 2021, ACM Multimedia.

[15]  Zhiwei Xiong,et al.  Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN , 2021, ACM Multimedia.

[16]  Keyan Ding,et al.  Locally Adaptive Structure and Texture Similarity for Image Quality Assessment , 2021, ACM Multimedia.

[17]  Phillip Isola,et al.  Contrastive Feature Loss for Image Prediction , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[18]  Manuel López Martín,et al.  Supervised contrastive learning over prototype-label embeddings for network intrusion detection , 2021, Inf. Fusion.

[19]  Lanjiang. Wang,et al.  A survey on IQA , 2021, ArXiv.

[20]  Kris M. Kitani,et al.  No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[21]  Xianmin Wang,et al.  Oblivious Transfer for Privacy-Preserving in VANET’s Feature Matching , 2021, IEEE Transactions on Intelligent Transportation Systems.

[22]  Dayong Ye,et al.  Resource Allocation in IoT Edge Computing via Concurrent Federated Reinforcement Learning , 2021, IEEE Internet of Things Journal.

[23]  Manri Cheon,et al.  Perceptual Image Quality Assessment with Transformers , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Danqi Chen,et al.  SimCSE: Simple Contrastive Learning of Sentence Embeddings , 2021, EMNLP.

[25]  Wei An,et al.  Unsupervised Degradation Representation Learning for Blind Super-Resolution , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Beliz Gunel,et al.  Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning , 2020, ICLR.

[27]  Tingting Jiang,et al.  Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment , 2020, ACM Multimedia.

[28]  Teng Yu,et al.  Full-reference Screen Content Image Quality Assessment by Fusing Multilevel Structure Similarity , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[29]  Yu Zhu,et al.  Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Xiaokang Yang,et al.  Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild , 2020, IEEE Transactions on Image Processing.

[31]  Ci Wang,et al.  Blind Quality Assessment for Multiply-Distorted Images via Distortion Decomposition , 2020, ICMSSP.

[32]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[33]  Eero P. Simoncelli,et al.  Image Quality Assessment: Unifying Structure and Texture Similarity , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Guangming Shi,et al.  MetaIQA: Deep Meta-Learning for No-Reference Image Quality Assessment , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[36]  Tong Li,et al.  NPMML: A Framework for Non-Interactive Privacy-Preserving Multi-Party Machine Learning , 2020, IEEE Transactions on Dependable and Secure Computing.

[37]  Praful Gupta,et al.  From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Yu Zhang,et al.  Cross-Reference Stitching Quality Assessment for 360° Omnidirectional Images , 2019, ACM Multimedia.

[40]  Weisi Lin,et al.  SGDNet: An End-to-End Saliency-Guided Deep Neural Network for No-Reference Image Quality Assessment , 2019, ACM Multimedia.

[41]  Ming Jiang,et al.  Quality Assessment of In-the-Wild Videos , 2019, ACM Multimedia.

[42]  Jin Li,et al.  The security of machine learning in an adversarial setting: A survey , 2019, J. Parallel Distributed Comput..

[43]  Zhou Wang,et al.  Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Shiqi Wang,et al.  Intrinsic Image Popularity Assessment , 2019, ACM Multimedia.

[45]  Liquan Shen,et al.  No-Reference Stereoscopic Image Quality Assessment Based on Image Distortion and Stereo Perceptual Information , 2019, IEEE Transactions on Emerging Topics in Computational Intelligence.

[46]  Wenjing Lou,et al.  Searchable Symmetric Encryption with Forward Search Privacy , 2019, IEEE Transactions on Dependable and Secure Computing.

[47]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[48]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[49]  Hossein Mobahi,et al.  Large Margin Deep Networks for Classification , 2018, NeurIPS.

[50]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[51]  Ming Jiang,et al.  Exploiting High-Level Semantics for No-Reference Image Quality Assessment of Realistic Blur Images , 2017, ACM Multimedia.

[52]  Cheng-Hsin Hsu,et al.  Is Foveated Rendering Perceivable in Virtual Reality?: Exploring the Efficiency and Consistency of Quality Assessment Methods , 2017, ACM Multimedia.

[53]  Guangming Shi,et al.  Hierarchical Feature Degradation Based Blind Image Quality Assessment , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[54]  Sanghoon Lee,et al.  Fully Deep Blind Image Quality Predictor , 2017, IEEE Journal of Selected Topics in Signal Processing.

[55]  Sebastian Bosse,et al.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment , 2016, IEEE Transactions on Image Processing.

[56]  Lei Zhang,et al.  A Feature-Enriched Completely Blind Image Quality Evaluator , 2015, IEEE Transactions on Image Processing.

[57]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[58]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[59]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[60]  Xinbo Gao,et al.  MaD-DLS: Mean and Deviation of Deep and Local Similarity for Image Quality Assessment , 2021, IEEE Transactions on Multimedia.

[61]  Reproducibility Summary.  Reproducibility Report: Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness , 2021 .