Deep multi-label learning for image distortion identification

Abstract Image Distortion Identification is important for image processing system enhancement, image distortion correction and image quality assessment. Although images may suffer various number of distortions while going through different systems, most of the previous researches of image distortion identification were focus on identifying single distortion in image. In this paper, we proposed a CNN-based multi-label learning model (called MLLNet) to identify distortions for different scenarios, including images having no distortion, single distortion and multiple distortions. Concretely, we transform the multi-label classification for image distortion identification to a number of multi-class classifications and use a deep multi-task CNN model to train all associated classifiers simultaneously. For unseen image, we use the trained CNN model to predict a number of classifications at same time and fuse them to final multi-label classification. The extensive experiments demonstrate that the propose algorithm can achieve good performance on several databases. Moreover, the network architecture of the CNN model can make flexible adjustment according to the different requirements.

[1]  Olac Fuentes,et al.  Removing JPEG blocking artifacts using machine learning , 2014, 2014 Southwest Symposium on Image Analysis and Interpretation.

[2]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[3]  Amanda Clare,et al.  Knowledge Discovery in Multi-label Phenotype Data , 2001, PKDD.

[4]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[5]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[6]  Aria Nosratinia.  Postprocessing of JPEG-2000 images to remove compression artifacts , 2003, IEEE Signal Processing Letters.

[7]  Nikolay N. Ponomarenko,et al.  Image database TID2013: Peculiarities, results and perspectives , 2015, Signal Process. Image Commun..

[8]  Weisi Lin,et al.  No-reference JPEG-2000 image quality metric , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[9]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[10]  Alan C. Bovik,et al.  Statistics of natural image distortions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  D. Chandler.  Seven Challenges in Image Quality Assessment: Past, Present, and Future Research , 2013 .

[12]  Zhengfang Duanmu,et al.  End-to-End Blind Image Quality Assessment Using Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[13]  Yuan Yan Tang,et al.  Simultaneous Spectral-Spatial Feature Selection and Extraction for Hyperspectral Images , 2019, IEEE Transactions on Cybernetics.

[14]  Ji Feng,et al.  Deep MIML Network , 2017, AAAI.

[15]  Yi Li,et al.  Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[16]  Gang Wang,et al.  Multi-Task CNN Model for Attribute Prediction , 2015, IEEE Transactions on Multimedia.

[17]  Mohamed A. Deriche,et al.  A hybrid system for distortion classification and image quality evaluation , 2012, Signal Process. Image Commun..

[18]  Mingxuan Sun,et al.  A Deep Multi-Modal CNN for Multi-Instance Multi-Label Image Classification , 2018, IEEE Transactions on Image Processing.

[19]  Jaime Lloret,et al.  Robust Image Hashing Based Efficient Authentication for Smart Industrial Environment , 2019, IEEE Transactions on Industrial Informatics.

[20]  Lina J. Karam,et al.  A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[21]  Wai-Yip Chan,et al.  Improving Robustness of Image Quality Measurement with Degradation Classification and Machine Learning , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[22]  Damon M. Chandler,et al.  Opinion-Unaware Blind Quality Assessment of Multiply and Singly Distorted Images via Distortion Parameter Estimation , 2018, IEEE Transactions on Image Processing.

[23]  Alan C. Bovik,et al.  No-reference quality assessment using natural scene statistics: JPEG2000 , 2005, IEEE Transactions on Image Processing.

[24]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[25]  Peng Peng,et al.  General-purpose image quality assessment based on distortion-aware decision fusion , 2014, Neurocomputing.

[26]  Alan C. Bovik,et al.  Objective quality assessment of multiply distorted images , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[27]  Jane You,et al.  Hyperspectral image unsupervised classification by robust manifold matrix factorization , 2019, Inf. Sci..

[28]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[29]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[30]  Lei Zhang,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models , 2017, IEEE Transactions on Image Processing.

[31]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[32]  Shan Suthaharan.  No-reference visually significant blocking artifact metric for natural scene images , 2009, Signal Process..

[33]  Grigorios Tsoumakas,et al.  Random K-labelsets for Multilabel Classification , 2022 .

[34]  Fen Chen,et al.  An Empirical Identification Method of Gaussian Blur Parameter for Image Deblurring , 2009, IEEE Transactions on Signal Processing.

[35]  Jie Fu,et al.  Distortion recognition for image quality assessment with convolutional neural network , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Xiang Zhu,et al.  A no-reference sharpness metric sensitive to blur and noise , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[38]  Xiaoming Liu,et al.  Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition , 2017, IEEE Transactions on Image Processing.

[39]  Zhou Wang,et al.  Local Phase Coherence and the Perception of Blur , 2003, NIPS.

[40]  G. Castellanos-Dominguez,et al.  Quantifying image distortion based on Gabor filter bank and multiple regression analysis , 2011, Electronic Imaging.

[41]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[42]  Yangqing Jia,et al.  Deep Convolutional Ranking for Multilabel Image Annotation , 2013, ICLR.

[43]  Fei Zhou,et al.  MDID: A multiply distorted image database for image quality assessment , 2017, Pattern Recognit..

[44]  Zhi-Hua Zhou,et al.  Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization , 2006, IEEE Transactions on Knowledge and Data Engineering.

[45]  Zhou Wang,et al.  No-reference perceptual quality assessment of JPEG compressed images , 2002, Proceedings. International Conference on Image Processing.

[46]  Fadi Al-Turjman,et al.  Secure data transmission framework for confidentiality in IoTs , 2019, Ad Hoc Networks.