Tackling Multiple Visual Artifacts: Blind Image Restoration Using Conditional Adversarial Networks

Restoring images that are degraded by visual artifacts like noise, blurness and other environmental visual artifacts like shadow, snow, rain, and haze is a challenging task. From literature, it can be seen that there are many model-based as well as blind restoration methods that have been proposed to restore an image degraded by a single artifact. In most practical cases, the image is degraded by more than one artifact. Complexity arises while trying to estimate degradation function using conventional techniques where images are degraded by multiple visual artifacts. To the best of our knowledge, there has not been any generalized method proposed to tackle this problem. In this paper, we propose a methodology using conditional adversarial networks for blind image restoration of images that are degraded by multiple artifacts. To analyze the performance, ISTD dataset (meant originally for shadow removal) is used by augmenting it with different types of noises and blurness. The network has been trained on this data and has been analyzed how it behaves during the addition of each artifact. Various image quality metrics like Peak signal-to-noise ratio (PSNR), Mean squared error (MSE), Structural similarity index (SSIM), Blind/reference-less image spatial quality evaluator (BRISQUE) and Naturalness image quality evaluator (NIQE) have been evaluated to validate the performance of the proposed method.

[1]  Graham D. Finlayson,et al.  Fast Re-integration of Shadow Free Images , 2004, CIC.

[2]  Wolfram Burgard,et al.  The limits and potentials of deep learning for robotics , 2018, Int. J. Robotics Res..

[3]  Licheng Jiao,et al.  SAR Image Despeckling Using Bayesian Nonlocal Means Filter With Sigma Preselection , 2011, IEEE Geoscience and Remote Sensing Letters.

[4]  Arshwinder Kaur,et al.  Blind and Non- blind Image Restoration Techniques , 2013 .

[5]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[6]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[7]  Holger Ulmer,et al.  Unpaired High-Resolution and Scalable Style Transfer Using Generative Adversarial Networks , 2018, ArXiv.

[8]  Hongliang Ren,et al.  Glioma Prognosis: Segmentation of the Tumor and Survival Prediction Using Shape, Geometric and Clinical Information , 2018, BrainLes@MICCAI.

[9]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Aggelos K. Katsaggelos,et al.  Digital image restoration , 2012, IEEE Signal Process. Mag..

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Qing Zhang,et al.  Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization , 2015, IEEE Transactions on Image Processing.

[13]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[15]  Hongliang Ren,et al.  Ischemic Stroke Lesion Segmentation Using Adversarial Learning , 2018, BrainLes@MICCAI.

[16]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[17]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[19]  Raymond H. Chan,et al.  Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization , 2005, IEEE Transactions on Image Processing.

[20]  Le Hui,et al.  Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.