Denoising Adversarial Networks for Rain Removal and Reflection Removal

This paper presents a novel adversarial scheme to perform image denoising for the tasks of rain streak removal and reflection removal. Similar to several previous works, the proposed method first estimates a prior image and then uses it to guide the inference of noise-free image. The novelty of our approach is to jointly learn the gradient and noise-free image based on an adversarial scheme. More specifically, we use the gradient map as the prior image. The inferred noise-free image guided by an estimated gradient is regarded as a negative sample, while the noise-free image guided by the ground truth of a gradient is taken as a positive sample. With the anchor defined by the ground truth of noise-free image, we play a min-max game to jointly train two optimizers for the estimation of the gradient and the inference of noise-free images. We show that both prior image and noise-free image can be accurately obtained under this adversarial scheme. Our state-of-the-art performance achieved on two public benchmark datasets validate the effectiveness of our approach.

[1]  Mohammed Ghanbari,et al.  Scope of validity of PSNR in image/video quality assessment , 2008 .

[2]  Wenhan Yang,et al.  Attentive Generative Adversarial Network for Raindrop Removal from A Single Image , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Frédo Durand,et al.  Reflection removal using ghosting cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Shuicheng Yan,et al.  Deep Joint Rain Detection and Removal from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sabine Süsstrunk,et al.  Single Image Reflection Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[10]  Ren Ng,et al.  Single Image Reflection Separation with Perceptual Losses , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Ling-Yu Duan,et al.  CRRN: Multi-scale Guided Concurrent Reflection Removal Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[12]  Wen Gao,et al.  Region-Aware Reflection Removal With Unified Content and Gradient Priors , 2018, IEEE Transactions on Image Processing.

[13]  Ling-Yu Duan,et al.  Benchmarking Single-Image Reflection Removal Algorithms , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yu Luo,et al.  Removing Rain from a Single Image via Discriminative Sparse Coding , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Liang Lin,et al.  Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining , 2018, ACM Multimedia.

[17]  Hongbin Zha,et al.  Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining , 2018, ECCV.

[18]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[20]  Yu-Chiang Frank Wang,et al.  Exploiting image structural similarity for single image rain removal , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[21]  Jiaolong Yang,et al.  A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing (Supplementary Material) , 2017 .

[22]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, ECCV.

[23]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[24]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[26]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.