Single Image De-Raining via Generative Adversarial Nets

In this paper, we propose a Generative Adversarial Network for Single Image De-raining(GAN-SID). We observe that batch normalization has side effects in the de-raining task. Therefore, we introduce instance normalization to replace the traditional batch normalization layers in both generator and discriminator. Motivated by the Squeeze-and-Excitation (SE) network that can learn the importance of channels, we introduce SE module in the generator to give different weights to the learned features. To preserve image details while removing rain streaks, we propose to utilize pixel-wise loss, perceptual loss, and adversarial loss to train the proposed network. Experiments on two synthetic datasets and real world images demonstrate that the proposed method outperforms state-of-the-art de-raining works in both objective and subjective measurements.

[1]  Vishal M. Patel,et al.  Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Xinghao Ding,et al.  Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal , 2016, IEEE Transactions on Image Processing.

[3]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[6]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[8]  Shuicheng Yan,et al.  Deep Joint Rain Detection and Removal from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Dacheng Tao,et al.  Perceptual Adversarial Networks for Image-to-Image Transformation , 2017, IEEE Transactions on Image Processing.

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Chul Lee,et al.  Single-image deraining using an adaptive nonlocal means filter , 2013, 2013 IEEE International Conference on Image Processing.

[14]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).