Single image deraining using deep convolutional networks

A deep learning-based single image deraining algorithm is proposed in this work. Instead of modeling a rain layer as a linear function between the rain image and its clear version as previous works do, we directly formulate the clear image as the result of a non-linear mapping of thrain image. We construct a coarse deraining convolutional network and a refinement convolutional network to learn this non-linear mapping function. The coarse deraining network is trained to detect the rain streaks with different directions, and restore a raw derained result. The refinement network aims at refining the result according to the raw derained image and the original rain image. By combining the two networks, we are able to well-restore the rain-free image. Experimental results demonstrate that the proposed deraining method can produce high-quality clear images from both synthetic and real-world rain images, outperforming the state-of-the-art methods qualitatively and quantitatively.

[1]  Jérémie Bossu,et al.  Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks , 2011, International Journal of Computer Vision.

[2]  S. Nayar,et al.  Detection and removal of rain from videos , 2004, CVPR 2004.

[3]  Hao Li,et al.  Rain Removal in Video by Combining Temporal and Chromatic Properties , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[4]  Shree K. Nayar,et al.  When does a camera see rain? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[6]  Sudipta Mukhopadhyay,et al.  Removal of rain from videos: a review , 2014, Signal Image Video Process..

[7]  Yu-Hsiang Fu,et al.  Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition , 2012, IEEE Transactions on Image Processing.

[8]  K Garg,et al.  DETECTION AND REMOVAL OF RAIN FROM VIDEOS IN COMPUTER VISION AND PATTERN RECOGNITION , 2004 .

[9]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[10]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Chul Lee,et al.  Single-image deraining using an adaptive nonlocal means filter , 2013, 2013 IEEE International Conference on Image Processing.

[14]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[15]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[16]  Shree K. Nayar,et al.  Vision and Rain , 2007, International Journal of Computer Vision.

[17]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Seunghoon Hong,et al.  Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network , 2015, ICML.

[19]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[20]  Yu Luo,et al.  Removing Rain from a Single Image via Discriminative Sparse Coding , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jonathan T. Barron,et al.  Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Mohammed Ghanbari,et al.  Scope of validity of PSNR in image/video quality assessment , 2008 .