Cycle-Spinning GAN for Raindrop Removal from Images

Weather events such as rain, snow, and fog degrade the quality of images taken under these conditions. Enhancement of such images is critical for intelligent transport and outdoor surveillance systems. Generative Adversarial Networks (GAN) based methods have been shown to be promising for enhancing these images in recent years. In this study, we adapt the cycle-spinning technique to GAN for removal of raindrops. The experimental evaluation of the proposed method shows that the performance is improved in terms of reference-based metrics (SSIM and PSNR). In addition, the approach also results in higher object detection performance in terms of mean average precision (mAP) metric when applied before the detection process.

[1]  David Hurych,et al.  Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving , 2019, Autonomous Vehicles and Machines.

[2]  Fei Xia,et al.  Understanding GANs: the LQG Setting , 2017, ArXiv.

[3]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[4]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Thomas B. Moeslund,et al.  Rain Removal in Traffic Surveillance: Does it Matter? , 2018, IEEE Transactions on Intelligent Transportation Systems.

[6]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Atsushi Yamashita,et al.  Removal of adherent waterdrops from images acquired with stereo camera , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[12]  Shree K. Nayar,et al.  Adaptive dynamic range imaging: optical control of pixel exposures over space and time , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[14]  Wenhan Yang,et al.  Attentive Generative Adversarial Network for Raindrop Removal from A Single Image , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Edward F. Kelley,et al.  Simulated-eye-design camera for high-contrast measurements , 1999, Electronic Imaging.

[16]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Katsushi Ikeuchi,et al.  Adherent Raindrop Detection and Removal in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[19]  Sakinah Ali Pitchay,et al.  A REVIEW OF RAIN STREAKS DETECTION AND REMOVAL TECHNIQUES FOR OUTDOOR SINGLE IMAGE Samer , 2016 .

[20]  Alptekin Temizel,et al.  Wavelet domain image resolution enhancement using cycle-spinning , 2005 .

[21]  Shree K. Nayar,et al.  Detection and removal of rain from videos , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[22]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[23]  Lei Wang,et al.  Single-Image De-Raining With Feature-Supervised Generative Adversarial Network , 2019, IEEE Signal Processing Letters.

[24]  N. BharathRaj,et al.  Single Image Haze Removal using a Generative Adversarial Network , 2018, ArXiv.

[25]  Hideo Saito,et al.  Removal of Glare Caused by Water Droplets , 2009 .

[26]  Atsushi Yamashita,et al.  Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[28]  Qiaojing Yan.  DCGANs for image super-resolution , denoising and debluring , 2017 .

[29]  Xiaochun Cao,et al.  Single Image Deraining: A Comprehensive Benchmark Analysis , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).