A2Net: Adjacent Aggregation Networks for Image Raindrop Removal

Existing methods for single images raindrop removal either have poor robustness or suffer from parameter burdens. In this paper, we propose a new Adjacent Aggregation Network (A2Net) with lightweight architectures to remove raindrops from single images. Instead of directly cascading convolutional layers, we design an adjacent aggregation architecture to better fuse features for rich representations generation, which can lead to high quality images reconstruction. To further simplify the learning process, we utilize a problem-specific knowledge to force the network focus on the luminance channel in the YUV color space instead of all RGB channels. By combining adjacent aggregating operation with color space transformation, the proposed A2Net can achieve state-of-the-art performances on raindrop removal with significant parameters reduction.

[1]  Dani Lischinski,et al.  Joint Bi-layer Optimization for Single-Image Rain Streak Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Chang-Hwan Son,et al.  Rain removal via shrinkage of sparse codes and learned rain dictionary , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[3]  Wenhan Yang,et al.  Erase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Trevor Darrell,et al.  Deep Layer Aggregation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Soo-Chang Pei,et al.  Removing rain and snow in a single image using saturation and visibility features , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[7]  Jian Sun,et al.  ExFuse: Enhancing Feature Fusion for Semantic Segmentation , 2018, ECCV.

[8]  Atsushi Yamashita,et al.  Removal of adherent noises from images of dynamic scenes by using a pan-tilt camera , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[9]  Wenhan Yang,et al.  Attentive Generative Adversarial Network for Raindrop Removal from A Single Image , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Rob Fergus,et al.  Restoring an Image Taken through a Window Covered with Dirt or Rain , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Jing Xu,et al.  An Improved Guidance Image Based Method to Remove Rain and Snow in a Single Image , 2012, Comput. Inf. Sci..

[15]  Vishal M. Patel,et al.  Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Katsushi Ikeuchi,et al.  Adherent Raindrop Modeling, Detectionand Removal in Video , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Atsushi Yamashita,et al.  Removal of Adherent Waterdrops from Images Acquired with a Stereo Camera System , 2006, IEICE Trans. Inf. Syst..

[20]  M. Iqbal Saripan,et al.  Skin Segmentation Using YUV and RGB Color Spaces , 2014, J. Inf. Process. Syst..

[21]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[23]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[24]  P. S. Mohod,et al.  Removing snow from an image via image decomposition , 2013, 2013 IEEE International Conference ON Emerging Trends in Computing, Communication and Nanotechnology (ICECCN).

[25]  Chiou-Ting Hsu,et al.  A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Daniel Malacara,et al.  Color Vision and Colorimetry: Theory and Applications , 2002 .

[27]  Andreas Geiger,et al.  Video-based raindrop detection for improved image registration , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[28]  Wei Guo,et al.  Single-Image-Based Rain and Snow Removal Using Multi-guided Filter , 2013, ICONIP.

[29]  Atsushi Yamashita,et al.  A virtual wiper - restoration of deteriorated images by using multiple cameras , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[30]  Shuicheng Yan,et al.  Deep Joint Rain Detection and Removal from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Takeo Kanade,et al.  Analysis of Rain and Snow in Frequency Space , 2008, International Journal of Computer Vision.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  I. Ide,et al.  Rainy weather recognition from in-vehicle camera images for driver assistance , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[34]  Ngoc Thang Vu,et al.  Densely Connected Convolutional Networks for Speech Recognition , 2018, ITG Symposium on Speech Communication.

[35]  Yixin Chen,et al.  Deep Learning for Seeing Through Window With Raindrops , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[36]  Chenyuan Zhang,et al.  Motion robust rain detection and removal from videos , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[37]  Shuicheng Yan,et al.  Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Yinglong Wang,et al.  A Hierarchical Approach for Rain or Snow Removing in a Single Color Image. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[39]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Jérémie Bossu,et al.  Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks , 2011, International Journal of Computer Vision.

[41]  Andreas Geiger,et al.  Realistic Modeling of Water Droplets for Monocular Adherent Raindrop Recognition Using Bézier Curves , 2010, ACCV Workshops.

[42]  Jenq-Neng Hwang,et al.  Single Image Snow Removal via Composition Generative Adversarial Networks , 2019, IEEE Access.

[43]  Xinghao Ding,et al.  Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal , 2016, IEEE Transactions on Image Processing.

[44]  Dong Liu,et al.  High-Resolution Representations for Labeling Pixels and Regions , 2019, ArXiv.

[45]  Jenq-Neng Hwang,et al.  DesnowNet: Context-Aware Deep Network for Snow Removal , 2017, IEEE Transactions on Image Processing.

[46]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.