An L0-regularized global anisotropic gradient prior for single-image de-raining

Abstract We propose a method based on a global anisotropic gradient prior (GAGP) for addressing the problem of rain streak removal. We observe that both images and rain streaks have anisotropy at edges, and that the rain distribution in images is sparse. We therefore propose an L0-regularized sparse term with an L0-regularized GAGP term to efficiently detect the rain streaks. We applied the L0-regularized GAGP as the image regularization term to protect the image structure and details. We also developed an alternating half-quadratic algorithm to solve the proposed L0 optimization model by introducing the variable splitting method and analyzing the convergence of the algorithm. Experimental results show that the proposed method outperforms state-of-the-art methods in rain streak removal and preserving image structure on both synthesized and real-world images with different levels and types of rain.

[1]  Dani Lischinski,et al.  Joint Bi-layer Optimization for Single-Image Rain Streak Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Jie Li,et al.  Single image deraining via decorrelating the rain streaks and background scene in gradient domain , 2018, Pattern Recognit..

[3]  Yu Luo,et al.  Removing Rain from a Single Image via Discriminative Sparse Coding , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Xi-Le Zhao,et al.  Video deraining via nonlocal low-rank regularization , 2020, Applied Mathematical Modelling.

[5]  Shree K. Nayar,et al.  Contrast Restoration of Weather Degraded Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Li Xu,et al.  Unnatural L0 Sparse Representation for Natural Image Deblurring , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  S. Nayar,et al.  Photorealistic rendering of rain streaks , 2006, SIGGRAPH 2006.

[8]  Ting-Zhu Huang,et al.  A directional global sparse model for single image rain removal , 2018, Applied Mathematical Modelling.

[9]  Nianjun Liu,et al.  Using the Shape Characteristics of Rain to Identify and Remove Rain from Video , 2008, SSPR/SPR.

[10]  Jie Chen,et al.  Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Soo-Chang Pei,et al.  Removing rain and snow in a single image using saturation and visibility features , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[12]  Chul Lee,et al.  Single-image deraining using an adaptive nonlocal means filter , 2013, 2013 IEEE International Conference on Image Processing.

[13]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[14]  Delu Zeng,et al.  Removing Rain from Single Images via a Deep Detail Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Cewu Lu,et al.  Image smoothing via L0 gradient minimization , 2011, ACM Trans. Graph..

[16]  Ting-Zhu Huang,et al.  FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors , 2018, IEEE Transactions on Image Processing.

[17]  Yang Song,et al.  Split Bregmanized anisotropic total variation model for image deblurring , 2015, J. Vis. Commun. Image Represent..

[18]  Jian Chen,et al.  Learning Bilevel Layer Priors for Single Image Rain Streaks Removal , 2019, IEEE Signal Processing Letters.

[19]  Jérémie Bossu,et al.  Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks , 2011, International Journal of Computer Vision.

[20]  Li-Wei Kang,et al.  Self-Learning Based Image Decomposition With Applications to Single Image Denoising , 2014, IEEE Transactions on Multimedia.

[21]  Haojie Li,et al.  Deep Layer Prior Optimization for Single Image Rain Streaks Removal , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Qi Xie,et al.  Should We Encode Rain Streaks in Video as Deterministic or Stochastic? , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Mila Nikolova,et al.  Local Strong Homogeneity of a Regularized Estimator , 2000, SIAM J. Appl. Math..

[24]  Zhenhua Li,et al.  An L0 regularized cartoon-texture decomposition model for restoring images corrupted by blur and impulse noise , 2020, Signal Process. Image Commun..

[25]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[27]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Bing Zeng,et al.  A Hierarchical Approach for Rain or Snow Removing in a Single Color Image , 2017, IEEE Transactions on Image Processing.

[30]  Yu-Hsiang Fu,et al.  Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition , 2012, IEEE Transactions on Image Processing.

[31]  Shuicheng Yan,et al.  Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ting-Zhu Huang,et al.  A total variation and group sparsity based tensor optimization model for video rain streak removal , 2019, Signal Process. Image Commun..

[33]  S. Nayar,et al.  Detection and removal of rain from videos , 2004, CVPR 2004.

[34]  Shuicheng Yan,et al.  Deep Joint Rain Detection and Removal from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Huasong Chen,et al.  Morphological Component Image Restoration by Employing Bregmanized Sparse Regularization and Anisotropic Total Variation , 2019, Circuits Syst. Signal Process..

[36]  Shree K. Nayar,et al.  Vision and Rain , 2007, International Journal of Computer Vision.

[37]  XuYi,et al.  Image smoothing via L0 gradient minimization , 2011 .

[38]  David Zhang,et al.  A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Chang-Hwan Son,et al.  Rain removal via shrinkage of sparse codes and learned rain dictionary , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[40]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Rob Fergus,et al.  Restoring an Image Taken through a Window Covered with Dirt or Rain , 2013, 2013 IEEE International Conference on Computer Vision.

[42]  Yu-Chiang Frank Wang,et al.  Exploiting image structural similarity for single image rain removal , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[43]  J. S. Moll The anisotropic total variation flow , 2005 .

[44]  Sheng Zhong,et al.  Transformed Low-Rank Model for Line Pattern Noise Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Takeo Kanade,et al.  Analysis of Rain and Snow in Frequency Space , 2008, International Journal of Computer Vision.

[46]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[47]  Xinghao Ding,et al.  Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal , 2016, IEEE Transactions on Image Processing.