Rain Streak Removal in a Video to Improve Visibility by TAWL Algorithm

In computer vision applications, the visibility of the video content is crucial to perform analysis for better accuracy. The visibility can be affected by several atmospheric interferences in challenging weather-one of them is the appearance of rain streak. In recent time, rain streak removal achieves lots of interest to the researchers as it has some exciting applications such as autonomous car, intelligent traffic monitoring system, multimedia, etc. In this paper, we propose a novel and simple method by combining three novel extracted features focusing on temporal appearance, wide shape and relative location of the rain streak and we called it TAWL (Temporal Appearance, Width, and Location) method. The proposed TAWL method adaptively uses features from different resolutions and frame rates. Moreover, it progressively processes features from the up-coming frames so that it can remove rain in the real-time. The experiments have been conducted using video sequences with both real rains and synthetic rains to compare the performance of the proposed method against the relevant state-of-the-art methods. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods by removing more rain streaks while keeping other moving regions.

[1]  Katsushi Ikeuchi,et al.  Adherent Raindrop Modeling, Detectionand Removal in Video , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jie Chen,et al.  Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Chang-Su Kim,et al.  Video Deraining and Desnowing Using Temporal Correlation and Low-Rank Matrix Completion , 2015, IEEE Transactions on Image Processing.

[4]  Ting-Zhu Huang,et al.  A Novel Tensor-Based Video Rain Streaks Removal Approach via Utilizing Discriminatively Intrinsic Priors , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Deyu Meng,et al.  Robust Low-Rank Matrix Factorization Under General Mixture Noise Distributions , 2016, IEEE Transactions on Image Processing.

[6]  Shree K. Nayar,et al.  Detection and removal of rain from videos , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Wenhan Yang,et al.  Single Image Deraining: From Model-Based to Data-Driven and Beyond , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Lei Zhang,et al.  Robust Principal Component Analysis with Complex Noise , 2014, ICML.

[9]  Shuai Yang,et al.  D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal , 2019, IEEE Transactions on Image Processing.

[10]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yan Yan,et al.  $L_{1}$ -Norm Low-Rank Matrix Factorization by Variational Bayesian Method , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chen Chen,et al.  Multi-Scale Progressive Fusion Network for Single Image Deraining , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Manoranjan Paul,et al.  Rain Streak Removal from Video Sequence using Spatiotemporal Appearance , 2019, 2019 Digital Image Computing: Techniques and Applications (DICTA).

[15]  Loong Fah Cheong,et al.  Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Lei Zhang,et al.  Learning Aggregated Transmission Propagation Networks for Haze Removal and Beyond , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Qi Xie,et al.  Should We Encode Rain Streaks in Video as Deterministic or Stochastic? , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Masaaki Ikehara,et al.  GAN-Based Rain Noise Removal From Single-Image Considering Rain Composite Models , 2020, IEEE Access.

[19]  Deyu Meng,et al.  Robust Matrix Factorization with Unknown Noise , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Sudipta Mukhopadhyay,et al.  Removal of rain from videos: a review , 2014, Signal Image Video Process..

[21]  Manoranjan Paul,et al.  Improved Gaussian mixtures for robust object detection by adaptive multi-background generation , 2008, 2008 19th International Conference on Pattern Recognition.

[22]  Deyu Meng,et al.  Denoising Hyperspectral Image With Non-i.i.d. Noise Structure , 2017, IEEE Transactions on Cybernetics.

[23]  Shree K. Nayar,et al.  Vision and Rain , 2007, International Journal of Computer Vision.

[24]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[25]  Shuicheng Yan,et al.  Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Ying Ding,et al.  Sequential Deep Unrolling With Flow Priors For Robust Video Deraining , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Jie Chen,et al.  A Rain Pixel Recovery Algorithm for Videos With Highly Dynamic Scenes , 2014, IEEE Transactions on Image Processing.

[28]  Jiandong Tian,et al.  Video Desnowing and Deraining Based on Matrix Decomposition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Jing Tao,et al.  Video Rain Streak Removal by Multiscale Convolutional Sparse Coding , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Vijayan K. Asari,et al.  Utilizing Local Phase Information to Remove Rain from Video , 2014, International Journal of Computer Vision.

[31]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[32]  Wenhan Yang,et al.  Frame-Consistent Recurrent Video Deraining With Dual-Level Flow , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).