Rain Streak Removal with Well-Recovered Moving Objects from Video Sequences Using Photometric Correlation

The main challenge in a rain removal algorithm is to differentiate rain streak from moving objects. This paper addresses this problem using the spatiotemporal appearance technique (STA). Although the STA-based technique can significantly remove rain from video, in some cases it cannot properly retain all the moving object regions. The photometric feature of rain streak was used to solve this issue. In this paper, a new algorithm combining STA and the photometric correlation between rain streak and background is proposed. Rain streak and moving objects were successfully detected and separated by combining both techniques, then fused to obtain well-recovered moving objects with rain-free video. The experimental results reveal that the proposed algorithm significantly outperforms the state-of-the-art methods for both real and synthetic rain streak.

[1]  Jie Chen,et al.  A Rain Pixel Recovery Algorithm for Videos With Highly Dynamic Scenes , 2014, IEEE Transactions on Image Processing.

[2]  Ting-Zhu Huang,et al.  A Novel Tensor-Based Video Rain Streaks Removal Approach via Utilizing Discriminatively Intrinsic Priors , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  S. Nayar,et al.  Detection and removal of rain from videos , 2004, CVPR 2004.

[4]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[5]  Jun Cai,et al.  Video-Based Automatic Incident Detection for Smart Roads: The Outdoor Environmental Challenges Regarding False Alarms , 2008, IEEE Transactions on Intelligent Transportation Systems.

[6]  Zhenjiang Miao,et al.  A Saliency Prior Context Model for Real-Time Object Tracking , 2017, IEEE Transactions on Multimedia.

[7]  Michael S. Brown,et al.  Rain Streak Removal Using Layer Priors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Xiang Zhang,et al.  A Bayesian Approach to Camouflaged Moving Object Detection , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Manoranjan Paul,et al.  Rain Streak Removal from Video Sequence using Spatiotemporal Appearance , 2019, 2019 Digital Image Computing: Techniques and Applications (DICTA).

[10]  Jing Tao,et al.  Video Rain Streak Removal by Multiscale Convolutional Sparse Coding , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Fatih Murat Porikli,et al.  Changedetection.net: A new change detection benchmark dataset , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Manoranjan Paul,et al.  A hybrid object detection technique from dynamic background using Gaussian mixture models , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[14]  Lei Zhang,et al.  Learning Aggregated Transmission Propagation Networks for Haze Removal and Beyond , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Qi Xie,et al.  Should We Encode Rain Streaks in Video as Deterministic or Stochastic? , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Sudipta Mukhopadhyay,et al.  Removal of rain from videos: a review , 2014, Signal Image Video Process..

[17]  Manoranjan Paul,et al.  Improved Gaussian mixtures for robust object detection by adaptive multi-background generation , 2008, 2008 19th International Conference on Pattern Recognition.

[18]  Manoranjan Paul,et al.  Virtual View Synthesis for Free Viewpoint Video and Multiview Video Compression using Gaussian Mixture Modelling , 2018, IEEE Transactions on Image Processing.

[19]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[20]  Manoranjan Paul,et al.  On Stable Dynamic Background Generation Technique Using Gaussian Mixture Models for Robust Object Detection , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.