Snowflake Removal for Videos via Global and Local Low-Rank Decomposition

Falling snow not only blocks human vision, but also significantly degrades the effectiveness of computer vision systems in outdoor environment. In this paper, we aim to remove snowflakes in videos by using the global and local low-rank property of snowflake-removed scenes. The stationary background and the mixture of moving foreground as well as falling snowflake are extracted via the global low-rank matrix decomposition. Some snowflake features, such as its color and size, are used to separate out the snowflakes from other moving objects. Then, the mean absolute difference based patch matching is applied to align every same moving object over frames to grab its low-rank structure. As such, the falling snowflake in front of moving objects can be removed via the local low-rank decomposition. Finally, the snowflake removed videos are generated by pasting moving foreground to stationary backgrounds. Experiments show that our method can remove snowflakes effectively and outperforms the comparison methods.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  Jing Xu,et al.  Removing rain and snow in a single image using guided filter , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE).

[3]  Hao Li,et al.  Rain Removal in Video by Combining Temporal and Chromatic Properties , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[4]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[5]  Zhen Chen,et al.  A New Algorithm of Rain (Snow) Removal in Video , 2013, J. Multim..

[6]  Shree K. Nayar,et al.  When does a camera see rain? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[8]  Zhihai He,et al.  Robust Generalized Low-Rank Decomposition of Multimatrices for Image Recovery , 2017, IEEE Transactions on Multimedia.

[9]  Wei Guo,et al.  Single-Image-Based Rain and Snow Removal Using Multi-guided Filter , 2013, ICONIP.

[10]  Michael T. Orchard,et al.  Overlapped block motion compensation: an estimation-theoretic approach , 1994, IEEE Trans. Image Process..

[11]  陳敦裕,et al.  Visual depth guided color image rain streaks removal using sparse coding , 2014 .

[12]  Hidetomo Sakaino,et al.  A Semitransparency-Based Optical-Flow Method With a Point Trajectory Model for Particle-Like Video , 2012, IEEE Transactions on Image Processing.

[13]  Anthony Rowe,et al.  Programmable Automotive Headlights , 2014, ECCV.

[14]  Bo Liu.  Real-time moving vehicle recognition under snowy condition , 2007, 2007 International Conference on Intelligent and Advanced Systems.

[15]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[16]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[17]  Yu-Hsiang Fu,et al.  Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition , 2012, IEEE Transactions on Image Processing.

[18]  Marco Tagliasacchi,et al.  Estimating Snow Cover From Publicly Available Images , 2015, IEEE Transactions on Multimedia.

[19]  Jérémie Bossu,et al.  Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks , 2011, International Journal of Computer Vision.

[20]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Chang-Su Kim,et al.  Video Deraining and Desnowing Using Temporal Correlation and Low-Rank Matrix Completion , 2015, IEEE Transactions on Image Processing.

[22]  Jing Xu,et al.  An Improved Guidance Image Based Method to Remove Rain and Snow in a Single Image , 2012, Comput. Inf. Sci..

[23]  Shree K. Nayar,et al.  Detection and removal of rain from videos , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Nianjun Liu,et al.  Using the Shape Characteristics of Rain to Identify and Remove Rain from Video , 2008, SSPR/SPR.

[25]  Anthony Rowe,et al.  Fast reactive control for illumination through rain and snow , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[26]  Lizhuang Ma,et al.  Falling snow motion estimation based on a semi-transparent and particle trajectory model , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[27]  Takeo Kanade,et al.  Analysis of Rain and Snow in Frequency Space , 2008, International Journal of Computer Vision.

[28]  Hidetomo Sakaino.  Nonlinear robust velocity estimation of vehicles from a snowfall traffic scene , 2002, Object recognition supported by user interaction for service robots.

[29]  Xiaowei Zhou,et al.  Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Li-Wei Kang,et al.  Self-Learning Based Image Decomposition With Applications to Single Image Denoising , 2014, IEEE Transactions on Multimedia.

[31]  Vijayan K. Asari,et al.  Utilizing Local Phase Information to Remove Rain from Video , 2014, International Journal of Computer Vision.