Video denoising via online sparse and low-rank matrix decomposition

Video denoising refers to the problem of removing “noise” from a video sequence. Here the term “noise” is used in a broad sense to refer to any corruption or outlier or interference that is not the quantity of interest. In this work, we develop a novel approach to video denoising that is based on the idea that most noisy or corrupted videos can be split into two parts - the approximate “low-rank” layer and the “sparse layer”. We first splitting the given video into these two layers, and then apply an existing state-of-the-art denoising algorithm on each layer. We show, using extensive experiments, that our denoising approach outperforms the state-of-the art denoising algorithms.

[1]  Zuowei Shen,et al.  Robust Video Restoration by Joint Sparse and Low Rank Matrix Approximation , 2011, SIAM J. Imaging Sci..

[2]  Jean-Michel Morel,et al.  Image denoising by non-local averaging , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[3]  Yoram Bresler,et al.  Online Sparsifying Transform Learning— Part I: Algorithms , 2015, IEEE Journal of Selected Topics in Signal Processing.

[4]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[5]  Michal Joachimiak,et al.  Multiview 3D video denoising in sliding 3D DCT domain , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[6]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[7]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[8]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[9]  Yoram Bresler,et al.  Online Sparsifying Transform Learning—Part II: Convergence Analysis , 2015, IEEE Journal of Selected Topics in Signal Processing.

[10]  Motaz El-Saban,et al.  FRPCA: Fast Robust Principal Component Analysis for online observations , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[11]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[12]  William T. Freeman,et al.  A High-Quality Video Denoising Algorithm Based on Reliable Motion Estimation , 2010, ECCV.

[13]  Oscar C. Au,et al.  Temporal Video Denoising Based on Multihypothesis Motion Compensation , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Morteza Mardani,et al.  Dynamic Anomalography: Tracking Network Anomalies Via Sparsity and Low Rank , 2012, IEEE Journal of Selected Topics in Signal Processing.

[15]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[17]  M. Omair Ahmad,et al.  Video Denoising Based on Inter-frame Statistical Modeling of Wavelet Coefficients , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[19]  Shie Mannor,et al.  Online PCA for Contaminated Data , 2013, NIPS.

[20]  M. Omair Ahmad,et al.  Video Denoising Using Motion Compensated 3-D Wavelet Transform With Integrated Recursive Temporal Filtering , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[22]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Saeed Gazor,et al.  Video denoising in three-dimensional complex wavelet domain using a doubly stochastic modelling , 2012 .

[24]  Martin Kleinsteuber,et al.  Robust PCA and subspace tracking from incomplete observations using $$\ell _0$$ℓ0-surrogates , 2012, Comput. Stat..

[25]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[26]  Shiliang Sun,et al.  Multitask Twin Support Vector Machines , 2012, ICONIP.

[27]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[28]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[30]  Yoram Bresler,et al.  Video denoising by online 3D sparsifying transform learning , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[31]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[32]  Honglak Lee,et al.  Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising , 2013, NIPS.

[33]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.