Enhancing Video Denoising Algorithms by Fusion from Multiple Views

Video denoising is highly desirable in many real world applications. It can enhance the perceived quality of video signals, and can also help improve the performance of subsequent processes such as compression, segmentation, and object recognition. In this paper, we propose a method to enhance existing video denoising algorithms by denoising a video signal from multiple views (front-, top-, and side-views). A fusion scheme is then proposed to optimally combine the denoised videos from multiple views into one. We show that such a conceptually simple and easy-to-use strategy, which we call multiple view fusion (MVF), leads to a computationally efficient algorithm that can significantly improve video denoising results upon state-of-the-art algorithms. The effect is especially strong at high noise levels, where the gain over the best video denoising results reported in the literature, can be as high as 2-3 dB in PSNR. Significant visual quality enhancement is also observed and evidenced by improvement in terms of SSIM evaluations.

[1]  Aleksandra Pizurica,et al.  Wavelet-Domain Video Denoising Based on Reliability Measures , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[3]  Thomas S. Huang,et al.  Image processing , 1971 .

[4]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[5]  Thierry Blu,et al.  SURE-LET for Orthonormal Wavelet-Domain Video Denoising , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  A. Murat Tekalp,et al.  Adaptive motion-compensated filtering of noisy image sequences , 1993, IEEE Trans. Circuits Syst. Video Technol..

[7]  Thierry Blu,et al.  The SURE-LET Approach to Image Denoising , 2007, IEEE Transactions on Image Processing.

[8]  John W. Woods,et al.  Spatio-temporal adaptive 3-D Kalman filter for video , 1997, IEEE Trans. Image Process..

[9]  Michael Elad,et al.  Image Sequence Denoising via Sparse and Redundant Representations , 2009, IEEE Transactions on Image Processing.

[10]  Alan C. Bovik,et al.  Handbook of Image and Video Processing (Communications, Networking and Multimedia) , 2005 .

[11]  Zhou Wang,et al.  Video Denoising Based on a Spatiotemporal Gaussian Scale Mixture Model , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Jean-Michel Morel,et al.  Denoising image sequences does not require motion estimation , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[13]  Aggelos K. Katsaggelos,et al.  Recursive displacement estimation and restoration of noisy-blurred image sequences , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[15]  Ivan W. Selesnick,et al.  Video denoising using 2D and 3D dual-tree complex wavelet transforms , 2003, SPIE Optics + Photonics.

[16]  Xin Li,et al.  Patch-Based Video Processing: A Variational Bayesian Approach , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[18]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[19]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[20]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[21]  John W. Woods,et al.  A new interpretation of ROMKF , 1997, IEEE Trans. Image Process..

[22]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[23]  Gonzalo R. Arce,et al.  Multistage order statistic filters for image sequence processing , 1991, IEEE Trans. Signal Process..

[24]  Jean-Michel Morel,et al.  Nonlocal Image and Movie Denoising , 2008, International Journal of Computer Vision.