Removing Multi-frame Gaussian Noise by Combining Patch-based Filters with Optical Flow.

Patch-based methods such as 3D block matching (BM3D) and the non-local Bayes (NLB) approach produce state-of-the-art results for removing Gaussian noise from single-frame images. In this work, we propose two extensions for these filters when there exist multiple frames of the same scene. To this end, we combine two novel inter-frame connectivity strategies with robust optical flow methods. Our extensions do not require additional parameters and outperform existing techniques qualitatively by a significant margin. By exploiting spatial and temporal separability, one of our approaches is also faster than its competitors. Since our strategy is not restricted to BM3D and NLB, it can be generalised to other similar single-frame patch-based methods.

[1]  W. Philips,et al.  Integrated approach for estimation and restoration of photon-limited images based on steerable pyramids , 2003, Proceedings EC-VIP-MC 2003. 4th EURASIP Conference focused on Video/Image Processing and Multimedia Communications (IEEE Cat. No.03EX667).

[2]  Matthew Uyttendaele,et al.  Deep Burst Denoising , 2017, ECCV.

[3]  Shree K. Nayar,et al.  Multiple view image denoising , 2009, CVPR.

[4]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[5]  Electron Tomography: Methods for Three-Dimensional Visualization of Structures in the Cell, 2nd Edition. Edited by Joachim Frank. Springer, Albany, NY; 2006, 455 pages. ISBN 0-387-31234-X (HB) , 2008, Microscopy and Microanalysis.

[6]  Joachim Weickert,et al.  Enhancing Patch-Based Methods with Inter-Frame Connectivity for Denoising Multi-Frame Images , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[7]  Alessandro Foi,et al.  Optimal Inversion of the Anscombe Transformation in Low-Count Poisson Image Denoising , 2011, IEEE Transactions on Image Processing.

[8]  Joachim Weickert,et al.  Evaluating Data Terms for Variational Multi-frame Super-Resolution , 2017, SSVM.

[9]  Joachim Weickert,et al.  Theoretical Foundations of Anisotropic Diffusion in Image Processing , 1994, Theoretical Foundations of Computer Vision.

[10]  Joachim Weickert,et al.  Hough Based Evolutions for Enhancing Structures in 3D Electron Microscopy , 2019, CAIP.

[11]  Yifei Lou,et al.  A note on multi-image denoising , 2009, 2009 International Workshop on Local and Non-Local Approximation in Image Processing.

[12]  Thierry Blu,et al.  Fast Haar-wavelet denoising of multidimensional fluorescence microscopy data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[13]  Patrick Bouthemy,et al.  Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences , 2010, IEEE Transactions on Medical Imaging.

[14]  Thomas S. Huang,et al.  Advances in computer vision and image processing : a research annual , 1984 .

[15]  Huijun Gao,et al.  A noise-suppressing and edge-preserving multiframe super-resolution image reconstruction method , 2015, Signal Process. Image Commun..

[16]  Michael Elad,et al.  Image Processing Using Smooth Ordering of its Patches , 2012, IEEE Transactions on Image Processing.

[17]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[18]  Guangming Shi,et al.  Robust Tensor Approximation With Laplacian Scale Mixture Modeling for Multiframe Image and Video Denoising , 2018, IEEE Journal of Selected Topics in Signal Processing.

[19]  José M. Bioucas-Dias,et al.  Denoising of medical images corrupted by Poisson noise , 2008, 2008 15th IEEE International Conference on Image Processing.

[20]  Murray Eden,et al.  Fundamentals of Digital Optics: Digital Signal Processing In Optics And Holography , 2012 .

[21]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[22]  Eric Moreau,et al.  A new denoising model for multi-frame super-resolution image reconstruction , 2017, Signal Process..

[23]  Alessandro Foi,et al.  Optimal Inversion of the Generalized Anscombe Transformation for Poisson-Gaussian Noise , 2013, IEEE Transactions on Image Processing.

[24]  Thierry Blu,et al.  Multiframe sure-let denoising of timelapse fluorescence microscopy images , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[25]  Marius Tico,et al.  Multi-frame image denoising and stabilization , 2008, 2008 16th European Signal Processing Conference.

[26]  Alessandro Foi,et al.  Nonlocality-Reinforced Convolutional Neural Networks for Image Denoising , 2018, IEEE Signal Processing Letters.

[27]  Boguslaw Obara,et al.  Coherence enhancing diffusion filtering based on the Phase Congruency Tensor , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[28]  J. L. Lisani,et al.  Enhancement of Noisy and Compressed Videos by Optical Flow and Non-Local Denoising , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Alessandro Foi,et al.  Variance Stabilization for Noisy+Estimate Combination in Iterative Poisson Denoising , 2016, IEEE Signal Processing Letters.

[30]  Javier Sánchez,et al.  Regularization Strategies for Discontinuity-Preserving Optical Flow Methods , 2016, IEEE Transactions on Image Processing.

[31]  Khan A. Wahid,et al.  Denoising Low-Dose CT Images Using Multiframe Blind Source Separation and Block Matching Filter , 2018, IEEE Transactions on Radiation and Plasma Medical Sciences.

[32]  Thomas Brox,et al.  Dense 3D Reconstruction with a Hand-Held Camera , 2012, DAGM/OAGM Symposium.

[33]  Alessandro Foi,et al.  Ieee Transactions on Image Processing a Closed-form Approximation of the Exact Unbiased Inverse of the Anscombe Variance-stabilizing Transformation , 2022 .

[34]  Zhixun Su,et al.  A patch-based low-rank tensor approximation model for multiframe image denoising , 2018, J. Comput. Appl. Math..

[35]  Shutao Li,et al.  Sparsity based denoising of spectral domain optical coherence tomography images , 2012, Biomedical optics express.

[36]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[37]  Jean-Michel Morel,et al.  Implementation of the "Non-Local Bayes" (NL-Bayes) Image Denoising Algorithm , 2013, Image Process. Line.

[38]  Jean-Michel Morel,et al.  A Nonlocal Bayesian Image Denoising Algorithm , 2013, SIAM J. Imaging Sci..

[39]  V. Radmilović,et al.  3-D reconstruction of the atomic positions in a simulated gold nanocrystal based on discrete tomography: prospects of atomic resolution electron tomography. , 2008, Ultramicroscopy.

[40]  Jean-Michel Morel,et al.  Video Denoising via Empirical Bayesian Estimation of Space-Time Patches , 2017, Journal of Mathematical Imaging and Vision.

[41]  Jose Luis Lisani,et al.  Patch-Based Video Denoising With Optical Flow Estimation , 2016, IEEE Transactions on Image Processing.

[42]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[43]  Javier Portilla,et al.  Efficient joint poisson-gauss restoration using multi-frame L2-relaxed-L0 analysis-based sparsity , 2011, 2011 18th IEEE International Conference on Image Processing.

[44]  Jonathan T. Barron,et al.  Burst Denoising with Kernel Prediction Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Julio Esclarín Monreal,et al.  A PDE model for computing the optimal flow , 1999 .

[47]  Linda G. Shapiro,et al.  Computer Vision , 2001 .

[48]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[49]  F. J. Anscombe,et al.  THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA , 1948 .

[50]  Karen O. Egiazarian,et al.  Video Denoising, Deblocking, and Enhancement Through Separable 4-D Nonlocal Spatiotemporal Transforms , 2012, IEEE Transactions on Image Processing.

[51]  Jean-Michel Morel,et al.  Video Denoising by Combining Patch Search and CNNs , 2020, J. Math. Imaging Vis..

[52]  Agustín Salgado de la Nuez,et al.  Robust Discontinuity Preserving Optical Flow Methods , 2016, Image Process. Line.

[53]  Nelson Monzón,et al.  Robust discontinuity preserving optical flow methods , 2016 .

[54]  Alessandro Foi,et al.  Modeling and Estimation of Signal-Dependent and Correlated Noise , 2018 .

[55]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[56]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[57]  C. Boncelet.  Image Noise Models , 2009 .

[58]  Marc Lebrun,et al.  An Analysis and Implementation of the BM3D Image Denoising Method , 2012, Image Process. Line.

[59]  T. S. Huang,et al.  Advances in computer vision & image processing , 1988 .

[60]  Yong Cheng,et al.  Comments on "Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering" , 2011, IEEE Trans. Image Process..

[61]  Jean-Michel Morel,et al.  A Non-Local CNN for Video Denoising , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[62]  Joachim Weickert,et al.  Poisson Noise Removal Using Multi-Frame 3D Block Matching , 2019, 2019 8th European Workshop on Visual Information Processing (EUVIP).

[63]  J. Morel,et al.  Multi image noise estimation and denoising , 2010 .

[64]  Guangming Shi,et al.  Low-Rank Tensor Approximation with Laplacian Scale Mixture Modeling for Multiframe Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[65]  Alan C. Bovik,et al.  The Essential Guide to Image Processing , 2009, J. Electronic Imaging.

[66]  Jean-Michel Morel,et al.  Denoising image sequences does not require motion estimation , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[67]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[68]  A. Buades,et al.  Motion-Compensated Spatio-Temporal Filtering for Multi-Image and Multimodal Super-Resolution , 2019, International Journal of Computer Vision.