Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Generalised Nonlocal Image Smoothing Generalised Nonlocal Image Smoothing

We propose a discrete variational approach for image smoothing consisting of nonlocal data and smoothness constraints that penalise general dissimilarity measures defined on image patches. One of such dissimilarity measures is the weighted L2 distance between patches. In such a case we derive an iterative neighbourhood filter that induces a new similarity measure in the photometric domain. It can be regarded as an extended patch similarity measure that evaluates not only the patch similarity of two chosen pixels, but also the similarity of their corresponding neighbours. This leads to a more robust smoothing process since the pixels selected for averaging are more coherent with the local image structure. By slightly modifying the way the similarities are computed we obtain two related filters: The NL-means filter of Buades et al. (SIAM Multiscale Model. Simul. 4(2):490–530, 2005b) and the NDS filter of Mrázek et al. (Geometric Properties for Incomplete Data, Computational Imaging and Vision, vol. 31, pp. 335–352, Springer, Dordrecht, 2006). In fact, the proposed approach can be considered as a generalisation of the latter filter to the space of patches. We also provide novel insights into relations of the NDS filter with diffusion/regularisation methods as well as with some recently proposed graph regularisation techniques. We evaluate our method for the task of denoising greyscale and colour images degraded with Gaussian and salt-and-pepper noise, demonstrating that it compares very well to other more sophisticated approaches.

[1]  Jean-Pierre Bourguignon,et al.  Mathematische Annalen , 1893 .

[2]  L. Brouwer Über Abbildung von Mannigfaltigkeiten , 1911 .

[3]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[4]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[5]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[6]  Jong-Sen Lee,et al.  Digital image smoothing and the sigma filter , 1983, Comput. Vis. Graph. Image Process..

[7]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[9]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[10]  T. Loupas,et al.  An adaptive weighted median filter for speckle suppression in medical ultrasonic images , 1989 .

[11]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Niklas Nordström,et al.  Biased anisotropic diffusion: a unified regularization and diffusion approach to edge detection , 1990, Image Vis. Comput..

[13]  Philippe Saint-Marc,et al.  Adaptive Smoothing: A General Tool for Early Vision , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[15]  D. Mumford The Bayesian Rationale for Energy Functionals 1 , 1994 .

[16]  Edward J. Delp,et al.  Discontinuity preserving regularization of inverse visual problems , 1994, IEEE Trans. Syst. Man Cybern..

[17]  Bart M. ter Haar Romeny,et al.  Geometry-Driven Diffusion in Computer Vision , 1994, Computational Imaging and Vision.

[18]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[20]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[21]  Michel Barlaud,et al.  Deterministic edge-preserving regularization in computed imaging , 1997, IEEE Trans. Image Process..

[22]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[23]  Joachim Weickert,et al.  A Review of Nonlinear Diffusion Filtering , 1997, Scale-Space.

[24]  J. Marron,et al.  Edge-Preserving Smoothers for Image Processing , 1998 .

[25]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[26]  G. Winkler,et al.  Noise Reduction in Images: Some Recent Edge-Preserving Methods , 1998 .

[27]  Bram van Ginneken,et al.  Applications of Locally Orderless Images , 1999, J. Vis. Commun. Image Represent..

[28]  Joost van de Weijer,et al.  Local Mode Filtering , 2001, CVPR.

[29]  Tony F. Chan,et al.  The digital TV filter and nonlinear denoising , 2001, IEEE Trans. Image Process..

[30]  Michael Elad,et al.  On the origin of the bilateral filter and ways to improve it , 2002, IEEE Trans. Image Process..

[31]  Joost van de Weijer,et al.  On the Equivalence of Local-Mode Finding, Robust Estimation and Mean-Shift Analysis as Used in Early Vision Tasks , 2002, ICPR.

[32]  Gerhard Winkler,et al.  Image Analysis, Random Fields and Markov Chain Monte Carlo Methods: A Mathematical Introduction , 2002 .

[33]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Eero P. Simoncelli,et al.  Image Denoising using Gaussian Scale Mixtures in the Wavelet Domain , 2002 .

[35]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[36]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[37]  Yvan G. Leclerc,et al.  Constructing simple stable descriptions for image partitioning , 1989, International Journal of Computer Vision.

[38]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[39]  Thomas Brox,et al.  On the Equivalence of Soft Wavelet Shrinkage, Total Variation Diffusion, Total Variation Regularization, and SIDEs , 2004, SIAM J. Numer. Anal..

[40]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[41]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[42]  Andrea J. van Doorn,et al.  The Structure of Locally Orderless Images , 1999, International Journal of Computer Vision.

[43]  Joachim Weickert,et al.  A Theoretical Framework for Convex Regularizers in PDE-Based Computation of Image Motion , 2001, International Journal of Computer Vision.

[44]  Joachim Weickert,et al.  Relations Between Regularization and Diffusion Filtering , 2000, Journal of Mathematical Imaging and Vision.

[45]  Pavel Mrázek,et al.  Selection of Optimal Stopping Time for Nonlinear Diffusion Filtering , 2001, International Journal of Computer Vision.

[46]  Dorin Comaniciu,et al.  A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift , 2004, Image Vis. Comput..

[47]  Bernhard Schölkopf,et al.  Regularization on Discrete Spaces , 2005, DAGM-Symposium.

[48]  Guillermo Sapiro,et al.  Fast image and video denoising via nonlocal means of similar neighborhoods , 2005, IEEE Signal Processing Letters.

[49]  Suyash P. Awate,et al.  Higher-order image statistics for unsupervised, information-theoretic, adaptive, image filtering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[50]  Raymond H. Chan,et al.  Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization , 2005, IEEE Transactions on Image Processing.

[51]  Mila Nikolova,et al.  Analysis of the Recovery of Edges in Images and Signals by Minimizing Nonconvex Regularized Least-Squares , 2005, Multiscale Model. Simul..

[52]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[53]  Stanley Osher,et al.  Deblurring and Denoising of Images by Nonlocal Functionals , 2005, Multiscale Model. Simul..

[54]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[55]  Christoph Schnörr,et al.  Unique reconstruction of piecewise-smooth images by minimizing strictly convex nonquadratic functionals , 1994, Journal of Mathematical Imaging and Vision.

[56]  Tony F. Chan,et al.  Image processing and analysis - variational, PDE, wavelet, and stochastic methods , 2005 .

[57]  Nahum Kiryati,et al.  Image Deblurring in the Presence of Impulsive Noise , 2006, International Journal of Computer Vision.

[58]  Charles Kervrann,et al.  Optimal Spatial Adaptation for Patch-Based Image Denoising , 2006, IEEE Transactions on Image Processing.

[59]  P. Mrázek,et al.  ON ROBUST ESTIMATION AND SMOOTHING WITH SPATIAL AND TONAL KERNELS , 2006 .

[60]  G. Aubert,et al.  Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations (Applied Mathematical Sciences) , 2006 .

[61]  Suyash P. Awate,et al.  Unsupervised, information-theoretic, adaptive image filtering for image restoration , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Joachim Weickert,et al.  From Adaptive Averaging to Accelerated Nonlinear Diffusion Filtering , 2006, DAGM-Symposium.

[63]  Jean-Michel Morel,et al.  Neighborhood filters and PDE’s , 2006, Numerische Mathematik.

[64]  Joachim Weickert,et al.  Evaluating a General Class of Filters for Image Denoising , 2007, SCIA.

[65]  Abderrahim Elmoataz,et al.  Discrete Regularization on Weighted Graphs for Image and Mesh Filtering , 2007, SSVM.

[66]  Charles Kervrann,et al.  Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation , 2008, International Journal of Computer Vision.

[67]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[68]  J. Weickert,et al.  INTEGRODIFFERENTIAL EQUATIONS FOR CONTINUOUS MULTISCALE WAVELET SHRINKAGE , 2007 .

[69]  Radu Ciprian Bilcu,et al.  Fast nonlocal means for image denoising , 2007, Electronic Imaging.

[70]  J. Weickert,et al.  Energy-Based Image Simplification with Nonlocal Data and Smoothness Terms , 2007 .

[71]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[72]  Guy Gilboa,et al.  Nonlocal Linear Image Regularization and Supervised Segmentation , 2007, Multiscale Model. Simul..

[73]  Nikos Paragios,et al.  Variable Bandwidth Image Denoising Using Image-based Noise Models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Jin Wang,et al.  A Robust and Fast Non-Local Means Algorithm for Image Denoising , 2007, 2007 10th IEEE International Conference on Computer-Aided Design and Computer Graphics.

[75]  Daniel Cremers,et al.  Iterated Nonlocal Means for Texture Restoration , 2007, SSVM.

[76]  Jianhua Ma,et al.  Nonlocal Prior Bayesian Tomographic Reconstruction , 2008, Journal of Mathematical Imaging and Vision.

[77]  Pierrick Coupé,et al.  Bayesian Non-local Means Filter, Image Redundancy and Adaptive Dictionaries for Noise Removal , 2007, SSVM.

[78]  Jian-Feng Cai,et al.  Two-phase approach for deblurring images corrupted by impulse plus gaussian noise , 2008 .

[79]  Mehran Ebrahimi,et al.  Efficient nonlocal-means denoising using the SVD , 2008, 2008 15th IEEE International Conference on Image Processing.

[80]  V. K. Govindan,et al.  Robust Estimation Approach for NL-Means Filter , 2008, ISVC.

[81]  Abderrahim Elmoataz,et al.  Local and Nonlocal Discrete Regularization on Weighted Graphs for Image and Mesh Processing , 2009, International Journal of Computer Vision.

[82]  A. Elmoataz,et al.  Author Manuscript, Published in "international Workshop on Local and Non-local Approximation in Image Processing, Suisse Unifying Local and Nonlocal Processing with Partial Difference Operators on Weighted Graphs , 2022 .

[83]  Abderrahim Elmoataz,et al.  Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing , 2008, IEEE Transactions on Image Processing.

[84]  J. Weickert,et al.  A ROTATIONALLY INVARIANT BLOCK MATCHING STRATEGY IMPROVING IMAGE DENOISING WITH NON-LOCAL MEANS , 2008 .

[85]  Dengyong Zhou,et al.  High-Order Regularization on Graphs , 2008 .

[86]  Jérôme Darbon,et al.  Fast nonlocal filtering applied to electron cryomicroscopy , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[87]  Pierrick Coupé,et al.  Bayesian non local means-based speckle filtering , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[88]  Daniel Cremers,et al.  Efficient Nonlocal Means for Denoising of Textural Patterns , 2008, IEEE Transactions on Image Processing.

[89]  Aleksandra Pizurica,et al.  An improved non-local denoising algorithm , 2008 .

[90]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[91]  Peyman Milanfar,et al.  A generalization of non-local means via kernel regression , 2008, Electronic Imaging.

[92]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[93]  Pierrick Coupé,et al.  An Optimized Blockwise Nonlocal Means Denoising Filter for 3-D Magnetic Resonance Images , 2008, IEEE Transactions on Medical Imaging.

[94]  T. Brox,et al.  Nonlocal texture filtering with efficient tree structures and invariant patch similarity measures , 2008 .

[95]  E. Vrscay A SIMPLE MODEL FOR AFFINE SELF-SIMILARITY OF IMAGES AND ITS APPLICATIONS , 2008 .

[96]  Abderrahim Elmoataz,et al.  Graph-based tools for microscopic cellular image segmentation , 2009, Pattern Recognit..

[97]  Stefano Soatto,et al.  Nonlocal Similarity Image Filtering , 2009, ICIAP.

[98]  Jun Liu,et al.  An Adaptive Method for Recovering Image from Mixed Noisy Data , 2009, International Journal of Computer Vision.

[99]  Alfred M. Bruckstein,et al.  Scale Space and Variational Methods in Computer Vision , 2011, Lecture Notes in Computer Science.

[100]  Julie Zhou Robust Estimationに , 2009 .