A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical

In this article, the author presents a practical and accessible framework to understand some of the basic underpinnings of these methods, with the intention of leading the reader to a broad understanding of how they interrelate. The author also illustrates connections between these techniques and more classical (empirical) Bayesian approaches. The proposed framework is used to arrive at new insights and methods, both practical and theoretical. In particular, several novel optimality properties of algorithms in wide use such as block-matching and three-dimensional (3-D) filtering (BM3D), and methods for their iterative improvement (or nonexistence thereof) are discussed. A general approach is laid out to enable the performance analysis and subsequent improvement of many existing filtering algorithms. While much of the material discussed is applicable to the wider class of linear degradation models beyond noise (e.g., blur,) to keep matters focused, we consider the problem of denoising here.

[1]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[2]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[3]  G. Pflug Kernel Smoothing. Monographs on Statistics and Applied Probability - M. P. Wand; M. C. Jones. , 1996 .

[4]  Bernhard Schölkopf,et al.  A kernel view of the dimensionality reduction of manifolds , 2004, ICML.

[5]  Tsachy Weissman,et al.  A Context Quantization Approach to Universal Denoising , 2007, IEEE Transactions on Signal Processing.

[6]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Generalised Nonlocal Image Smoothing Generalised Nonlocal Image Smoothing , 2022 .

[7]  Philip A. Knight,et al.  The Sinkhorn-Knopp Algorithm: Convergence and Applications , 2008, SIAM J. Matrix Anal. Appl..

[8]  William Kahan,et al.  Spectra of nearly Hermitian matrices , 1975 .

[9]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[10]  Olivier Salvado,et al.  Hashed Nonlocal Means for Rapid Image Filtering , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[12]  P. Lancaster,et al.  Surfaces generated by moving least squares methods , 1981 .

[13]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[14]  William J. Stewart,et al.  Introduction to the numerical solution of Markov Chains , 1994 .

[15]  A. Rukhin Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[16]  Stephen M. Smith,et al.  SUSAN—A New Approach to Low Level Image Processing , 1997, International Journal of Computer Vision.

[17]  Jongmin Baek,et al.  Accelerating spatially varying Gaussian filters , 2010, ACM Trans. Graph..

[18]  Alfred M. Bruckstein,et al.  On Globally Optimal Local Modeling: From Moving Least Squares to Over-parametrization , 2013, Innovations for Shape Analysis, Models and Algorithms.

[19]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Dorin Comaniciu,et al.  A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift , 2004, Image Vis. Comput..

[21]  P. Bühlmann,et al.  Boosting with the L2-loss: regression and classification , 2001 .

[22]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[23]  Niklas Nordström,et al.  Biased anisotropic diffusion: a unified regularization and diffusion approach to edge detection , 1990, Image Vis. Comput..

[24]  Alfred M. Bruckstein,et al.  Diffusions and Confusions in Signal and Image Processing , 2001, Journal of Mathematical Imaging and Vision.

[25]  C. Westin,et al.  Normalized and differential convolution , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Peyman Milanfar,et al.  On Iterative Regularization and Its Application , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  B. Yu,et al.  Boosting with the L_2-Loss: Regression and Classification , 2001 .

[29]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[30]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[31]  Tsachy Weissman,et al.  Universal discrete denoising: known channel , 2003, IEEE Transactions on Information Theory.

[32]  Peyman Milanfar,et al.  Symmetrizing Smoothing Filters , 2013, SIAM J. Imaging Sci..

[33]  Peter Buhlmann,et al.  BOOSTING ALGORITHMS: REGULARIZATION, PREDICTION AND MODEL FITTING , 2007, 0804.2752.

[34]  Joachim Weickert,et al.  Relations Between Regularization and Diffusion Filtering , 2000, Journal of Mathematical Imaging and Vision.

[35]  Charles R. Johnson,et al.  On the diagonal scaling of Euclidean distance matrices to doubly stochastic matrices , 2005 .

[36]  Ling Huang,et al.  Spectral Clustering with Perturbed Data , 2008, NIPS.

[37]  Stergios B. Fotopoulos,et al.  All of Nonparametric Statistics , 2007, Technometrics.

[38]  Suyash P. Awate,et al.  Unsupervised, information-theoretic, adaptive image filtering for image restoration , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[40]  Ling Huang,et al.  Fast approximate spectral clustering , 2009, KDD.

[41]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Ron Kimmel,et al.  A Short- Time Beltrami Kernel for Smoothing Images and Manifolds , 2007, IEEE Transactions on Image Processing.

[43]  W. Härdle Applied Nonparametric Regression , 1992 .

[44]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[45]  Peyman Milanfar,et al.  Is Denoising Dead? , 2010, IEEE Transactions on Image Processing.

[46]  A. Odlyzko,et al.  Bounds for eigenvalues of certain stochastic matrices , 1981 .

[47]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[48]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[49]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[50]  Michael Elad,et al.  On the origin of the bilateral filter and ways to improve it , 2002, IEEE Trans. Image Process..

[51]  Daniel Pérez Palomar,et al.  Gradient of mutual information in linear vector Gaussian channels , 2006, IEEE Transactions on Information Theory.

[52]  Yoel Shkolnisky,et al.  Diffusion Interpretation of Nonlocal Neighborhood Filters for Signal Denoising , 2009, SIAM J. Imaging Sci..

[53]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[54]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[55]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[56]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[57]  Yoram Singer,et al.  Logistic Regression, AdaBoost and Bregman Distances , 2000, Machine Learning.

[58]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[59]  Carl-Fredrik Westin,et al.  Normalized and differential convolution , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[60]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[61]  Peyman Milanfar,et al.  Patch-Based Near-Optimal Image Denoising , 2012, IEEE Transactions on Image Processing.

[62]  Raja Nicolas Khoury Closest Matrices in the Space of Generalized Doubly Stochastic Matrices , 1998 .

[63]  Suyash P. Awate,et al.  Feature-Preserving MRI Denoising: A Nonparametric Empirical Bayes Approach , 2007, IEEE Transactions on Medical Imaging.

[64]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[65]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[66]  Danny Barash,et al.  A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[67]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[68]  Suyash P. Awate,et al.  Higher-order image statistics for unsupervised, information-theoretic, adaptive, image filtering , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[69]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[70]  Charles Kervrann,et al.  Optimal Spatial Adaptation for Patch-Based Image Denoising , 2006, IEEE Transactions on Image Processing.

[71]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[72]  Cordelia Schmid,et al.  Accurate Image Search Using the Contextual Dissimilarity Measure , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Thierry Blu,et al.  A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding , 2007, IEEE Transactions on Image Processing.

[74]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, ACM Trans. Graph..