Demystifying Symmetric Smoothing Filters

Many patch-based image denoising algorithms can be formulated as applying a smoothing filter to the noisy image. Expressed as matrices, the smoothing filters must be row normalized so that each row sums to unity. Surprisingly, if we apply a column normalization before the row normalization, the performance of the smoothing filter can often be significantly improved. Prior works showed that such performance gain is related to the Sinkhorn-Knopp balancing algorithm, an iterative procedure that symmetrizes a row-stochastic matrix to a doubly-stochastic matrix. However, a complete understanding of the performance gain phenomenon is still lacking. In this paper, we study the performance gain phenomenon from a statistical learning perspective. We show that Sinkhorn-Knopp is equivalent to an Expectation-Maximization (EM) algorithm of learning a Product of Gaussians (PoG) prior of the image patches. By establishing the correspondence between the steps of Sinkhorn-Knopp and the EM algorithm, we provide a geometrical interpretation of the symmetrization process. The new PoG model also allows us to develop a new denoising algorithm called Product of Gaussian Non-Local-Means (PoG-NLM). PoG-NLM is an extension of the Sinkhorn-Knopp and is a generalization of the classical non-local means. Despite its simple formulation, PoG-NLM outperforms many existing smoothing filters and has a similar performance compared to BM3D.

[1]  Maya R. Gupta,et al.  Theory and Use of the EM Algorithm , 2011, Found. Trends Signal Process..

[2]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering , 2013 .

[3]  Ivanka Horová,et al.  Linear Smoothers and Additive Models , 1998 .

[4]  Olivier Lezoray,et al.  Image Processing and Analysis With Graphs: theory and Practice , 2017 .

[5]  Stanley H. Chan,et al.  Understanding symmetric smoothing filters via Gaussian mixtures , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[6]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[7]  Thierry Blu,et al.  Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms , 2008, IEEE Transactions on Image Processing.

[8]  S. Shott,et al.  Nonparametric Statistics , 2018, The Encyclopedia of Archaeological Sciences.

[9]  François G. Meyer,et al.  A Random Walk on Image Patches , 2011, SIAM J. Imaging Sci..

[10]  Stanley H. Chan,et al.  Fast non-local filtering by random sampling: It works, especially for large images , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Charles Kervrann,et al.  Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation , 2008, International Journal of Computer Vision.

[12]  Stanley H. Chan,et al.  Monte Carlo Non-Local Means: Random Sampling for Large-Scale Image Filtering , 2013, IEEE Transactions on Image Processing.

[13]  Truong Q. Nguyen,et al.  Adaptive non-local means for multiview image denoising: Searching for the right patches via a statistical approach , 2013, 2013 IEEE International Conference on Image Processing.

[14]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[15]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, International Journal of Computer Vision.

[16]  R. Tibshirani,et al.  Linear Smoothers and Additive Models , 1989 .

[17]  Xiang Zhu,et al.  How to SAIF-ly Boost Denoising Performance , 2013, IEEE Transactions on Image Processing.

[18]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[19]  Peyman Milanfar,et al.  Symmetrizing Smoothing Filters , 2013, SIAM J. Imaging Sci..

[20]  C. Stein Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[21]  Ken D. Sauer,et al.  A Gaussian Mixture MRF for Model-Based Iterative Reconstruction With Applications to Low-Dose X-Ray CT , 2016, IEEE Transactions on Computational Imaging.

[22]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[23]  Stergios B. Fotopoulos,et al.  All of Nonparametric Statistics , 2007, Technometrics.

[24]  Marc Levoy,et al.  Gaussian KD-trees for fast high-dimensional filtering , 2009, ACM Trans. Graph..

[25]  Manuel Menezes de Oliveira Neto,et al.  Adaptive manifolds for real-time high-dimensional filtering , 2012, ACM Trans. Graph..

[26]  Xilin Shen,et al.  Perturbation of the Eigenvectors of the Graph Laplacian: Application to Image Denoising , 2012, ArXiv.

[27]  Jean-Michel Morel,et al.  A Nonlocal Bayesian Image Denoising Algorithm , 2013, SIAM J. Imaging Sci..

[28]  Guillermo Sapiro,et al.  Fast image and video denoising via nonlocal means of similar neighborhoods , 2005, IEEE Signal Processing Letters.

[29]  Peyman Milanfar,et al.  Global Image Denoising , 2014, IEEE Transactions on Image Processing.

[30]  Gabriel Peyré,et al.  Manifold models for signals and images , 2009, Comput. Vis. Image Underst..

[31]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[32]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[33]  Michael Elad,et al.  Patch-Ordering-Based Wavelet Frame and Its Use in Inverse Problems , 2014, IEEE Transactions on Image Processing.

[34]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[35]  Subhasis Chaudhuri,et al.  Novel Speed-Up Strategies for Non-Local Means Denoising With Patch and Edge Patch Based Dictionaries , 2014, IEEE Transactions on Image Processing.

[36]  Arthur Cohen,et al.  All Admissible Linear Estimates of the Mean Vector , 1966 .

[37]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.