The Little Engine That Could: Regularization by Denoising (RED)

Removal of noise from an image is an extensively studied problem in image processing. Indeed, the recent advent of sophisticated and highly effective denoising algorithms has led some to believe that existing methods are touching the ceiling in terms of noise removal performance. Can we leverage this impressive achievement to treat other tasks in image processing? Recent work has answered this question positively, in the form of the Plug-and-Play Prior ($P^3$) method, showing that any inverse problem can be handled by sequentially applying image denoising steps. This relies heavily on the ADMM optimization technique in order to obtain this chained denoising interpretation. Is this the only way in which tasks in image processing can exploit the image denoising engine? In this paper we provide an alternative, more powerful, and more flexible framework for achieving the same goal. As opposed to the $P^3$ method, we offer Regularization by Denoising (RED): using the denoising engine in defining the regulariza...

[1]  Frédo Durand,et al.  Patch Complexity, Finite Pixel Correlations and Optimal Denoising , 2012, ECCV.

[2]  OsherStanley,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[3]  Michael Elad,et al.  Improving K-SVD denoising by post-processing its method-noise , 2013, 2013 IEEE International Conference on Image Processing.

[4]  Stanley H. Chan.  Algorithm-Induced Prior for Image Restoration , 2016, ArXiv.

[5]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[6]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[7]  Venu Madhav Govindu,et al.  Symmetric Smoothing Filters From Global Consistency Constraints , 2015, IEEE Transactions on Image Processing.

[8]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[9]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[10]  Wen Gao,et al.  Progressive Image Denoising Through Hybrid Graph Laplacian Regularization: A Unified Framework , 2014, IEEE Transactions on Image Processing.

[11]  Laurent D. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[12]  Peyman Milanfar,et al.  Symmetrizing Smoothing Filters , 2013, SIAM J. Imaging Sci..

[13]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[14]  Cheng-Shang Chang.  Calculus , 2020, Bicycle or Unicycle?.

[15]  Andrew Zisserman,et al.  Efficient additive kernels via explicit feature maps , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Peyman Milanfar,et al.  Global Image Denoising , 2014, IEEE Transactions on Image Processing.

[17]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[19]  Florin Popentiu,et al.  Iterative identification and restoration of images , 1993, Comput. Graph..

[20]  E.E. Pissaloux,et al.  Image Processing , 1994, Proceedings. Second Euromicro Workshop on Parallel and Distributed Processing.

[21]  Charles A. Bouman,et al.  Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation , 2015, IEEE Transactions on Computational Imaging.

[22]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[23]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[24]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[25]  Anat Levin,et al.  Natural image denoising: Optimality and inherent bounds , 2011, CVPR 2011.

[26]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[27]  Richard G. Baraniuk,et al.  Optimal recovery from compressive measurements via denoising-based approximate message passing , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[28]  Peyman Milanfar,et al.  A General Framework for Regularized, Similarity-Based Image Restoration , 2014, IEEE Transactions on Image Processing.

[29]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[30]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[31]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Matthias Zwicker,et al.  Dual-domain image denoising , 2013, 2013 IEEE International Conference on Image Processing.

[33]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[34]  Andrea L. Bertozzi,et al.  Higher-Order Feature-Preserving Geometric Regularization , 2010, SIAM J. Imaging Sci..

[35]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[36]  Michael Elad,et al.  Convolutional Neural Networks Analyzed via Convolutional Sparse Coding , 2016, J. Mach. Learn. Res..

[37]  Jean-Michel Morel,et al.  Secrets of image denoising cuisine* , 2012, Acta Numerica.

[38]  Peyman Milanfar,et al.  Patch-Based Near-Optimal Image Denoising , 2012, IEEE Transactions on Image Processing.

[39]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  José M. Bioucas-Dias,et al.  Image restoration with locally selected class-adapted models , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[41]  BeckAmir,et al.  Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems , 2009 .

[42]  Abderrahim Elmoataz,et al.  Local and Nonlocal Discrete Regularization on Weighted Graphs for Image and Mesh Processing , 2009, International Journal of Computer Vision.

[43]  Jean-Michel Morel,et al.  DA3D: Fast and data adaptive dual domain denoising , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[44]  Xilin Shen,et al.  Perturbation of the Eigenvectors of the Graph Laplacian: Application to Image Denoising , 2012, ArXiv.

[45]  Tolga Tasdizen,et al.  Principal Neighborhood Dictionaries for Nonlocal Means Image Denoising , 2009, IEEE Transactions on Image Processing.

[46]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[47]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[48]  M Bioucas-DiasJosé,et al.  Fast image recovery using variable splitting and constrained optimization , 2010 .

[49]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[50]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[51]  José M. Bioucas-Dias,et al.  Hyperspectral Sharpening using Scene-adapted Gaussian Mixture Priors , 2017, ArXiv.

[52]  Jean-Michel Morel,et al.  Implementation of the "Non-Local Bayes" (NL-Bayes) Image Denoising Algorithm , 2013, Image Process. Line.

[53]  Xiang Zhu,et al.  How to SAIF-ly Boost Denoising Performance , 2013, IEEE Transactions on Image Processing.

[54]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[55]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[56]  Michael Elad,et al.  Postprocessing of Compressed Images via Sequential Denoising , 2015, IEEE Transactions on Image Processing.

[57]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[58]  Guangming Shi,et al.  Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture , 2015, International Journal of Computer Vision.

[59]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[60]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Michael Elad,et al.  Turning a denoiser into a super-resolver using plug and play priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[62]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[63]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[65]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[66]  Charles A. Bouman,et al.  Model based image reconstruction with physics based priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[67]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[68]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[69]  Abderrahim Elmoataz,et al.  Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing , 2008, IEEE Transactions on Image Processing.

[70]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[71]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[72]  Vahid Tarokh,et al.  Low‐dimensional‐structure self‐learning and thresholding: Regularization beyond compressed sensing for MRI Reconstruction , 2011, Magnetic resonance in medicine.

[73]  Charles Kervrann,et al.  Optimal Spatial Adaptation for Patch-Based Image Denoising , 2006, IEEE Transactions on Image Processing.

[74]  Peyman Milanfar,et al.  Is Denoising Dead? , 2010, IEEE Transactions on Image Processing.

[75]  Ronald R. Coifman,et al.  Regularization on Graphs with Function-adapted Diffusion Processes , 2008, J. Mach. Learn. Res..

[76]  Michael Elad,et al.  Boosting of Image Denoising Algorithms , 2015, SIAM J. Imaging Sci..

[77]  Michael Elad,et al.  Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization , 2007 .

[78]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.