Class-Aware Fully Convolutional Gaussian and Poisson Denoising

We propose a fully convolutional neural-network architecture for image denoising which is simple yet powerful. Its structure allows to exploit the gradual nature of the denoising process, in which the shallow layers handle local noise statistics, while deeper layers recover edges and enhance textures. Our method advances the state of the art when trained for different noise levels and distributions (both Gaussian and Poisson). In addition, we show that making the denoiser class-aware by exploiting semantic class information boosts the performance, enhances the textures, and reduces the artifacts.

[1]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[3]  Mohamed-Jalal Fadili,et al.  Wavelets, Ridgelets, and Curvelets for Poisson Noise Removal , 2008, IEEE Transactions on Image Processing.

[4]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  José M. Bioucas-Dias,et al.  Image restoration and reconstruction using variable splitting and class-adapted image priors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[6]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[7]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[8]  Matthew Uyttendaele,et al.  Deep Burst Denoising , 2017, ECCV.

[9]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[10]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[11]  José M. Bioucas-Dias,et al.  Class-specific poisson denoising by patch-based importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[12]  Alessandro Foi,et al.  Optimal Inversion of the Anscombe Transformation in Low-Count Poisson Image Denoising , 2011, IEEE Transactions on Image Processing.

[13]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[14]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Alessandro Foi,et al.  Noise Parameter Mismatch in Variance Stabilization, With an Application to Poisson–Gaussian Noise Estimation , 2014, IEEE Transactions on Image Processing.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Anat Levin,et al.  Natural image denoising: Optimality and inherent bounds , 2011, CVPR 2011.

[18]  Michael Elad,et al.  Sparsity-Based Poisson Denoising With Dictionary Learning , 2013, IEEE Transactions on Image Processing.

[19]  Frédo Durand,et al.  Patch Complexity, Finite Pixel Correlations and Optimal Denoising , 2012, ECCV.

[20]  Peyman Milanfar,et al.  Is Denoising Dead? , 2010, IEEE Transactions on Image Processing.

[21]  M. Fisz The limiting distribution of a function of two independent random variables and its statistical application , 1955 .

[22]  Charles A. Bouman,et al.  Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation , 2015, IEEE Transactions on Computational Imaging.

[23]  Michael Elad,et al.  Boosting of Image Denoising Algorithms , 2015, SIAM J. Imaging Sci..

[24]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Alessandro Foi,et al.  Variance Stabilization for Noisy+Estimate Combination in Iterative Poisson Denoising , 2016, IEEE Signal Processing Letters.

[26]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[27]  Michael Elad,et al.  Poisson Inverse Problems by the Plug-and-Play scheme , 2015, J. Vis. Commun. Image Represent..

[28]  José M. Bioucas-Dias,et al.  Class-specific image denoising using importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[29]  Raja Giryes,et al.  Image Restoration by Iterative Denoising and Backward Projections , 2017, IEEE Transactions on Image Processing.

[30]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[31]  Mário A. T. Figueiredo,et al.  Blind image deblurring using class-adapted image priors , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[32]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[34]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[35]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Shutao Li,et al.  Segmentation Based Sparse Reconstruction of Optical Coherence Tomography Images , 2017, IEEE Transactions on Medical Imaging.

[37]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[38]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[39]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[40]  Peyman Milanfar,et al.  Practical Bounds on Image Denoising: From Estimation to Information , 2011, IEEE Transactions on Image Processing.

[41]  Qi Gao,et al.  A generative perspective on MRFs in low-level vision , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[43]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[44]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[45]  WenSen Feng,et al.  Fast and Accurate Poisson Denoising with Optimized Nonlinear Diffusion , 2015, ArXiv.

[46]  F. J. Anscombe,et al.  THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA , 1948 .

[47]  Karen O. Egiazarian,et al.  Video Denoising, Deblocking, and Enhancement Through Separable 4-D Nonlocal Spatiotemporal Transforms , 2012, IEEE Transactions on Image Processing.

[48]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Jean-Michel Morel,et al.  A Nonlocal Bayesian Image Denoising Algorithm , 2013, SIAM J. Imaging Sci..

[50]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[52]  Alexander M. Bronstein,et al.  Deep class-aware image denoising , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[53]  Edward H. Adelson,et al.  Personal photo enhancement using example images , 2010, TOGS.

[54]  Michael Elad,et al.  Expected Patch Log Likelihood with a Sparse Prior , 2014, EMMCVPR.

[55]  Peyman Milanfar,et al.  Global Image Denoising , 2014, IEEE Transactions on Image Processing.

[56]  Ming-Yu Liu,et al.  Deep Gaussian Conditional Random Field Network: A Model-Based Deep Network for Discriminative Denoising , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[58]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[60]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[61]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[62]  Rebecca Willett,et al.  Poisson Noise Reduction with Non-local PCA , 2012, Journal of Mathematical Imaging and Vision.

[63]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[64]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[65]  Alessandro Foi,et al.  Cross-color BM3D filtering of noisy raw data , 2009, 2009 International Workshop on Local and Non-Local Approximation in Image Processing.

[66]  Michael Elad,et al.  Compression of facial images using the K-SVD algorithm , 2008, J. Vis. Commun. Image Represent..

[67]  Guangming Shi,et al.  Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture , 2015, International Journal of Computer Vision.

[68]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.