When AWGN-based Denoiser Meets Real Noises

Discriminative learning-based image denoisers have achieved promising performance on synthetic noises such as Additive White Gaussian Noise (AWGN). The synthetic noises adopted in most previous work are pixel-independent, but real noises are mostly spatially/channel-correlated and spatially/channel-variant. This domain gap yields unsatisfied performance on images with real noises if the model is only trained with AWGN. In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN. First, we train a deep model that consists of a noise estimator and a denoiser with mixed AWGN and Random Value Impulse Noise (RVIN). We then investigate Pixel-shuffle Down-sampling (PD) strategy to adapt the trained model to real noises. Extensive experiments demonstrate the effectiveness and generalization of the proposed approach. Notably, our method achieves state-of-the-art performance on real sRGB images in the DND benchmark among models trained with synthetic noises. Codes are available at this https URL.

[1]  Larry S. Davis,et al.  Generalized Deep Image to Image Regression , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  David Zhang,et al.  Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[4]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[5]  Xinhao Liu,et al.  Practical Signal-Dependent Noise Parameter Estimation From a Single Noisy Image , 2014, IEEE Transactions on Image Processing.

[6]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[7]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[8]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[9]  Karen O. Egiazarian,et al.  Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data , 2008, IEEE Transactions on Image Processing.

[10]  Ming Yang,et al.  Image Blind Denoising with Generative Adversarial Network Based Noise Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Jungwon Lee,et al.  DN-ResNet: Efficient Deep Residual Network for Image Denoising , 2018, ACCV.

[13]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[14]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[15]  Chuan Wang,et al.  Video Inpainting by Jointly Learning Temporal Structure and Spatial Details , 2018, AAAI.

[16]  Yang Wang,et al.  GIF2Video: Color Dequantization and Temporal Interpolation of GIF Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jonathan T. Barron,et al.  Unprocessing Images for Learned Raw Denoising , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Tomer Michaeli,et al.  Multi-scale Weighted Nuclear Norm Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Pheng-Ann Heng,et al.  From Noise Modeling to Blind Image Denoising , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[21]  David Zhang,et al.  External Prior Guided Internal Prior Learning for Real-World Noisy Image Denoising , 2017, IEEE Transactions on Image Processing.

[22]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[23]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[24]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[25]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Daniel P.K. Lun,et al.  Enhancement of a CNN-Based Denoiser Based on Spatial and Spectral Analysis , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[27]  Dong-Wook Kim,et al.  NTIRE 2019 Challenge on Real Image Denoising: Methods and Results , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  Richard Szeliski,et al.  Automatic Estimation and Removal of Noise from a Single Image , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Nick Barnes,et al.  Real Image Denoising With Feature Attention , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  David Zhang,et al.  Patch Group Based Bayesian Learning for Blind Image Denoising , 2016, ACCV Workshops.

[32]  Stephen Lin,et al.  A High-Quality Denoising Dataset for Smartphone Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[34]  Eero P. Simoncelli,et al.  Multiscale Denoising of Photographic Images , 2009 .

[35]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[36]  Xinhao Liu,et al.  Single-Image Noise Level Estimation for Blind Denoising , 2013, IEEE Transactions on Image Processing.

[37]  Xianming Liu,et al.  When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach , 2017, IJCAI.

[38]  Thomas Huang,et al.  Adaptation Strategies for Applying AWGN-Based Denoiser to Realistic Noise , 2019, AAAI.

[39]  Xianming Liu,et al.  Connecting Image Denoising and High-Level Vision Tasks via Deep Learning , 2018, IEEE Transactions on Image Processing.

[40]  Xiaoou Tang,et al.  Path-Restore: Learning Network Path Selection for Image Restoration , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  José M. Bioucas-Dias,et al.  Class-specific poisson denoising by patch-based importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[42]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[43]  Raja Giryes,et al.  Class-Aware Fully Convolutional Gaussian and Poisson Denoising , 2018, IEEE Transactions on Image Processing.

[44]  David Zhang,et al.  A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising , 2018, ECCV.

[45]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[47]  Thomas S. Huang,et al.  Survey of Face Detection on Low-Quality Images , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[48]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Jong-Sen Lee,et al.  Refined filtering of image noise using local statistics , 1981 .

[50]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Feng Liu,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries in Wavelet Domain , 2009, 2009 Fifth International Conference on Image and Graphics.

[52]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Jean-Michel Morel,et al.  Multiscale Image Blind Denoising , 2015, IEEE Transactions on Image Processing.