Recorrupted-to-Recorrupted: Unsupervised Deep Learning for Image Denoising

Deep denoiser, the deep network for denoising, has been the focus of the recent development on image denoising. In the last few years, there is an increasing interest in developing unsupervised deep denoisers which only call unorganized noisy images without ground truth for training. Nevertheless, the performance of these unsupervised deep denoisers is not competitive to their supervised counterparts. Aiming at developing a more powerful unsupervised deep denoiser, this paper proposed a data augmentation technique, called recorrupted-to-recorrupted (R2R), to address the overfitting caused by the absence of truth images. For each noisy image, we showed that the cost function defined on the noisy/noisy image pairs constructed by the R2R method is statistically equivalent to its supervised counterpart defined on the noisy/truth image pairs. Extensive experiments showed that the proposed R2R method noticeably outperformed existing unsupervised deep denoisers, and is competitive to representative supervised deep denoisers.

[1]  David Zhang,et al.  Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Taesup Moon,et al.  GAN2GAN: Generative Noise Learning for Blind Image Denoising with Single Noisy Images , 2019, ArXiv.

[3]  David Zhang,et al.  Real-world Noisy Image Denoising: A New Benchmark , 2018, ArXiv.

[4]  Yasuyuki Matsushita,et al.  A Holistic Approach to Cross-Channel Image Noise Modeling and Its Application to Image Denoising , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Mingqin Chen,et al.  Self2Self With Dropout: Learning Self-Supervised Denoising From Single Image , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Richard G. Baraniuk,et al.  Unsupervised Learning with Stein's Unbiased Risk Estimator , 2018, ArXiv.

[8]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[9]  Jian-Feng Cai,et al.  Data-driven tight frame construction and image denoising , 2014 .

[10]  Ming-Yu Liu,et al.  Deep Gaussian Conditional Random Field Network: A Model-Based Deep Network for Discriminative Denoising , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[12]  Stephen Lin,et al.  A High-Quality Denoising Dataset for Smartphone Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Florian Jug,et al.  Noise2Void - Learning Denoising From Single Noisy Images , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Xiangchu Feng,et al.  FOCNet: A Fractional Optimal Control Network for Image Denoising , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Se Young Chun,et al.  Training Deep Learning based Denoisers without Ground Truth Data , 2018, NeurIPS.

[17]  Ming Yang,et al.  Image Blind Denoising with Generative Adversarial Network Based Noise Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[20]  Jaakko Lehtinen,et al.  Self-Supervised Deep Image Denoising , 2019, ArXiv.

[21]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[22]  Nick Barnes,et al.  Real Image Denoising With Feature Attention , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Syed Waqas Zamir,et al.  Learning Enriched Features for Real Image Restoration and Enhancement , 2020, ECCV.

[24]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[25]  Guangyong Chen,et al.  An Efficient Statistical Method for Image Noise Level Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[27]  Loïc Royer,et al.  Noise2Self: Blind Denoising by Self-Supervision , 2019, ICML.

[28]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[29]  Yong Xu,et al.  Image denoising using complex-valued deep CNN , 2021, Pattern Recognit..

[30]  Stamatios Lefkimmiatis,et al.  Universal Denoising Networks : A Novel CNN Architecture for Image Denoising , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Adrian Barbu,et al.  RENOIR - A dataset for real low-light image noise reduction , 2014, J. Vis. Commun. Image Represent..

[33]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[34]  Rihuan Ke,et al.  Unsupervised Image Restoration Using Partially Linear Denoisers , 2021, IEEE transactions on pattern analysis and machine intelligence.

[35]  Ling Shao,et al.  Noisy-As-Clean: Learning Unsupervised Denoising from the Corrupted Image , 2019, ArXiv.

[36]  Nick Moran,et al.  Noisier2Noise: Learning to Denoise From Unpaired Noisy Data , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .