Variational Denoising Network: Toward Blind Noise Modeling and Removal

Blind image denoising is an important yet very challenging problem in computer vision due to the complicated acquisition process of real images. In this work we propose a new variational inference method, which integrates both noise estimation and image denoising into a unique Bayesian framework, for blind image denoising. Specifically, an approximate posterior, parameterized by deep neural networks, is presented by taking the intrinsic clean image and noise variances as latent variables conditioned on the input noisy image. This posterior provides explicit parametric forms for all its involved hyper-parameters, and thus can be easily implemented for blind image denoising with automatic noise estimation for the test noisy image. On one hand, as other data-driven deep learning methods, our method, namely variational denoising network (VDN), can perform denoising efficiently due to its explicit form of posterior expression. On the other hand, VDN inherits the advantages of traditional model-driven approaches, especially the good generalization capability of generative models. VDN has good interpretability and can be flexibly utilized to estimate and remove complicated non-i.i.d. noise collected in real scenarios. Comprehensive experiments are performed to substantiate the superiority of our method in blind image denoising.

[1]  Pheng-Ann Heng,et al.  From Noise Modeling to Blind Image Denoising , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[3]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[4]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[5]  Lei Zhang,et al.  Robust Online Matrix Factorization for Dynamic Background Subtraction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Deyu Meng,et al.  Robust Matrix Factorization with Unknown Noise , 2013, 2013 IEEE International Conference on Computer Vision.

[7]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[8]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[9]  Adrian Barbu,et al.  RENOIR - A dataset for real low-light image noise reduction , 2014, J. Vis. Commun. Image Represent..

[10]  Kede Ma,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models. , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[11]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[12]  Michael S. Bernstein,et al.  Scalable multi-label annotation , 2014, CHI.

[13]  Deyu Meng,et al.  Hyperspectral Image Restoration under Complex Multi-Band Noises , 2018, Remote. Sens..

[14]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[15]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  David Zhang,et al.  A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising , 2018, ECCV.

[17]  Edward H. Adelson,et al.  Noise removal via Bayesian wavelet coring , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[18]  Stefan Roth,et al.  Neural Nearest Neighbors Networks , 2018, NeurIPS.

[19]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Stamatios Lefkimmiatis,et al.  Universal Denoising Networks : A Novel CNN Architecture for Image Denoising , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[22]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jean-Michel Morel,et al.  Multiscale Image Blind Denoising , 2015, IEEE Transactions on Image Processing.

[25]  Jonathan T. Barron,et al.  Unprocessing Images for Learned Raw Denoising , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Fengyuan Zhu,et al.  Blind Image Denoising via Dependent Dirichlet Process Tree. , 2017, IEEE transactions on pattern analysis and machine intelligence.

[27]  Takeo Kanade,et al.  Statistical calibration of CCD imaging process , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[34]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[35]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[36]  Adrian Barbu,et al.  RENOIR - A dataset for real low-light image noise reduction , 2014, Journal of Visual Communication and Image Representation.

[37]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[38]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[39]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[40]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[41]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[42]  Lei Zhang,et al.  Robust Principal Component Analysis with Complex Noise , 2014, ICML.

[43]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[45]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[46]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[47]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Honglak Lee,et al.  Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising , 2013, NIPS.

[49]  Stephen Lin,et al.  A High-Quality Denoising Dataset for Smartphone Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[50]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Thomas S. Huang,et al.  Non-Local Recurrent Network for Image Restoration , 2018, NeurIPS.