Blind Image Denoising via Dynamic Dual Learning

Existing discriminative learning methods for image denoising use either a single residual learning or a nonresidual learning design. However, we observe that these two schemes perform differently with the same noise level, and yet, there have been no explorations regarding whether residual or nonresidual designs are better suited for denoising. Additionally, many discriminative denoisers are designed to learn a model that corresponds to a fixed noise level, which means that multiple models are required to recover corrupted images with noise at different levels. In this paper, we propose a dynamic dual learning network for blind image denoising, namely, DualBDNet. Instead of modeling a sole task prediction network, the proposed DualBDNet investigates the inherent relations between the residual estimation and the nonresidual estimation. In particular, DualBDNet produces task-dependent feature maps, and each part of the features is devoted to one specific task (residual/nonresidual mapping). To address different noise levels with a single network or even cases where the statistics of noise are unknown, we further introduce an embedded subnetwork into DualBDNet. One output of the subnetwork is the learning of a dynamic compositional attention to highlight the more significant task-dependent feature maps, adaptively coinciding with the extent of corruption. The other output is the learning of a weight used for fusion of the results to ensure an end-to-end manner. Extensive experiments demonstrate that the proposed DualBDNet outperforms the state-of-the-art methods on both synthetic and real noisy images without estimating the noise levels as input.

[1]  Tomer Michaeli,et al.  xUnit: Learning a Spatial Activation Function for Efficient Image Restoration , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[3]  Pasi Fränti,et al.  Adaptive Context-Tree-Based Statistical Filtering for Raster Map Image Denoising , 2011, IEEE Transactions on Multimedia.

[4]  Michael I. Mandel,et al.  Speech Denoising by Parametric Resynthesis , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Jian Sun,et al.  BM3D-Net: A Convolutional Neural Network for Transform-Domain Collaborative Filtering , 2018, IEEE Signal Processing Letters.

[6]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[7]  Xiaolan Liu,et al.  A New 4-D Nonlocal Transform-Domain Filter for 3-D Magnetic Resonance Images Denoising , 2018, IEEE Transactions on Medical Imaging.

[8]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[9]  Suk-Ju Kang,et al.  Gradient Prior-Aided CNN Denoiser With Separable Convolution-Based Optimization of Feature Dimension , 2019, IEEE Transactions on Multimedia.

[10]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[11]  Stefan Roth,et al.  Shrinkage Fields for Effective Image Restoration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Thomas S. Huang,et al.  Non-Local Recurrent Network for Image Restoration , 2018, NeurIPS.

[16]  David Zhang,et al.  A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising , 2018, ECCV.

[17]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alessandro Foi,et al.  Noise Parameter Mismatch in Variance Stabilization, With an Application to Poisson–Gaussian Noise Estimation , 2014, IEEE Transactions on Image Processing.

[20]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[22]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[24]  Yi Yang,et al.  Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Stefan Roth,et al.  Neural Nearest Neighbors Networks , 2018, NeurIPS.

[26]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Liang Lin,et al.  Multi-level Wavelet-CNN for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[28]  Yun Fu,et al.  Residual Non-local Attention Networks for Image Restoration , 2019, ICLR.

[29]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[30]  Rogério Schmidt Feris,et al.  Joint Super Resolution and Denoising From a Single Depth Image , 2015, IEEE Transactions on Multimedia.

[31]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[32]  ChanKevin,et al.  Sleep Apnea Detection via Depth Video and Audio Feature Learning , 2017 .

[33]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Suk-Ju Kang,et al.  Geodesic Path-Based Diffusion Acceleration for Image Denoising , 2018, IEEE Transactions on Multimedia.

[35]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[36]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[37]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[38]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[41]  David Zhang,et al.  Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[42]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[43]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[44]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[46]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[47]  Nick Barnes,et al.  Real Image Denoising With Feature Attention , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[48]  Gene Cheung,et al.  Estimating Heart Rate and Rhythm via 3D Motion Tracking in Depth Video , 2017, IEEE Transactions on Multimedia.

[49]  Zhihai He,et al.  Robust Generalized Low-Rank Decomposition of Multimatrices for Image Recovery , 2017, IEEE Transactions on Multimedia.

[50]  Weisi Lin,et al.  Maximum a Posterior and Perceptually Motivated Reconstruction Algorithm: A Generic Framework , 2017, IEEE Transactions on Multimedia.

[51]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[52]  Jong-Sen Lee,et al.  Refined filtering of image noise using local statistics , 1981 .

[53]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Zhenzhong Chen,et al.  Multimodal Deep Denoise Framework for Affective Video Content Analysis , 2019, ACM Multimedia.

[56]  Lei Zhang,et al.  Waterloo Exploration Database: New Challenges for Image Quality Assessment Models , 2017, IEEE Transactions on Image Processing.

[57]  Vladimir Stankovic,et al.  Sleep Apnea Detection via Depth Video and Audio Feature Learning , 2017, IEEE Transactions on Multimedia.