Optimal Combination of Image Denoisers

Given a set of image denoisers, each having a different denoising capability, is there a provably optimal way of combining these denoisers to produce an overall better result? An answer to this question is fundamental to designing an ensemble of weak estimators for complex scenes. In this paper, we present an optimal combination scheme by leveraging the deep neural networks and the convex optimization. The proposed framework, called the Consensus Neural Network (CsNet), introduces three new concepts in image denoising: 1) a provably optimal procedure to combine the denoised outputs via convex optimization; 2) a deep neural network to estimate the mean squared error (MSE) of denoised images without needing the ground truths; and 3) an image boosting procedure using a deep neural network to improve the contrast and to recover the lost details of the combined images. Experimental results show that CsNet can consistently improve the denoising performance for both deterministic and neural network denoisers.

[1]  Truong Q. Nguyen,et al.  Adaptive Image Denoising by Targeted Databases , 2014, IEEE Transactions on Image Processing.

[2]  David Zhang,et al.  Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  Thierry Blu,et al.  The SURE-LET Approach to Image Denoising , 2007, IEEE Transactions on Image Processing.

[4]  Alexander M. Bronstein,et al.  Deep class-aware image denoising , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[5]  Sebastian Bosse,et al.  A deep neural network for image quality assessment , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[6]  Truong Q. Nguyen,et al.  Adaptive Image Denoising by Mixture Adaptation , 2016, IEEE Transactions on Image Processing.

[7]  Stamatios Lefkimmiatis,et al.  Non-local Color Image Denoising with Convolutional Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Florence Tupin,et al.  Closed-form expressions of the eigen decomposition of 2 x 2 and 3 x 3 Hermitian matrices , 2017 .

[9]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[10]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[11]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[12]  Lei Zhang,et al.  Deep Convolutional Neural Models for Picture-Quality Prediction: Challenges and Solutions to Data-Driven Image Quality Assessment , 2017, IEEE Signal Processing Magazine.

[13]  Alessandro Foi,et al.  Clipped noisy images: Heteroskedastic modeling and practical denoising , 2009, Signal Process..

[14]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[15]  Xiang Zhu,et al.  How to SAIF-ly Boost Denoising Performance , 2013, IEEE Transactions on Image Processing.

[16]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[17]  Julien Mairal,et al.  Optimization with First-Order Surrogate Functions , 2013, ICML.

[18]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[19]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[23]  Frédéric Lavancier,et al.  A general procedure to combine estimators , 2014, Comput. Stat. Data Anal..

[24]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[25]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[26]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[27]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Thierry Blu,et al.  Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms , 2008, IEEE Transactions on Image Processing.

[29]  P. Bühlmann,et al.  Boosting With the L2 Loss , 2003 .

[30]  David Zhang,et al.  Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Yong Li,et al.  Image quality assessment using deep convolutional networks , 2017 .

[33]  Patrick L. Odell,et al.  Estimator models for combining vector estimators , 1989 .

[34]  Taesup Moon,et al.  Neural Adaptive Image Denoiser , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  C. Stein.  Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[36]  Honglak Lee,et al.  Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising , 2013, NIPS.

[37]  Lai-Man Po,et al.  No-reference image quality assessment with deep convolutional neural networks , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[38]  Laurent Condat.  Least-Squares on the Simplex for Multispectral Unmixing , 2017 .

[39]  Ester Samuel-Cahn.  Combining unbiased estimators , 1994 .

[40]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[41]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[42]  Michael Elad,et al.  Boosting of Image Denoising Algorithms , 2015, SIAM J. Imaging Sci..

[43]  Stanley H. Chan,et al.  Monte Carlo Non-Local Means: Random Sampling for Large-Scale Image Filtering , 2013, IEEE Transactions on Image Processing.

[44]  Stephen P. Boyd,et al.  Method of centers for minimizing generalized eigenvalues , 1993, Linear Algebra and its Applications.

[45]  Stanley H. Chan,et al.  Fast And Robust Recursive Filter for Image Denoising , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[47]  Michael Elad,et al.  A General Iterative Regularization Framework For Image Denoising , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[48]  Kunal N. Chaudhury,et al.  Image denoising using optimally weighted bilateral filters: A sure and fast approach , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[49]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[50]  Thierry Blu,et al.  A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding , 2007, IEEE Transactions on Image Processing.

[51]  Ingram Olkin,et al.  Combining correlated unbiased estimators of the mean of a normal distribution , 2002 .

[52]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[53]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[54]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[55]  Alessandro Foi,et al.  Gaussian-Cauchy mixture modeling for robust signal-dependent noise estimation , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[56]  Donald B. Rubin,et al.  THE VARIANCE OF A LINEAR COMBINATION OF INDEPENDENT ESTIMATORS USING ESTIMATED WEIGHTS , 1974 .

[57]  Yi Li,et al.  Convolutional Neural Networks for No-Reference Image Quality Assessment , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[59]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[60]  Wu Cheng,et al.  Towards Optimal Denoising of Image Contrast , 2018, IEEE Transactions on Image Processing.

[61]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[63]  Keigo Hirakawa,et al.  Improved Denoising via Poisson Mixture Modeling of Image Sensor Noise , 2017, IEEE Transactions on Image Processing.

[64]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[65]  J. Morel,et al.  On image denoising methods , 2004 .

[66]  David Zhang,et al.  External Prior Guided Internal Prior Learning for Real-World Noisy Image Denoising , 2017, IEEE Transactions on Image Processing.

[67]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[68]  David Zhang,et al.  A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising , 2018, ECCV.

[69]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[70]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[71]  Sebastian Nowozin,et al.  Loss-Specific Training of Non-Parametric Image Restoration Models: A New State of the Art , 2012, ECCV.

[72]  José M. Bioucas-Dias,et al.  Class-specific image denoising using importance sampling , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[73]  Martin Jaggi,et al.  Convex Optimization without Projection Steps , 2011, ArXiv.

[74]  Peyman Milanfar,et al.  On Iterative Regularization and Its Application , 2008, IEEE Transactions on Circuits and Systems for Video Technology.