Plug-and-Play external and internal priors for image restoration

Image restoration problems were traditionally formulated as the minimization of variational models, including data-fidelity and regularization terms, performed by optimization methods with wellestablished convergence properties. Recently, Plug-and-Play (PnP) methods for image restoration have obtained very good results and popularity by introducing, in iterative proximal algorithms, any off-the-shelf denoiser as priors. Deep Convolutional Neural Network (CNN) denoisers specify external priors (related to an outer training set) which well reflect image statistics; however they fail when dealing with unseen noise variance and image patterns in the given image. Conversely, the so-called internal denoisers induce internal priors tailored on the observed data, by forcing specific features on the desired image. We propose a new PnP scheme, based on the Half-Quadratic Splitting proximal algorithm, combining external and internal priors. Moreover, differently from other existing PnP methods, we propose a deep denoiser acting on the image gradient domain. Finally, we prove that a fixed point convergence is guaranteed for the proposed scheme under suitable conditions. In the experimental part, we use CNN denoisers and the Total Variation functional specifying external and internal priors, respectively. We prove the effectiveness of the proposed method in restoring blurred noisy images, both in simulated and real medical settings.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  Luc Van Gool,et al.  Plug-and-Play Image Restoration With Deep Denoiser Prior , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[4]  José M. Bioucas-Dias,et al.  Scene-Adapted plug-and-play algorithm with convergence guarantees , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[5]  Jianhong Shen,et al.  Deblurring images: Matrices, spectra, and filtering , 2007, Math. Comput..

[6]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[7]  Bei Li,et al.  Medical Images Denoising Based on Total Variation Algorithm , 2011 .

[8]  Michael Möller,et al.  Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Stephan Antholzer,et al.  NETT: solving inverse problems with deep neural networks , 2018, Inverse Problems.

[12]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[14]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[15]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[16]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[17]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[18]  E. Loli Piccolomini,et al.  An efficient method for nonnegatively constrained Total Variation-based denoising of medical images corrupted by Poisson noise , 2012, Comput. Medical Imaging Graph..

[19]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[20]  Carola-Bibiane Schönlieb,et al.  Adversarial Regularizers in Inverse Problems , 2018, NeurIPS.

[21]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[22]  Donald Geman,et al.  Nonlinear image recovery with half-quadratic regularization , 1995, IEEE Trans. Image Process..

[23]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[24]  Shunsuke Ono,et al.  Primal-Dual Plug-and-Play Image Restoration , 2017, IEEE Signal Processing Letters.

[25]  Hassan Mansour,et al.  A Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems , 2017, IEEE Signal Processing Letters.

[26]  Brendt Wohlberg,et al.  An Online Plug-and-Play Algorithm for Regularized Image Reconstruction , 2018, IEEE Transactions on Computational Imaging.

[27]  Charles A. Bouman,et al.  Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation , 2015, IEEE Transactions on Computational Imaging.

[28]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[29]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[30]  Michal Irani,et al.  Combining the power of Internal and External denoising , 2013, IEEE International Conference on Computational Photography (ICCP).

[31]  Lei Zhang,et al.  Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[33]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.