A Neural-Network-Based Convex Regularizer for Image Reconstruction

—The emergence of deep-learning-based methods for solving inverse problems has enabled a significant increase in reconstruction quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these shortcomings while retaining the performance. In this work, this problem is tackled by revisiting regularizers that are the sum of convex-ridge functions. The gradient of such regularizers is parametrized by a neural network that has a single hidden layer with increasing and learnable activation functions. This neural network is trained within a few minutes as a multi-step Gaussian denoiser. The numerical experiments for denoising, CT, and MRI reconstruction show improvements over methods that offer similar reliability guarantees.

[1]  K. Chaudhury,et al.  On the Construction of Averaged Deep Denoisers for Image Regularization , 2022, 2207.07321.

[2]  M. Unser,et al.  Approximation of Lipschitz Functions using Deep Spline Neural Networks , 2022, SIAM J. Math. Data Sci..

[3]  N. Papadakis,et al.  Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization , 2022, ICML.

[4]  Mathews Jacob,et al.  Improved Model Based Deep Learning Using Monotone Operator Learning (Mol) , 2021, 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI).

[5]  N. Papadakis,et al.  Gradient Step Denoiser for convergent Plug-and-Play , 2021, ICLR.

[6]  Samy Wu Fung,et al.  JFB: Jacobian-Free Backpropagation for Implicit Networks , 2021, AAAI.

[7]  Mikael Le Pendu,et al.  Learned Gradient of a Regularizer for Plug-and-Play Gradient Descent , 2022, ArXiv.

[8]  Carola-Bibiane Schönlieb,et al.  Learning convex regularizers satisfying the variational source condition for inverse problems , 2021, ArXiv.

[9]  Matthias Joachim Ehrhardt,et al.  Regularising Inverse Problems with Generative Machine Learning Models , 2021, J. Math. Imaging Vis..

[10]  Brendt Wohlberg,et al.  Recovery Analysis for Plug-and-Play Priors using the Restricted Eigenvalue Condition , 2021, NeurIPS.

[11]  Jean-Christophe Pesquet,et al.  Learning Maximally Monotone Operators for Image Recovery , 2020, SIAM J. Imaging Sci..

[12]  Gabriele Steidl,et al.  Convolutional Proximal Neural Networks and Plug-and-Play Algorithms , 2020, Linear Algebra and its Applications.

[13]  M. Unser,et al.  Learning Lipschitz-Controlled Activation Functions in Neural Networks for Plug-and-Play Image Reconstruction Methods , 2021 .

[14]  E. Rivlin,et al.  It Has Potential: Gradient-Driven Denoisers for Convergent Solutions to Inverse Problems , 2021, NeurIPS.

[15]  S. Neumayer,et al.  Stabilizing invertible neural networks using mixture models , 2020, Inverse Problems.

[16]  C. Schonlieb,et al.  Learned convex regularizers for inverse problems , 2020, ArXiv.

[17]  Alexandros G. Dimakis,et al.  Deep Learning Techniques for Inverse Problems in Imaging , 2020, IEEE Journal on Selected Areas in Information Theory.

[18]  Brendt Wohlberg,et al.  Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling , 2020, 2020 54th Asilomar Conference on Signals, Systems, and Computers.

[19]  Michael Rabbat,et al.  fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning. , 2020, Radiology. Artificial intelligence.

[20]  K. Kunisch,et al.  Total Deep Variation for Linear Inverse Problems , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Ben Adcock,et al.  The troublesome kernel: why deep learning for inverse problems is typically unstable , 2020, ArXiv.

[22]  Gabriele Steidl,et al.  Parseval Proximal Neural Networks , 2019, Journal of Fourier Analysis and Applications.

[23]  Karl Kunisch,et al.  Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration , 2020, Journal of Mathematical Imaging and Vision.

[24]  Francesco Renna,et al.  On instabilities of deep learning in image reconstruction and the potential costs of AI , 2019, Proceedings of the National Academy of Sciences.

[25]  Stephan Antholzer,et al.  NETT: solving inverse problems with deep neural networks , 2018, Inverse Problems.

[26]  Michael Unser,et al.  Learning Activation Functions in Deep (Spline) Neural Networks , 2020, IEEE Open Journal of Signal Processing.

[27]  J. Z. Kolter,et al.  Deep Equilibrium Models , 2019, NeurIPS.

[28]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[29]  Simon R. Arridge,et al.  Solving inverse problems using data-driven models , 2019, Acta Numerica.

[30]  Michael T. McCann,et al.  Biomedical Image Reconstruction: From the Foundations to Deep Neural Networks , 2019, Found. Trends Signal Process..

[31]  Cem Anil,et al.  Sorting out Lipschitz function approximation , 2018, ICML.

[32]  Michael Unser,et al.  A representer theorem for deep neural networks , 2018, J. Mach. Learn. Res..

[33]  Ritu Chadha,et al.  Limitations of the Lipschitz constant as a defense against adversarial examples , 2018, Nemesis/UrbReas/SoGood/IWAISe/GDM@PKDD/ECML.

[34]  Carola-Bibiane Schönlieb,et al.  Adversarial Regularizers in Inverse Problems , 2018, NeurIPS.

[35]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[36]  Jin Keun Seo,et al.  Deep learning for undersampled MRI reconstruction , 2017, Physics in medicine and biology.

[37]  Michael Unser,et al.  CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[38]  Michael Unser,et al.  Learning Convex Regularizers for Optimal Bayesian Denoising , 2017, IEEE Transactions on Signal Processing.

[39]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[40]  Thomas Pock,et al.  Variational Networks: Connecting Variational Methods and Deep Learning , 2017, GCPR.

[41]  Hu Chen,et al.  Low-dose CT via convolutional neural network. , 2017, Biomedical optics express.

[42]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[43]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[44]  Lei Xu,et al.  Input Convex Neural Networks : Supplementary Material , 2017 .

[45]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[46]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[47]  C. McCollough.  TU-FG-207A-04: Overview of the Low Dose CT Grand Challenge. , 2016, Medical physics.

[48]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[49]  Horst Bischof,et al.  Learning ℓ1-based analysis and synthesis sparsity priors using bi-level optimization , 2014, NIPS 2014.

[50]  Yunjin Chen,et al.  Insights Into Analysis Operator Learning: From Patch-Based Sparse Models to Higher Order MRFs , 2014, IEEE Transactions on Image Processing.

[51]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[52]  Michael Unser,et al.  A Unifying Parametric Framework for 2D Steerable Wavelet Transforms , 2013, SIAM J. Imaging Sci..

[53]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Gabriel Peyré,et al.  Learning Analysis Sparsity Priors , 2011 .

[55]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[56]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[57]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[58]  F. Schmitt,et al.  Linear inverse problems in imaging , 2008, IEEE Signal Processing Magazine.

[59]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[60]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[61]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[62]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[63]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  L. Willner.  On the distance between polytopes , 1968 .

[65]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .