Deep Equilibrium Architectures for Inverse Problems in Imaging

Recent efforts on solving inverse problems in imaging via deep neural networks use architectures inspired by a fixed number of iterations of an optimization method. The number of iterations is typically quite small due to difficulties in training networks corresponding to more iterations; the resulting solvers cannot be run for more iterations at test time without incurring significant errors. This paper describes an alternative approach corresponding to an infinite number of iterations, yielding a consistent improvement in reconstruction accuracy above state-of-the-art alternatives and where the computational budget can be selected at test time to optimize context-dependent trade-offs between accuracy and computation. The proposed approach leverages ideas from Deep Equilibrium Models, where the fixed-point iteration is constructed to incorporate a known forward model and insights from classical optimization-based reconstruction methods.

[1]  Wotao Yin,et al.  Feasibility-based fixed point networks , 2021, Fixed Point Theory and Algorithms for Sciences and Engineering.

[2]  Homer F. Walker,et al.  Anderson Acceleration for Fixed-Point Iterations , 2011, SIAM J. Numer. Anal..

[3]  Quanzheng Li,et al.  Computationally Efficient Cascaded Training for Deep Unrolled Network in CT Imaging , 2018, Medical physics.

[4]  Morteza Mardani,et al.  Neural Proximal Gradient Descent for Compressive Imaging , 2018, NeurIPS.

[5]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[6]  Rebecca Willett,et al.  Neumann Networks for Linear Inverse Problems in Imaging , 2020, IEEE Transactions on Computational Imaging.

[7]  Jan MacDonald,et al.  Solving Inverse Problems With Deep Neural Networks - Robustness Included? , 2020, ArXiv.

[8]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[9]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[10]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[11]  Mario Bertero,et al.  The Stability of Inverse Problems , 1980 .

[12]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[13]  Tom Tirer,et al.  Super-Resolution via Image-Adapted Denoising CNNs: Incorporating External and Internal Learning , 2018, IEEE Signal Processing Letters.

[14]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[17]  Eldad Haber,et al.  Stable architectures for deep neural networks , 2017, ArXiv.

[18]  Alexandros G. Dimakis,et al.  Deep Learning Techniques for Inverse Problems in Imaging , 2020, IEEE Journal on Selected Areas in Information Theory.

[19]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[21]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[22]  Francesco Renna,et al.  On instabilities of deep learning in image reconstruction and the potential costs of AI , 2019, Proceedings of the National Academy of Sciences.

[23]  Jeffrey A. Fessler,et al.  Momentum-Net: Fast and Convergent Iterative Neural Network for Inverse Problems , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ulugbek S. Kamilov,et al.  RARE: Image Reconstruction Using Deep Priors Learned Without Groundtruth , 2019, IEEE Journal of Selected Topics in Signal Processing.

[25]  Vladlen Koltun,et al.  Deep Equilibrium Models , 2019, NeurIPS.

[26]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[27]  Atsushi Fujita,et al.  Recurrent Stacking of Layers for Compact Neural Machine Translation Models , 2018, AAAI.

[28]  Michael Möller,et al.  Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Andrew J. Reader,et al.  Model-Based Deep Learning PET Image Reconstruction Using Forward-Backward Splitting Expectation Maximisation , 2019, 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC).

[30]  Laurent El Ghaoui,et al.  Implicit Deep Learning , 2019, SIAM J. Math. Data Sci..

[31]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[33]  Vladlen Koltun,et al.  Trellis Networks for Sequence Modeling , 2018, ICLR.

[34]  Pascal Vincent,et al.  fastMRI: An Open Dataset and Benchmarks for Accelerated MRI , 2018, ArXiv.

[35]  Bailin Deng,et al.  Accelerating ADMM for efficient simulation and optimization , 2019, ACM Trans. Graph..

[36]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[37]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[38]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Jean-Luc Starck,et al.  State-of-the-Art Machine Learning MRI Reconstruction in 2020: Results of the Second fastMRI Challenge , 2020, ArXiv.

[40]  Bo Li,et al.  Improving Robustness of Deep-Learning-Based Image Reconstruction , 2020, ICML.