Self-Supervised Deep Equilibrium Models With Theoretical Guarantees and Applications to MRI Reconstruction

Deep equilibrium models (DEQ) have emerged as a powerful alternative to deep unfolding (DU) for image reconstruction. DEQ models—implicit neural networks with effectively infinite number of layers—were shown to achieve state-of-the-art image reconstruction without the memory complexity associated with DU. While the performance of DEQ has been widely investigated, the existing work has primarily focused on the settings where ground truth data is available for training. We present self-supervised deep equilibrium model (SelfDEQ) as the first self-supervised reconstruction framework for training model-based implicit networks from undersampled and noisy MRI measurements. Our theoretical results show that SelfDEQ can compensate for unbalanced sampling across multiple acquisitions and match the performance of fully supervised DEQ. Our numerical results on in-vivo MRI data show that SelfDEQ leads to state-of-the-art performance using only undersampled and noisy training data.

[1]  Siming Zheng,et al.  Deep Equilibrium Models for Snapshot Compressive Imaging , 2023, AAAI.

[2]  Weijie Gan,et al.  Online Deep Equilibrium Learning for Regularization by Denoising , 2022, NeurIPS.

[3]  M. Chiew,et al.  A Theoretical Framework for Self-Supervised MR Image Reconstruction Using Sub-Sampling via Variable Density Noisier2Noise , 2022, IEEE Transactions on Computational Imaging.

[4]  S. Moeller,et al.  Distributed Memory-Efficient Physics-Guided Deep Learning Reconstruction for Large-Scale 3d Non-Cartesian MRI , 2022, 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI).

[5]  M. Davies,et al.  Sensing Theorems for Unsupervised Learning in Linear Inverse Problems , 2022, J. Mach. Learn. Res..

[6]  D. Rueckert,et al.  Physics-Driven Deep Learning for Computational Magnetic Resonance Imaging: Combining physics and machine learning for improved medical imaging , 2022, IEEE Signal Processing Magazine.

[7]  Di Guo,et al.  A review on deep learning MRI reconstruction without fully sampled k-space , 2021, BMC Medical Imaging.

[8]  Mathews Jacob,et al.  Improved Model Based Deep Learning Using Monotone Operator Learning (Mol) , 2021, 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI).

[9]  Yu Sun,et al.  Deformation-Compensated Learning for Image Reconstruction Without Ground Truth , 2021, IEEE Transactions on Medical Imaging.

[10]  J. Starck,et al.  SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models , 2021, ICLR.

[11]  H. An,et al.  Phase2Phase , 2021, Investigative radiology.

[12]  Burhaneddin Yaman,et al.  Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement: An overview from a signal processing perspective , 2021, IEEE Signal Processing Magazine.

[13]  Mike E. Davies,et al.  Equivariant Imaging: Learning Beyond the Range Space , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Samy Wu Fung,et al.  JFB: Jacobian-Free Backpropagation for Implicit Networks , 2021, AAAI.

[15]  Michael Lustig,et al.  Memory-efficient Learning for High-Dimensional MRI Reconstruction , 2021, MICCAI.

[16]  Bo Chen,et al.  Memory-Efficient Network for Large-scale Video Compressive Sensing , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  R. Willett,et al.  Deep Equilibrium Architectures for Inverse Problems in Imaging , 2021, IEEE Transactions on Computational Imaging.

[18]  Kun Yang,et al.  Weakly Supervised Deep Learning-Based Optical Coherence Tomography Angiography , 2020, IEEE Transactions on Medical Imaging.

[19]  K. Uğurbil,et al.  Multi‐mask self‐supervised learning for physics‐guided neural networks in highly accelerated magnetic resonance imaging , 2020, NMR in biomedicine.

[20]  Matthew J. Johnson,et al.  Learning Differential Equations that are Easy to Solve , 2020, NeurIPS.

[21]  Alexandros G. Dimakis,et al.  Deep Learning Techniques for Inverse Problems in Imaging , 2020, IEEE Journal on Selected Areas in Information Theory.

[22]  Michael Unser,et al.  CryoGAN: A New Reconstruction Paradigm for Single-Particle Cryo-EM via Deep Adversarial Learning , 2020, bioRxiv.

[23]  K. Batenburg,et al.  Noise2Inverse: Self-Supervised Deep Convolutional Denoising for Tomography , 2020, IEEE Transactions on Computational Imaging.

[24]  Steen Moeller,et al.  Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data , 2019, Magnetic resonance in medicine.

[25]  Ulugbek S. Kamilov,et al.  RARE: Image Reconstruction Using Deep Priors Learned Without Groundtruth , 2019, IEEE Journal of Selected Topics in Signal Processing.

[26]  Laura Waller,et al.  Memory-Efficient Learning for Large-Scale Computational Imaging , 2019, IEEE Transactions on Computational Imaging.

[27]  Nick Moran,et al.  Noisier2Noise: Learning to Denoise From Unpaired Noisy Data , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  J. Z. Kolter,et al.  Deep Equilibrium Models , 2019, NeurIPS.

[29]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[30]  Florian Jug,et al.  Noise2Void - Learning Denoising From Single Noisy Images , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[32]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[33]  Alexandros G. Dimakis,et al.  AmbientGAN: Generative models from lossy measurements , 2018, ICLR.

[34]  Aggelos K. Katsaggelos,et al.  Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods , 2018, IEEE Signal Processing Magazine.

[35]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[36]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[37]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[38]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[39]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[40]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[41]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[42]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Michael Elad,et al.  ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA , 2014, Magnetic resonance in medicine.

[45]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[46]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[47]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[48]  Donald G. M. Anderson.  Iterative Procedures for Nonlinear Integral Equations , 1965, JACM.

[49]  M. Jacob,et al.  Stable and memory-efficient image recovery using monotone operator learning (MOL) , 2022, ArXiv.

[50]  Fred Greguras,et al.  IN SHANGHAI, CHINA , 2007 .