An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Dataset

This work aims at developing a generalizable Magnetic Resonance Imaging (MRI) reconstruction method in the meta-learning framework. Specifically, we develop a deep reconstruction network induced by a learnable optimization algorithm (LOA) to solve the nonconvex nonsmooth variational model of MRI image reconstruction. In this model, the nonconvex nonsmooth regularization term is parameterized as a structured deep network where the network parameters can be learned from data. We partition these network parameters into two parts: a task-invariant part for the common feature encoder component of the regularization, and a task-specific part to account for the variations in the heterogeneous training and testing data. We train the regularization parameters in a bilevel optimization framework which significantly improves the robustness of the training process and the generalization ability of the network. We conduct a series of numerical experiments using heterogeneous MRI data sets with various undersampling patterns, ratios, and acquisition settings. The experimental results show that our network yields greatly improved reconstruction quality over existing methods and can generalize well to new reconstruction problems whose undersampling patterns/trajectories are not present during training.

[1]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[2]  Sergey Levine,et al.  Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning , 2019, CoRL.

[3]  Dong Liang,et al.  Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks , 2020, IEEE Signal Processing Magazine.

[4]  Ruth Wario,et al.  A survey on deep learning in medical image reconstruction , 2021 .

[5]  Justin P. Haldar,et al.  Compressed-Sensing MRI With Random Encoding , 2011, IEEE Transactions on Medical Imaging.

[6]  Hong Yu,et al.  Meta Networks , 2017, ICML.

[7]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[8]  Ronald M. Summers,et al.  A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises , 2020, Proceedings of the IEEE.

[9]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[10]  Ying Wei,et al.  Hierarchically Structured Meta-learning , 2019, ICML.

[11]  Mario C. Cirillo,et al.  On the use of the normalized mean square error in evaluating dispersion model performance , 1993 .

[12]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[13]  Swami Sankaranarayanan,et al.  MetaReg: Towards Domain Generalization using Meta-Regularization , 2018, NeurIPS.

[14]  Chelsea Finn,et al.  Meta-Learning without Memorization , 2020, ICLR.

[15]  Jihun Hamm,et al.  Penalty Method for Inversion-Free Deep Bilevel Optimization , 2019, ArXiv.

[16]  Sebastian Thrun,et al.  Learning to Learn: Introduction and Overview , 1998, Learning to Learn.

[17]  Aske Plaat,et al.  A survey of deep meta-learning , 2020, Artificial Intelligence Review.

[18]  Yongxin Yang,et al.  Learning to Generalize: Meta-Learning for Domain Generalization , 2017, AAAI.

[19]  Sergey Levine,et al.  Probabilistic Model-Agnostic Meta-Learning , 2018, NeurIPS.

[20]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[21]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[22]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[23]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[24]  Taghi M. Khoshgoftaar,et al.  A survey on heterogeneous transfer learning , 2017, Journal of Big Data.

[25]  Dong Liang,et al.  DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution. , 2020, Magnetic resonance imaging.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sergey Levine,et al.  Online Meta-Learning , 2019, ICML.

[28]  Morteza Mardani,et al.  Compressed Sensing: From Research to Clinical Practice With Deep Neural Networks: Shortening Scan Times for Magnetic Resonance Imaging , 2020, IEEE Signal Processing Magazine.

[29]  Razvan Pascanu,et al.  Meta-Learning with Latent Embedding Optimization , 2018, ICLR.

[30]  Paolo Favaro,et al.  Deep Bilevel Learning , 2018, ECCV.

[31]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[32]  Hugo Larochelle,et al.  Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples , 2019, ICLR.

[33]  Amos J. Storkey,et al.  How to train your MAML , 2018, ICLR.

[34]  Thomas L. Griffiths,et al.  Recasting Gradient-Based Meta-Learning as Hierarchical Bayes , 2018, ICLR.

[35]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[36]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[37]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[38]  Andrea Vedaldi,et al.  Learning multiple visual domains with residual adapters , 2017, NIPS.

[39]  Yoshua Bengio,et al.  Bayesian Model-Agnostic Meta-Learning , 2018, NeurIPS.

[40]  M. Bernstein,et al.  Effect of windowing and zero‐filled reconstruction of MRI data on spatial resolution and acquisition strategy , 2001, Journal of magnetic resonance imaging : JMRI.

[41]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[42]  Pieter Abbeel,et al.  A Simple Neural Attentive Meta-Learner , 2017, ICLR.

[43]  Seungjin Choi,et al.  Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace , 2018, ICML.

[44]  Yu-Chiang Frank Wang,et al.  A Closer Look at Few-shot Classification , 2019, ICLR.

[45]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[46]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[47]  Xiaojing Ye,et al.  Learnable Descent Algorithm for Nonsmooth Nonconvex Image Reconstruction , 2021, SIAM J. Imaging Sci..

[48]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[49]  S. Crozier,et al.  Deep learning in magnetic resonance image reconstruction , 2021, Journal of medical imaging and radiation oncology.

[50]  Wei Zhou,et al.  Feature-Critic Networks for Heterogeneous Domain Generalization , 2019, ICML.

[51]  Arvid Lundervold,et al.  An overview of deep learning in medical imaging focusing on MRI , 2018, Zeitschrift fur medizinische Physik.

[52]  Xian Wu,et al.  Automated Relational Meta-learning , 2020, ICLR.

[53]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[54]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[55]  Hang Li,et al.  Meta-SGD: Learning to Learn Quickly for Few Shot Learning , 2017, ArXiv.

[56]  Han Liu,et al.  Challenges of Big Data Analysis. , 2013, National science review.

[57]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[58]  Amos Storkey,et al.  Meta-Learning in Neural Networks: A Survey , 2020, IEEE transactions on pattern analysis and machine intelligence.

[59]  Bernard Ghanem,et al.  ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[60]  Tony F. Chan,et al.  Color TV: total variation methods for restoration of vector-valued images , 1998, IEEE Trans. Image Process..

[61]  Wei Shen,et al.  Few-Shot Image Recognition by Predicting Parameters from Activations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[62]  Joseph J. Lim,et al.  Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation , 2019, NeurIPS.

[63]  Junzhou Huang,et al.  Improving Generalization in Meta-learning via Task Augmentation , 2020, ICML.

[64]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[65]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[66]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.