Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction

Accelerating MRI scans is one of the principal outstanding problems in the MRI research community. Towards this goal, we hosted the second fastMRI competition targeted towards reconstructing MR images with subsampled k-space data. We provided participants with data from 7,299 clinical brain scans (de-identified via a HIPAA-compliant procedure by NYU Langone Health), holding back the fully-sampled data from 894 of these scans for challenge evaluation purposes. In contrast to the 2019 challenge, we focused our radiologist evaluations on pathological assessment in brain images. We also debuted a new Transfer track that required participants to submit models evaluated on MRI scanners from outside the training set. We received 19 submissions from eight different groups. Results showed one team scoring best in both SSIM scores and qualitative radiologist evaluations. We also performed analysis on alternative metrics to mitigate the effects of background noise and collected feedback from the participants to inform future challenges. Lastly, we identify common failure modes across the submissions, highlighting areas of need for future research in the MRI reconstruction community.

[1]  Michael Rabbat,et al.  fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning. , 2020, Radiology. Artificial intelligence.

[2]  Guang Yang,et al.  DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction , 2018, IEEE Transactions on Medical Imaging.

[3]  Jean-Luc Starck,et al.  Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets , 2020, Applied Sciences.

[4]  Aaron Defazio,et al.  MRI Banding Removal via Adversarial Training , 2020, NeurIPS.

[5]  Jan-Jakob Sonke,et al.  Recurrent inference machines for reconstructing heterogeneous MRI data , 2019, Medical Image Anal..

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Jechang Jeong,et al.  Deep Iterative Down-Up CNN for Image Denoising , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Steen Moeller,et al.  Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data , 2019, Magnetic resonance in medicine.

[9]  Ronald M. Summers,et al.  Σ-net: Systematic Evaluation of Iterative Deep Neural Networks for Fast Parallel MR Image Reconstruction , 2019, ArXiv.

[10]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  W. Manning,et al.  Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays , 1997, Magnetic resonance in medicine.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[14]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[15]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[16]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[17]  Daniel K Sodickson,et al.  Using Deep Learning to Accelerate Knee MRI at 3T: Results of an Interchangeability Study. , 2020, AJR. American journal of roentgenology.

[18]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[19]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[20]  Max Welling,et al.  Recurrent Inference Machines for Solving Inverse Problems , 2017, ArXiv.

[21]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[22]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[23]  Michael G. Rabbat,et al.  Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge , 2020, Magnetic resonance in medicine.

[24]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Liyuan Liu,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[26]  Liang Lin,et al.  Multi-level Wavelet-CNN for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Mariya Doneva,et al.  An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction , 2020, IEEE Access.

[28]  Taejoon Eo,et al.  Joint Deep Model-based MR Image and Coil Sensitivity Reconstruction Network (Joint-ICNet) for Fast MRI , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Max Welling,et al.  i-RIM applied to the fastMRI challenge , 2019, ArXiv.

[31]  Pascal Vincent,et al.  fastMRI: An Open Dataset and Benchmarks for Accelerated MRI , 2018, ArXiv.

[32]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[33]  Taeseong Kim,et al.  KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images , 2018, Magnetic resonance in medicine.

[34]  Aaron Defazio,et al.  End-to-End Variational Networks for Accelerated MRI Reconstruction , 2020, MICCAI.

[35]  Edmund Taylor Whittaker.  XVIII.—On the Functions which are represented by the Expansions of the Interpolation-Theory , 1915 .

[36]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[37]  J. Starck,et al.  XPDNet for MRI Reconstruction: an application to the 2020 fastMRI challenge , 2020, 2010.07290.

[38]  Richard Frayne,et al.  Multi-channel MR Reconstruction (MC-MRRec) Challenge - Comparing Accelerated MR Reconstruction Models and Assessing Their Genereralizability to Datasets Collected with Different Coils , 2020, ArXiv.

[39]  Kunihiko Fukushima,et al.  Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition , 1982 .

[40]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[41]  Kawin Setsompop,et al.  Advancing RF pulse design using an open‐competition format: Report from the 2015 ISMRM challenge , 2017, Magnetic resonance in medicine.

[42]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[43]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[44]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[45]  Bennett A Landman,et al.  Challenges in diffusion MRI tractography - Lessons learned from international benchmark competitions. , 2019, Magnetic resonance imaging.

[46]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[47]  Steen Moeller,et al.  Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues , 2020, IEEE Signal Processing Magazine.

[48]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Lars Kasper,et al.  CG‐SENSE revisited: Results from the first ISMRM reproducibility challenge , 2020, Magnetic Resonance in Medicine.

[50]  D. O. Walsh,et al.  Adaptive reconstruction of phased array MR imagery , 2000, Magnetic resonance in medicine.

[51]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[52]  H. Nyquist,et al.  Certain Topics in Telegraph Transmission Theory , 1928, Transactions of the American Institute of Electrical Engineers.

[53]  P. Boesiger,et al.  SENSE: Sensitivity encoding for fast MRI , 1999, Magnetic resonance in medicine.

[54]  Daniel K Sodickson,et al.  Assessment of the generalization of learned image reconstruction and the potential for transfer learning , 2019, Magnetic resonance in medicine.

[55]  Rajat Raina,et al.  Large-scale deep unsupervised learning using graphics processors , 2009, ICML '09.

[56]  Vishal M. Patel,et al.  Pyramid Convolutional RNN for MRI Reconstruction , 2019, ArXiv.

[57]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[58]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[59]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[60]  Michael Elad,et al.  ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA , 2014, Magnetic resonance in medicine.

[61]  L. M. Ward,et al.  Stochastic resonance and sensory information processing: a tutorial and review of application , 2004, Clinical Neurophysiology.

[62]  P. Roemer,et al.  The NMR phased array , 1990, Magnetic resonance in medicine.

[63]  Francesco Renna,et al.  On instabilities of deep learning in image reconstruction and the potential costs of AI , 2019, Proceedings of the National Academy of Sciences.

[64]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.