Reducing Uncertainty in Undersampled MRI Reconstruction With Active Acquisition

The goal of MRI reconstruction is to restore a high fidelity image from partially observed measurements. This partial view naturally induces reconstruction uncertainty that can only be reduced by acquiring additional measurements. In this paper, we present a novel method for MRI reconstruction that, at inference time, dynamically selects the measurements to take and iteratively refines the prediction in order to best reduce the reconstruction error and, thus, its uncertainty. We validate our method on a large scale knee MRI dataset, as well as on ImageNet. Results show that (1) our system successfully outperforms active acquisition baselines; (2) our uncertainty estimates correlate with error maps; and (3) our ResNet-based architecture surpasses standard pixel-to-pixel models in the task of MRI reconstruction. The proposed method not only shows high-quality reconstructions but also paves the road towards more applicable solutions for accelerating MRI.

[1]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[2]  A. Kiureghian,et al.  Aleatory or epistemic? Does it matter? , 2009 .

[3]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[4]  Graham W. Taylor,et al.  Leveraging Uncertainty Estimates for Predicting Segmentation Quality , 2018, ArXiv.

[5]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[6]  Ce Liu,et al.  Deep Convolutional Neural Network for Image Deconvolution , 2014, NIPS.

[7]  Lei Zhao,et al.  Real-Time Adaptive Functional MRI , 1999, NeuroImage.

[8]  Volkan Cevher,et al.  Learning-Based Compressive MRI , 2018, IEEE Transactions on Medical Imaging.

[9]  Xiaoou Tang,et al.  Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net , 2018, ECCV.

[10]  Yoram Bresler,et al.  Adaptive sampling design for compressed sensing MRI , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Yoshua Bengio,et al.  On the Iterative Refinement of Densely Connected Representation Levels for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Wei Xu,et al.  Multi-channel Generative Adversarial Network for Parallel Magnetic Resonance Image Reconstruction in K-space , 2018, MICCAI.

[15]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Jasper Snoek,et al.  Spectral Representations for Convolutional Neural Networks , 2015, NIPS.

[19]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[20]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[21]  Evan Levine,et al.  On-the-Fly Adaptive ${k}$ -Space Sampling for Linear MRI Reconstruction Using Moment-Based Spectral Analysis , 2017, IEEE Transactions on Medical Imaging.

[22]  Mark Tygert,et al.  Compressed sensing with a jackknife and a bootstrap , 2018, J. Data Sci. Stat. Vis..

[23]  D Moratal,et al.  k-Space tutorial: an MRI educational tool for a better understanding of k-space , 2008, Biomedical imaging and intervention journal.

[24]  Bernhard Schölkopf,et al.  Optimization of k‐space trajectories for compressed sensing by Bayesian experimental design , 2010, Magnetic resonance in medicine.

[25]  Hannes Stuke,et al.  Learning uncertainty in regression tasks by deep neural networks , 2017, ArXiv.

[26]  F. Jolesz,et al.  Dynamically adaptive MRI with encoding by singular value decomposition , 1994, Magnetic resonance in medicine.

[27]  Jin Keun Seo,et al.  Deep learning for undersampled MRI reconstruction , 2017, Physics in medicine and biology.

[28]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[29]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[30]  Siegfried Wahl,et al.  Leveraging uncertainty information from deep neural networks for disease detection , 2016, Scientific Reports.

[31]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[32]  Yudong Zhang,et al.  Energy Preserved Sampling for Compressed Sensing MRI , 2014, Comput. Math. Methods Medicine.

[33]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Guang Yang,et al.  DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction , 2018, IEEE Transactions on Medical Imaging.

[35]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[36]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[37]  Pascal Vincent,et al.  fastMRI: An Open Dataset and Benchmarks for Accelerated MRI , 2018, ArXiv.

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[40]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[41]  Jong Chul Ye,et al.  ${k}$ -Space Deep Learning for Accelerated MRI , 2020, IEEE Transactions on Medical Imaging.

[42]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[43]  Guang Yang,et al.  Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction , 2018, MICCAI.

[44]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[45]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[46]  Patrick Putzky,et al.  Recurrent inference machines for accelerated MRI reconstruction. , 2018 .

[47]  Leon Axel,et al.  Combination of Compressed Sensing and Parallel Imaging for Highly-Accelerated 3 D First-Pass Cardiac Perfusion MRI , 2009 .

[48]  L P Panych,et al.  Applicability and efficiency of near‐optimal spatial encoding for dynamically adaptive MRI , 1998, Magnetic resonance in medicine.

[49]  L P Panych,et al.  Implementation of a fast gradient‐echo SVD encoding technique for dynamic imaging , 1996, Magnetic resonance in medicine.

[50]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[51]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[52]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.