Active MR k-space Sampling with Reinforcement Learning

Deep learning approaches have recently shown great promise in accelerating magnetic resonance image (MRI) acquisition. The majority of existing work have focused on designing better reconstruction models given a pre-determined acquisition trajectory, ignoring the question of trajectory optimization. In this paper, we focus on learning acquisition trajectories given a fixed image reconstruction model. We formulate the problem as a sequential decision process and propose the use of reinforcement learning to solve it. Experiments on a large scale public MRI dataset of knees show that our proposed models significantly outperform the state-of-the-art in active MRI acquisition, over a large range of acceleration factors.

[1]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[2]  Pascal Vincent,et al.  fastMRI: An Open Dataset and Benchmarks for Accelerated MRI , 2018, ArXiv.

[3]  Jiming Liu,et al.  Reinforcement Learning in Healthcare: A Survey , 2019, ACM Comput. Surv..

[4]  Bernhard Schölkopf,et al.  Optimization of k‐space trajectories for compressed sensing by Bayesian experimental design , 2010, Magnetic resonance in medicine.

[5]  Jan-Jakob Sonke,et al.  Recurrent inference machines for reconstructing heterogeneous MRI data , 2019, Medical Image Anal..

[6]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[7]  Yoram Bresler,et al.  Adaptive sampling design for compressed sensing MRI , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Michael Unser,et al.  Self-Supervised Deep Active Accelerated MRI , 2019, ArXiv.

[9]  Wei Xu,et al.  Multi-channel Generative Adversarial Network for Parallel Magnetic Resonance Image Reconstruction in K-space , 2018, MICCAI.

[10]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[11]  Dong Liang,et al.  DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution. , 2020, Magnetic resonance imaging.

[12]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[13]  Society of magnetic resonance in medicine , 1990 .

[14]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[15]  Leslie Ying,et al.  Accelerating magnetic resonance imaging via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[16]  Pascal Vincent,et al.  Reducing Uncertainty in Undersampled MRI Reconstruction With Active Acquisition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[18]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[19]  Jonathan I. Tamir,et al.  Variable-Density Single-Shot Fast Spin-Echo MRI with Deep Learning Reconstruction by Using Variational Networks. , 2018, Radiology.

[20]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  Yudong Zhang,et al.  Energy Preserved Sampling for Compressed Sensing MRI , 2014, Comput. Math. Methods Medicine.

[23]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[24]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[25]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[26]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[27]  L P Panych,et al.  Applicability and efficiency of near‐optimal spatial encoding for dynamically adaptive MRI , 1998, Magnetic resonance in medicine.

[28]  L P Panych,et al.  Implementation of a fast gradient‐echo SVD encoding technique for dynamic imaging , 1996, Magnetic resonance in medicine.

[29]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[30]  et al.,et al.  Jupyter Notebooks - a publishing format for reproducible computational workflows , 2016, ELPUB.

[31]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[32]  Lei Zhao,et al.  Real-Time Adaptive Functional MRI , 1999, NeuroImage.

[33]  Volkan Cevher,et al.  Learning-Based Compressive MRI , 2018, IEEE Transactions on Medical Imaging.

[34]  F. Jolesz,et al.  Dynamically adaptive MRI with encoding by singular value decomposition , 1994, Magnetic resonance in medicine.

[35]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[36]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.