Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k-space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-off. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k-space reference data for supervised training, which is not suitable for tMRA. This is because high spatio-temporal resolution ground-truth images are not available for tMRA. To address this problem, here we propose a novel unsupervised deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler and improves the performance. Reconstruction results using in vivo tMRA data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

[1]  HyunWook Park,et al.  A parallel MR imaging method using multilayer perceptron , 2017, Medical physics.

[2]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jong Chul Ye,et al.  CycleGAN With a Blur Kernel for Deconvolution Microscopy: Optimal Transport Geometry , 2019, IEEE Transactions on Computational Imaging.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[6]  Jong Chul Ye,et al.  Reference‐free single‐pass EPI Nyquist ghost correction using annihilating filter‐based low rank Hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[7]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[8]  Jaejun Yoo,et al.  Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks , 2018, IEEE Transactions on Biomedical Engineering.

[9]  L. Turnbull Dynamic contrast‐enhanced MRI in the diagnosis and management of breast cancer , 2009, NMR in biomedicine.

[10]  Mathews Jacob,et al.  Accelerated Dynamic MRI Exploiting Sparsity and Low-Rank Structure: k-t SLR , 2011, IEEE Transactions on Medical Imaging.

[11]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[12]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[13]  J. C. Ye,et al.  Acceleration of MR parameter mapping using annihilating filter‐based low rank hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[14]  G. Laub,et al.  syngo TWIST for Dynamic Time-Resolved MR Angiography , 2006 .

[15]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[16]  Jong Chul Ye,et al.  Optimal Transport Driven CycleGAN for Unsupervised Learning in Inverse Problems , 2019, SIAM J. Imaging Sci..

[17]  Leslie Ying,et al.  Accelerating magnetic resonance imaging via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[18]  Jong Chul Ye,et al.  k‐t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI , 2009, Magnetic resonance in medicine.

[19]  G. Laub,et al.  3D Time-Resolved MR Angiography (MRA) of the Carotid Arteries with Time-Resolved Imaging with Stochastic Trajectories: Comparison with 3D Contrast-Enhanced Bolus-Chase MRA and 3D Time-Of-Flight MRA , 2008, American Journal of Neuroradiology.

[20]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[21]  Jong Chul Ye,et al.  Deep learning with domain adaptation for accelerated projection‐reconstruction MR , 2018, Magnetic resonance in medicine.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[24]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[25]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[26]  Jong Chul Ye,et al.  A General Framework for Compressed Sensing and Parallel MRI Using Annihilating Filter Based Low-Rank Hankel Matrix , 2015, IEEE Transactions on Computational Imaging.

[27]  Jong Chul Ye,et al.  ${k}$ -Space Deep Learning for Accelerated MRI , 2020, IEEE Transactions on Medical Imaging.

[28]  Thomas E Yankeelov,et al.  Dynamic Contrast Enhanced Magnetic Resonance Imaging in Oncology: Theory, Data Acquisition, Analysis, and Examples. , 2007, Current medical imaging reviews.

[29]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jong Chul Ye,et al.  Optimal Transport, CycleGAN, and Penalized LS for Unsupervised Learning in Inverse Problems , 2019, ArXiv.

[31]  C. Villani Optimal Transport: Old and New , 2008 .

[32]  Yoon-Chul Kim Advanced Methods in Dynamic Contrast Enhanced Arterial Phase Imaging of the Liver , 2019, Investigative Magnetic Resonance Imaging.

[33]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[34]  Jong Chul Ye,et al.  True Temporal Resolution TWIST Imaging using Annihilating Filter-based Low-rank wrap around Hankel Matrix , 2017 .

[35]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.