Unpaired Training of Deep Learning tMRA for Flexible Spatio-Temporal Resolution

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the ${\textit k}$ -space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-offs. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled ${\textit k}$ -space reference data for supervised training, which is not suitable for tMRA due to the lack of high spatio-temporal resolution ground-truth images. To address this problem, here we propose a novel unpaired training scheme for deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler but still improves the performance. Reconstruction results using in vivo tMRA and simulation data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

[1]  L. Turnbull Dynamic contrast‐enhanced MRI in the diagnosis and management of breast cancer , 2009, NMR in biomedicine.

[2]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[3]  Jong Chul Ye,et al.  Deep learning with domain adaptation for accelerated projection‐reconstruction MR , 2018, Magnetic resonance in medicine.

[4]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[5]  Jong Chul Ye,et al.  True Temporal Resolution TWIST Imaging using Annihilating Filter-based Low-rank wrap around Hankel Matrix , 2017 .

[6]  Dong Liang,et al.  DeepcomplexMRI: Exploiting deep residual network for fast parallel MR imaging with complex convolution. , 2020, Magnetic resonance imaging.

[7]  J. C. Ye,et al.  Acceleration of MR parameter mapping using annihilating filter‐based low rank hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[8]  Mathews Jacob,et al.  Accelerated Dynamic MRI Exploiting Sparsity and Low-Rank Structure: k-t SLR , 2011, IEEE Transactions on Medical Imaging.

[9]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[11]  Jong Chul Ye,et al.  CycleGAN With a Blur Kernel for Deconvolution Microscopy: Optimal Transport Geometry , 2019, IEEE Transactions on Computational Imaging.

[12]  Leslie Ying,et al.  Accelerating magnetic resonance imaging via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[13]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[14]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Steen Moeller,et al.  Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data , 2019, Magnetic resonance in medicine.

[16]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[17]  HyunWook Park,et al.  A parallel MR imaging method using multilayer perceptron , 2017, Medical physics.

[18]  Jaejun Yoo,et al.  Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks , 2018, IEEE Transactions on Biomedical Engineering.

[19]  Jong Chul Ye,et al.  ${k}$ -Space Deep Learning for Accelerated MRI , 2020, IEEE Transactions on Medical Imaging.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[22]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[23]  Jong Chul Ye,et al.  k‐t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI , 2009, Magnetic resonance in medicine.

[24]  Jong Chul Ye,et al.  Reference‐free single‐pass EPI Nyquist ghost correction using annihilating filter‐based low rank Hankel matrix (ALOHA) , 2016, Magnetic resonance in medicine.

[25]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[26]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Yoon-Chul Kim Advanced Methods in Dynamic Contrast Enhanced Arterial Phase Imaging of the Liver , 2019, Investigative Magnetic Resonance Imaging.

[29]  Thomas E Yankeelov,et al.  Dynamic Contrast Enhanced Magnetic Resonance Imaging in Oncology: Theory, Data Acquisition, Analysis, and Examples. , 2007, Current medical imaging reviews.

[30]  G. Laub,et al.  syngo TWIST for Dynamic Time-Resolved MR Angiography , 2006 .

[31]  Jong Chul Ye,et al.  A General Framework for Compressed Sensing and Parallel MRI Using Annihilating Filter Based Low-Rank Hankel Matrix , 2015, IEEE Transactions on Computational Imaging.

[32]  G. Laub,et al.  3D Time-Resolved MR Angiography (MRA) of the Carotid Arteries with Time-Resolved Imaging with Stochastic Trajectories: Comparison with 3D Contrast-Enhanced Bolus-Chase MRA and 3D Time-Of-Flight MRA , 2008, American Journal of Neuroradiology.

[33]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[34]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[35]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.