Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI

Multi-contrast MRI acquisitions of an anatomy enrich the magnitude of information available for diagnosis. Yet, excessive scan times associated with additional contrasts may be a limiting factor. Two mainstream approaches for enhanced scan efficiency are reconstruction of undersampled acquisitions and synthesis of missing acquisitions. In reconstruction, performance decreases towards higher acceleration factors with diminished sampling density particularly at high-spatial-frequencies. In synthesis, the absence of data samples from the target contrast can lead to artefactual sensitivity or insensitivity to image features. Here we propose a new approach for synergistic reconstruction-synthesis of multi-contrast MRI based on conditional generative adversarial networks. The proposed method preserves high-frequency details of the target contrast by relying on the shared high-frequency information available from the source contrast, and prevents feature leakage or loss by relying on the undersampled acquisitions of the target contrast. Demonstrations on brain MRI datasets from healthy subjects and patients indicate the superior performance of the proposed method compared to previous state-of-the-art. The proposed method can help improve the quality and scan efficiency of multi-contrast MRI exams.

[1]  Snehashis Roy,et al.  Random forest regression for magnetic resonance image synthesis , 2017, Medical Image Anal..

[2]  P. Boesiger,et al.  SENSE: Sensitivity encoding for fast MRI , 1999, Magnetic resonance in medicine.

[3]  Hervé Delingette,et al.  Extended Modality Propagation: Image Synthesis of Pathological Cases , 2016, IEEE Transactions on Medical Imaging.

[4]  Marta M. Betcke,et al.  Multicontrast MRI Reconstruction with Structure-Guided Total Variation , 2015, SIAM J. Imaging Sci..

[5]  Morteza Mardani,et al.  Recurrent Generative Adversarial Networks for Proximal Learning and Automated Compressive Image Recovery , 2017, ArXiv.

[6]  Snehashis Roy,et al.  Atlas Based Intensity Transformation of Brain MR Images , 2013, MBIA.

[7]  Jong Chul Ye,et al.  Deep Residual Learning for Compressed Sensing CT Reconstruction via Persistent Homology Analysis , 2016, ArXiv.

[8]  Jian Sun,et al.  ADMM-Net: A Deep Learning Approach for Compressive Sensing MRI , 2017, ArXiv.

[9]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[10]  Ling Shao,et al.  Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning , 2018, IEEE Transactions on Medical Imaging.

[11]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[12]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction , 2017, IPMI.

[13]  Snehashis Roy,et al.  MR contrast synthesis for lesion segmentation , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[14]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Nannan Li,et al.  MRI Image-to-Image Translation for Cross-Modality Image Registration and Segmentation , 2018, ArXiv.

[16]  Vivek K Goyal,et al.  Multi‐contrast reconstruction with Bayesian compressed sensing , 2011, Magnetic resonance in medicine.

[17]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[18]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[19]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[20]  Snehashis Roy,et al.  MR image synthesis by contrast learning on neighborhood ensembles , 2015, Medical Image Anal..

[21]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[22]  J. K. Smith,et al.  Vessel tortuosity and brain tumor malignancy: a blinded study. , 2005, Academic radiology.

[23]  Shaohua Kevin Zhou,et al.  Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network , 2015, MICCAI.

[24]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[25]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[26]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[27]  Snehashis Roy,et al.  A Compressed Sensing Approach for MR Tissue Contrast Synthesis , 2011, IPMI.

[28]  Sotirios A. Tsaftaris,et al.  Robust Multi-modal MR Image Synthesis , 2017, MICCAI.

[29]  Snehashis Roy,et al.  Magnetic resonance image synthesis through patch regression , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[30]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[31]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[32]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[33]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[34]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction with Cyclic Loss in Generative Adversarial Networks , 2017, ArXiv.

[35]  Shaohua Kevin Zhou,et al.  Unsupervised Cross-Modal Synthesis of Subject-Specific Scans , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Aykut Erdem,et al.  Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks , 2018, IEEE Transactions on Medical Imaging.

[37]  Ling Shao,et al.  Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[39]  Sotirios A. Tsaftaris,et al.  Whole Image Synthesis Using a Deep Encoder-Decoder Network , 2016, SASHIMI@MICCAI.

[40]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[41]  Snehashis Roy,et al.  Magnetic Resonance Image Example-Based Contrast Synthesis , 2013, IEEE Transactions on Medical Imaging.

[42]  Guang Yang,et al.  Deep De-Aliasing for Fast Compressive Sensing MRI , 2017, ArXiv.

[43]  M. Lustig,et al.  SPIRiT: Iterative self‐consistent parallel imaging reconstruction from arbitrary k‐space , 2010, Magnetic resonance in medicine.

[44]  Ben Glocker,et al.  Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization , 2013, MICCAI.