Synthesize High-Quality Multi-Contrast Magnetic Resonance Imaging From Multi-Echo Acquisition Using Multi-Task Deep Generative Model

Multi-echo saturation recovery sequence can provide redundant information to synthesize multi-contrast magnetic resonance imaging. Traditional synthesis methods, such as GE’s MAGiC platform, employ a model-fitting approach to generate parameter-weighted contrasts. However, models’ over-simplification, as well as imperfections in the acquisition, can lead to undesirable reconstruction artifacts, especially in T2-FLAIR contrast. To improve the image quality, in this study, a multi-task deep learning model is developed to synthesize multi-contrast neuroimaging jointly using both signal relaxation relationships and spatial information. Compared with previous deep learning-based synthesis, the correlation between different destination contrast is utilized to enhance reconstruction quality. To improve model generalizability and evaluate clinical significance, the proposed model was trained and tested on a large multi-center dataset, including healthy subjects and patients with pathology. Results from both quantitative comparison and clinical reader study demonstrate that the multi-task formulation leads to more efficient and accurate contrast synthesis than previous methods.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[3]  Stefan Klein,et al.  SimpleElastix: A User-Friendly, Multi-lingual Library for Medical Image Registration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  J. Duerk,et al.  Magnetic Resonance Fingerprinting , 2013, Nature.

[5]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Peter R Seevinck,et al.  Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy , 2018, Physics in medicine and biology.

[7]  A. McMillan,et al.  Deep learning Mr imaging–based attenuation correction for PeT/Mr imaging 1 , 2017 .

[8]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[9]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[10]  Dong-Hyun Kim,et al.  Data‐driven synthetic MRI FLAIR artifact correction via deep neural network , 2019, Journal of magnetic resonance imaging : JMRI.

[11]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[12]  Daniel S. Weller,et al.  Image quality affects deep learning reconstruction of MRI , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[13]  Shaohua Kevin Zhou,et al.  Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network , 2015, MICCAI.

[14]  Jinhua Yu,et al.  A Universal Intensity Standardization Method Based on a Many-to-One Weak-Paired Cycle Generative Adversarial Network for Magnetic Resonance Images , 2019, IEEE Transactions on Medical Imaging.

[15]  Guang Yang,et al.  Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction , 2018, MICCAI.

[16]  Justin P. Haldar,et al.  The Fourier radial error spectrum plot: A more nuanced quantitative evaluation of image reconstruction quality , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[17]  J N Lee,et al.  Cerebral magnetic resonance image synthesis. , 1985, AJNR. American journal of neuroradiology.

[18]  P. Lundberg,et al.  Novel method for rapid, simultaneous T1, T*2, and proton density quantification , 2007, Magnetic resonance in medicine.

[19]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[20]  J. Pauly,et al.  Deep learning enables reduced gadolinium dose for contrast‐enhanced brain MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[21]  Yong Chen,et al.  Deep-Learning-Based Multi-Modal Fusion for Fast MR Reconstruction , 2019, IEEE Transactions on Biomedical Engineering.

[22]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[23]  Sina Honari,et al.  Distribution Matching Losses Can Hallucinate Features in Medical Image Translation , 2018, MICCAI.

[24]  Ghassan Hamarneh,et al.  Missing MRI Pulse Sequence Synthesis Using Multi-Modal Generative Adversarial Network , 2019, IEEE Transactions on Medical Imaging.

[25]  P. Lundberg,et al.  Rapid magnetic resonance quantification on the brain: Optimization for clinical usage , 2008, Magnetic resonance in medicine.

[26]  Yinghuan Shi,et al.  Ea-GANs: Edge-Aware Generative Adversarial Networks for Cross-Modality MR Image Synthesis , 2019, IEEE Transactions on Medical Imaging.

[27]  Paul Babyn,et al.  Generative Adversarial Network in Medical Imaging: A Review , 2018, Medical Image Anal..

[28]  Aykut Erdem,et al.  Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks , 2018, IEEE Transactions on Medical Imaging.

[29]  Anne G. Osborn,et al.  Osborn's brain : imaging, pathology, and anatomy , 2013 .

[30]  T. Naidich,et al.  Synthetic MRI for Clinical Neuroimaging: Results of the Magnetic Resonance Image Compilation (MAGiC) Prospective, Multicenter, Multireader Trial , 2017, American Journal of Neuroradiology.

[31]  Yin Fei,et al.  Medical Image Fusion Based on Feature Extraction and Sparse Representation , 2017, Int. J. Biomed. Imaging.

[32]  J N Lee,et al.  Automated MR image synthesis: feasibility studies. , 1984, Radiology.

[33]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[34]  Jan Kautz,et al.  Multimodal Unsupervised Image-to-Image Translation , 2018, ECCV.

[35]  Dinggang Shen,et al.  Synthesizing Missing PET from MRI with Cycle-consistent Generative Adversarial Networks for Alzheimer's Disease Diagnosis , 2018, MICCAI.

[36]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[37]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[38]  Sana Boudabbous,et al.  Feasibility of synthetic MRI in knee imaging in routine practice , 2018, Acta radiologica open.

[39]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Bin Yang,et al.  MedGAN: Medical Image Translation using GANs , 2018, Comput. Medical Imaging Graph..

[41]  Andrew P. Leynes,et al.  Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI Attenuation Correction Using Deep Convolutional Neural Networks with Multiparametric MRI , 2017, The Journal of Nuclear Medicine.

[42]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Jong Chul Ye,et al.  Deep residual learning for compressed sensing MRI , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[44]  Jong Chul Ye,et al.  CollaGAN: Collaborative GAN for Missing Image Data Imputation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Junshen Xu,et al.  200x Low-dose PET Reconstruction using Deep Learning , 2017, ArXiv.

[46]  O Smedby,et al.  Synthetic Mri of the Brain in a Clinical Setting , 2012, Acta radiologica.

[47]  Lei Wang,et al.  3D cGAN based cross-modality MR image synthesis for brain tumor segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[48]  P Aspelin,et al.  Clinical Feasibility of Synthetic MRI in Multiple Sclerosis: A Diagnostic and Volumetric Validation Study , 2016, American Journal of Neuroradiology.

[49]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[50]  John M Pauly,et al.  Ultra-Low-Dose 18F-Florbetaben Amyloid PET Imaging Using Deep Learning with Multi-Contrast MRI Inputs. , 2019, Radiology.

[51]  Tolga Çukur,et al.  Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI , 2018, ArXiv.

[52]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[53]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[54]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[55]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .