Generative Adversarial Training for MRA Image Synthesis Using Multi-Contrast MRI

Magnetic Resonance Angiography (MRA) has become an essential MR contrast for imaging and evaluation of vascular anatomy and related diseases. MRA acquisitions are typically ordered for vascular interventions, whereas in typical scenarios, MRA sequences can be absent in the patient scans. This motivates the need for a technique that generates inexistent MRA from existing MR multi-contrast, which could be a valuable tool in retrospective subject evaluations and imaging studies. In this paper, we present a generative adversarial network (GAN) based technique to generate MRA from T1-weighted and T2-weighted MRI images, for the first time to our knowledge. To better model the representation of vessels which the MRA inherently highlights, we design a loss term dedicated to a faithful reproduction of vascularities. To that end, we incorporate steerable filter responses of the generated and reference images inside a Huber function loss term. Extending the well- established generator-discriminator architecture based on the recent PatchGAN model with the addition of steerable filter loss, the proposed steerable GAN (sGAN) method is evaluated on the large public database IXI. Experimental results show that the sGAN outperforms the baseline GAN method in terms of an overlap score with similar PSNR values, while it leads to improved visual perceptual quality.

[1]  Stanley Durrleman,et al.  FLAIR MR Image Synthesis By Using 3D Fully Convolutional Networks for Multiple Sclerosis , 2018 .

[2]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[3]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  D G Nishimura,et al.  Time‐of‐flight MR angiography , 1990, Magnetic resonance in medicine.

[5]  Thomas M Grist,et al.  Magnetic resonance angiography: current status and future directions , 2011, Journal of cardiovascular magnetic resonance : official journal of the Society for Cardiovascular Magnetic Resonance.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  J M Wardlaw,et al.  Intracranial aneurysms: CT angiography and MR angiography for detection prospective blinded comparison in a large patient cohort. , 2001, Radiology.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[11]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[12]  Bo Zhang,et al.  A comparison of contrast-free MRA at 3.0T in cases of intracranial aneurysms with or without subarachnoid hemorrhage. , 2017, Clinical imaging.

[13]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Marcello Cadioli,et al.  Comparison of 3D TOF-MRA and 3D CE-MRA at 3T for imaging of intracranial aneurysms. , 2013, European journal of radiology.

[15]  R Gilberto González,et al.  Magnetic resonance angiography: physical principles and applications. , 2016, Handbook of clinical neurology.

[16]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[17]  Guang Yang,et al.  DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction , 2018, IEEE Transactions on Medical Imaging.

[18]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[19]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[20]  Ciprian Catana,et al.  Toward Implementing an MRI-Based PET Attenuation-Correction Method for Neurologic Studies on the MR-PET Brain Prototype , 2010, The Journal of Nuclear Medicine.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[23]  Guido Gerig,et al.  Multiscale detection of curvilinear structures in 2-D and 3-D image data , 1995, Proceedings of IEEE International Conference on Computer Vision.

[24]  Aykut Erdem,et al.  Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks , 2018, IEEE Transactions on Medical Imaging.

[25]  Joachim E. Wildberger,et al.  Cost-effectiveness of CTA, MRA and DSA in patients with non-traumatic subarachnoid haemorrhage , 2013, Insights into Imaging.

[26]  H. Zaidi,et al.  Magnetic resonance imaging-guided attenuation and scatter corrections in three-dimensional brain positron emission tomography. , 2003, Medical physics.

[27]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[28]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[29]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[30]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Bostjan Likar,et al.  Enhancement of Vascular Structures in 3D and 2D Angiographic Images , 2016, IEEE Transactions on Medical Imaging.

[32]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[33]  R. Wilson,et al.  Anisotropic Nonstationary Image Estimation and Its Applications: Part I - Restoration of Noisy Images , 1983, IEEE Transactions on Communications.

[34]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[35]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[36]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[37]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .