Medical Image Synthesis with Context-Aware Generative Adversarial Networks
Abstract:Computed tomography (CT) is critical for various clinical applications, e.g., radiation treatment planning and also PET attenuation correction in MRI/PET scanner. However, CT exposes radiation during acquisition, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve radiations. Therefore, recently researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiation planning. In this paper, we propose a data-driven approach to address this challenging problem. Specifically, we train a fully convolutional network (FCN) to generate CT given the MR image. To better model the nonlinear mapping from MRI to CT and produce more realistic images, we propose to use the adversarial training strategy to train the FCN. Moreover, we propose an image-gradient-difference based loss function to alleviate the blurriness of the generated CT. We further apply Auto-Context Model (ACM) to implement a context-aware generative adversarial network. Experimental results show that our method is accurate and robust for predicting CT images from MR images, and also outperforms three state-of-the-art methods under comparison.
摘要：计算机断层扫描(CT)在各种临床应用中都是至关重要的，例如在MRI/PET扫描仪中进行放射治疗计划和PET衰减校正。然而，CT在采集过程中会暴露出辐射，这可能会给患者带来副作用。与CT相比，磁共振成像(MRI)要安全得多，而且不涉及辐射。因此，近年来，在进行放射治疗计划的情况下，研究人员极大地推动了从同一对象的相应MR图像来估计CT图像。在本文中，我们提出了一种数据驱动的方法来解决这个具有挑战性的问题。具体地说，我们训练一个完全卷积网络(FCN)来生成给定的MR图像的CT。为了更好地模拟从MRI到CT的非线性映射，并生成更逼真的图像，我们提出了使用对抗性训练策略来训练FCN。此外，我们还提出了一种基于图像梯度差的损失函数来改善生成的CT图像的模糊性。在此基础上，应用自动上下文模型(ACM)实现了一个上下文感知的产生式对抗性网络。实验结果表明，该方法对于从MR图像中预测CT图像是准确和稳健的，并且在性能上也优于三种最先进的方法。
暂无分享，去 创建一个
[1] Yaozong Gao,et al. Fully convolutional networks for multi-modality isointense infant brain image segmentation , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).
[2] Amod Jog,et al. Improving magnetic resonance resolution with supervised learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).
[3] Yoshua Bengio,et al. Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.
[4] P. Cochat,et al. Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.
[5] D. Brenner,et al. Computed tomography--an increasing source of radiation exposure. , 2007, The New England journal of medicine.
[6] Zhuowen Tu,et al. Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.
[7] Yann LeCun,et al. Deep multi-scale video prediction beyond mean square error , 2015, ICLR.
[8] Yoshua Bengio,et al. Gradient-based learning applied to document recognition , 1998, Proc. IEEE.
[9] Dinggang Shen,et al. Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.
[10] Yoshua Bengio,et al. Generative Adversarial Nets , 2014, NIPS.
[11] H. Zaidi,et al. Magnetic resonance imaging-guided attenuation and scatter corrections in three-dimensional brain positron emission tomography. , 2003, Medical physics.
[12] Yaozong Gao,et al. LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2014, NeuroImage.
[13] Geoffrey E. Hinton,et al. Deep Learning , 2015, Nature.
[14] Christian Ledig,et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[15] Yaozong Gao,et al. Estimating CT Image From MRI Data Using Structured Random Forest and Auto-Context Model , 2016, IEEE Transactions on Medical Imaging.
[16] Ming Yang,et al. 3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.
[17] Lorenzo Torresani,et al. Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).
[18] Ciprian Catana,et al. Toward Implementing an MRI-Based PET Attenuation-Correction Method for Neurologic Studies on the MR-PET Brain Prototype , 2010, The Journal of Nuclear Medicine.
[19] Soumith Chintala,et al. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.
[20] C. Kuhl,et al. MRI-Based Attenuation Correction for Hybrid PET/MRI Systems: A 4-Class Tissue Segmentation Technique Using a Combined Ultrashort-Echo-Time/Dixon MRI Sequence , 2012, The Journal of Nuclear Medicine.
[21] Heung-Il Suk,et al. Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.
[22] Paul Kinahan,et al. Attenuation correction for a combined 3D PET/CT scanner. , 1998, Medical physics.
[23] Tom Vercauteren,et al. Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.
[24] Xiaoou Tang,et al. Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.
[25] Xiaoou Tang,et al. Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.