MR to CT synthesis with multicenter data in the pelvic area using a conditional generative adversarial network

The establishment of an MRI-only workflow in radiotherapy depends on the ability to generate an accurate synthetic-CT (sCT) for dose calculation. Previously proposed methods have used a Generative Adversarial Network (GAN) for fast sCT generation in order to simplify the clinical workflow and reduces uncertainties. In the current paper we use a conditional Generative Adversarial Network (cGAN) framework called pix2pixHD to create a robust model prone to multicenter data. This study included T2-weighted MR and CT images of 19 patients in treatment position from 3 different sites. The cGAN was trained on 2D transverse slices of 11 patients from 2 different sites. Once trained, the network was used to generate sCT images of 8 patients coming from a third site. The Mean Absolute Errors (MAE) for each patient were evaluated between real and synthetic CTs. A radiotherapy plan was optimized on the sCT series and re-calculated on CTs to assess the dose distribution in terms of voxel-wise dose difference and Dose Volume Histograms (DVH) analysis. It takes on average of 7.5 s to generate a complete sCT (88 slices) for a patient on our GPU. The average MAE in HU between the sCT and actual patient CT (within the body contour) is 48.5 ± 6 HU with our method. The maximum dose difference to the target is 1.3%. This study demonstrates that an sCT can be generated in a multicentric context, with fewer pre-processing steps while being fast and accurate.

[1]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[2]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[3]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[4]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[6]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[8]  Jelmer M. Wolterink,et al.  MR-Only Brain Radiation Therapy: Dosimetric Evaluation of Synthetic CTs Generated by a Dilated Convolutional Neural Network. , 2018, International journal of radiation oncology, biology, physics.

[9]  Tufve Nyholm,et al.  MR and CT data with multiobserver delineations of organs in the pelvic area—Part of the Gold Atlas project , 2018, Medical physics.

[10]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Peter R Seevinck,et al.  Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy , 2018, Physics in medicine and biology.

[12]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.