Patch-Based Generative Adversarial Neural Network Models for Head and Neck MR-Only Planning.

PURPOSE To evaluate pix2pix and CycleGAN and to assess the effects of multiple combination strategies on accuracy for patch-based synthetic CT (sCT) generation for MR-only treatment planning in head and neck (HN) cancer patients. MATERIALS AND METHODS Twenty-three deformably registered pairs of CT and mDixon FFE MR datasets from HN cancer patients treated at our institution were retrospectively analyzed to evaluate patch-based sCT accuracy via the pix2pix and CycleGAN models. To test effects of overlapping sCT patches on estimations, we 1) trained the models for three orthogonal views to observe the effects of spatial context, 2) we increased effective set size by using per-epoch data augmentation, and 3) we evaluated the performance of three different approaches for combining overlapping Hounsfield Unit (HU) estimations for varied patch overlap parameters. Twelve of twenty-three cases corresponded to a curated dataset previously used for atlas-based sCT generation and were used for training with leave-two-out cross-validation. Eight cases were used for independent testing and included previously unseen image features such as fused vertebrae, a small protruding bone, and tumors large enough to deform normal body contours. We analyzed the impact of MR image preprocessing including histogram standardization and intensity clipping on sCT generation accuracy. Effects of mDixon contrast (in-phase vs water) differences were tested with three additional cases. The sCT generation accuracy was evaluated using Mean Absolute Error (MAE) and Mean Error (ME) in HU between the plan CT and sCT images. Dosimetric accuracy was evaluated for all clinically relevant structures in the independent testing set and digitally reconstructed radiographs (DRRs) were evaluated with respect to the plan CT images. RESULTS The cross-validated MAEs for the whole-HN region using pix2pix and CycleGAN were 66.9±7.3 HU vs. 82.3±6.4 HU, respectively. On the independent testing set with additional artifacts and previously unseen image features, whole-HN region MAEs were 94.0±10.6 HU and 102.9±14.7 HU for pix2pix and CycleGAN, respectively. For patients with different tissue contrast (water mDixon MR images), the MAEs increased to 122.1±6.3 and 132.8±5.5 HU for pix2pix and CycleGAN, respectively. Our results suggest that combining overlapping sCT estimations at each voxel reduced both MAE and ME compared to single-view non-overlapping patch results. Absolute percent mean/max dose errors were 2% or less for the PTV and all clinically relevant structures in our independent testing set, including structures with image artifacts. Quantitative DRR comparison between planning CTs and sCTs showed agreement of bony region positions to less than 1 mm. CONCLUSIONS The dosimetric and MAE based accuracy, along with the similarity between DRRs from sCTs, indicate that pix2pix and CycleGAN are promising methods for MR-only treatment planning for HN cancer. Our methods investigated for overlapping patch-based HU estimations also indicate that combining transformation estimations of overlapping patches is a potential method to reduce generation errors while also providing a tool to potentially estimate the MR to CT aleatoric model transformation uncertainty. However, because of small patient sample sizes, further studies are required.

[1]  Ming Dong,et al.  Generating synthetic CTs from magnetic resonance images using generative adversarial networks , 2018, Medical physics.

[2]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[3]  Guoyan Zheng,et al.  Novel adversarial semantic structure deep learning for MRI-guided attenuation correction in brain PET/MRI , 2019, European Journal of Nuclear Medicine and Molecular Imaging.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Chunming Li,et al.  A Level Set Method for Image Segmentation in the Presence of Intensity Inhomogeneities With Application to MRI , 2011, IEEE Transactions on Image Processing.

[6]  L. Ting,et al.  Impact of magnetic resonance imaging versus CT on nasopharyngeal carcinoma: primary tumor target delineation for radiotherapy , 2004, Head & neck.

[7]  Jonathan J Wyatt,et al.  Systematic Review of Synthetic Computed Tomography Generation Methodologies for Use in Magnetic Resonance Imaging-Only Radiation Therapy. , 2018, International journal of radiation oncology, biology, physics.

[8]  T. Nyholm,et al.  A review of substitute CT generation for MRI-only radiation therapy , 2017, Radiation oncology.

[9]  Harini Veeraraghavan,et al.  Multiatlas approach with local registration goodness weighting for MRI‐based electron density mapping of head and neck anatomy† , 2017, Medical physics.

[10]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[11]  Nikolaus M. Szeverenyi,et al.  Ultrashort echo time (UTE) magnetic resonance imaging of the short T2 components in white matter of the brain using a clinical 3T scanner , 2014, NeuroImage.

[12]  Di Yan,et al.  Technical Note: U‐net‐generated synthetic CT images for magnetic resonance imaging‐only prostate intensity‐modulated radiation therapy treatment planning , 2018, Medical physics.

[13]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Marcel van Herk,et al.  Decreased 3D observer variation with matched CT-MRI, for target delineation in Nasopharynx cancer , 2010, Radiation oncology.

[15]  Sina Honari,et al.  Distribution Matching Losses Can Hallucinate Features in Medical Image Translation , 2018, MICCAI.

[16]  Jelmer M. Wolterink,et al.  MR-Only Brain Radiation Therapy: Dosimetric Evaluation of Synthetic CTs Generated by a Dilated Convolutional Neural Network. , 2018, International journal of radiation oncology, biology, physics.

[17]  Karin Haustermans,et al.  The value of magnetic resonance imaging for radiotherapy planning. , 2014, Seminars in radiation oncology.

[18]  Koen Van Leemput,et al.  A patch-based pseudo-CT approach for MRI-only radiotherapy in the pelvis. , 2016, Medical physics.

[19]  Carri Glide-Hurst,et al.  Implementation of a novel algorithm for generating synthetic CT images from magnetic resonance imaging data sets for prostate cancer radiation therapy. , 2015, International journal of radiation oncology, biology, physics.

[20]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[21]  Peter R Seevinck,et al.  Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy , 2018, Physics in medicine and biology.

[22]  Joseph O. Deasy,et al.  Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation , 2018, MICCAI.

[23]  Jinsoo Uh,et al.  MRI-based treatment planning with pseudo CT generated through atlas registration. , 2014, Medical physics.

[24]  Jiang Du,et al.  Three‐dimensional ultrashort echo time cones T1ρ (3D UTE‐cones‐T1ρ) imaging , 2017, NMR in biomedicine.

[25]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Carri K Glide-Hurst,et al.  MRI-only treatment planning: benefits and challenges , 2018, Physics in medicine and biology.

[28]  Ninon Burgos,et al.  Comparative study of algorithms for synthetic CT generation from MRI: Consequences for MRI‐guided radiation planning in the pelvic region , 2018, Medical physics.

[29]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[30]  M. Maspero,et al.  OC-0294: MR-based synthetic CT with conditional Generative Adversarial Network for prostate RT planning , 2018 .

[31]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[32]  Anil Sethi,et al.  Influence of MRI on target volume delineation and IMRT planning in nasopharyngeal carcinoma. , 2003, International journal of radiation oncology, biology, physics.

[33]  J. L. Herraiz,et al.  Fast Patch-Based Pseudo-CT Synthesis from T1-Weighted MR Images for PET/MR Attenuation Correction in Brain Studies , 2016, The Journal of Nuclear Medicine.