MR‐based treatment planning in radiation therapy using a deep learning approach

Abstract Purpose To develop and evaluate the feasibility of deep learning approaches for MR‐based treatment planning (deepMTP) in brain tumor radiation therapy. Methods and materials A treatment planning pipeline was constructed using a deep learning approach to generate continuously valued pseudo CT images from MR images. A deep convolutional neural network was designed to identify tissue features in volumetric head MR images training with co‐registered kVCT images. A set of 40 retrospective 3D T1‐weighted head images was utilized to train the model, and evaluated in 10 clinical cases with brain metastases by comparing treatment plans using deep learning generated pseudo CT and using an acquired planning kVCT. Paired‐sample Wilcoxon signed rank sum tests were used for statistical analysis to compare dosimetric parameters of plans made with pseudo CT images generated from deepMTP to those made with kVCT‐based clinical treatment plan (CTTP). Results deepMTP provides an accurate pseudo CT with Dice coefficients for air: 0.95 ± 0.01, soft tissue: 0.94 ± 0.02, and bone: 0.85 ± 0.02 and a mean absolute error of 75 ± 23 HU compared with acquired kVCTs. The absolute percentage differences of dosimetric parameters between deepMTP and CTTP was 0.24% ± 0.46% for planning target volume (PTV) volume, 1.39% ± 1.31% for maximum dose and 0.27% ± 0.79% for the PTV receiving 95% of the prescribed dose (V95). Furthermore, no significant difference was found for PTV volume (P = 0.50), the maximum dose (P = 0.83) and V95 (P = 0.19) between deepMTP and CTTP. Conclusions We have developed an automated approach (deepMTP) that allows generation of a continuously valued pseudo CT from a single high‐resolution 3D MR image and evaluated it in partial brain tumor treatment planning. The deepMTP provided dose distribution with no significant difference relative to a kVCT‐based standard volumetric modulated arc therapy plans.

[1]  Paul Kinahan,et al.  Attenuation correction for a combined 3D PET/CT scanner. , 1998, Medical physics.

[2]  Anne Bol,et al.  Evaluation of a multimodality image (CT, MRI and PET) coregistration procedure on phantom and head and neck cancer patients: accuracy, reproducibility and consistency. , 2003, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  B. Dawant,et al.  Phantom validation of coregistration of PET and CT for image-guided radiotherapy. , 2004, Medical physics.

[4]  David M. Doddrell,et al.  Geometric distortion in clinical MRI systems Part II: correction using a 3D phantom. , 2004, Magnetic resonance imaging.

[5]  Deming Wang,et al.  Geometric distortion in clinical MRI systems Part I: evaluation using a 3D phantom. , 2004, Magnetic resonance imaging.

[6]  Marc L Kessler,et al.  Use and uncertainties of mutual information for computed tomography/ magnetic resonance (CT/MR) registration post permanent implant of the prostate. , 2005, Medical physics.

[7]  B. Schölkopf,et al.  Towards quantitative PET/MRI: a review of MR-based attenuation correction techniques , 2009, European Journal of Nuclear Medicine and Molecular Imaging.

[8]  Tufve Nyholm,et al.  Systematisation of spatial uncertainties for comparison between a MR and a CT-based radiotherapy workflow for prostate treatments , 2009, Radiation oncology.

[9]  Juan de Lara,et al.  Supporting user-oriented analysis for multi-view domain-specific visual languages , 2009, Inf. Softw. Technol..

[10]  B. Zackrisson,et al.  Dedicated magnetic resonance imaging in the radiotherapy clinic. , 2009, International journal of radiation oncology, biology, physics.

[11]  Nassir Navab,et al.  Tissue Classification as a Potential Approach for Attenuation Correction in Whole-Body PET/MRI: Evaluation with PET/CT Data , 2009, Journal of Nuclear Medicine.

[12]  B. Fallone,et al.  First MR images obtained during megavoltage photon irradiation from a prototype integrated linac-MR system. , 2009, Medical physics.

[13]  Paul Kinahan,et al.  Positron emission tomography-computed tomography standardized uptake values in clinical practice and assessing response to therapy. , 2010, Seminars in ultrasound, CT, and MR.

[14]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[15]  Tufve Nyholm,et al.  Treatment planning using MRI data: an analysis of the dose calculation accuracy for different treatment regions , 2010, Radiation oncology.

[16]  Kenneth Ulin,et al.  Results of a multi-institutional benchmark test for cranial CT/MR image registration. , 2010, International journal of radiation oncology, biology, physics.

[17]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[18]  Aliaksandr Karotki,et al.  Comparison of bulk electron density and voxel‐based electron density treatment planning , 2011, Journal of applied clinical medical physics.

[19]  S. Mutic,et al.  WE-G-BRB-08: TG-51 Calibration of First Commercial MRI-Guided IMRT System in the Presence of 0.35 Tesla Magnetic Field. , 2012, Medical physics.

[20]  V. Schulz,et al.  Challenges and current methods for attenuation correction in PET/MR , 2013, Magnetic Resonance Materials in Physics, Biology and Medicine.

[21]  Olivier Salvado,et al.  An atlas-based electron density mapping method for magnetic resonance imaging (MRI)-alone treatment planning and adaptive MRI-based prostate radiation therapy. , 2012, International journal of radiation oncology, biology, physics.

[22]  N. Schwenzer,et al.  A strategy for multimodal deformable image registration to integrate PET/MR into radiotherapy treatment planning , 2013, Acta oncologica.

[23]  Christopher M. Rank,et al.  MRI-based simulation of treatment plans for ion radiotherapy in the brain region. , 2013, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[24]  D. Jaffray,et al.  Harmonic analysis for the characterization and correction of geometric distortion in MRI. , 2014, Medical physics.

[25]  Jinsoo Uh,et al.  MRI-based treatment planning with pseudo CT generated through atlas registration. , 2014, Medical physics.

[26]  Sasa Mutic,et al.  The ViewRay system: magnetic resonance-guided and controlled radiotherapy. , 2014, Seminars in radiation oncology.

[27]  Tiina Seppälä,et al.  A dual model HU conversion from MRI intensity values within and outside of bone segment for MRI-based radiotherapy treatment planning of prostate cancer. , 2013, Medical physics.

[28]  Lei Xing,et al.  A unifying probabilistic Bayesian approach to derive electron density from MRI for radiation therapy treatment planning , 2014, Physics in medicine and biology.

[29]  Karin Haustermans,et al.  The value of magnetic resonance imaging for radiotherapy planning. , 2014, Seminars in radiation oncology.

[30]  John E. Bayouth,et al.  A dose homogeneity and conformity evaluation between ViewRay and pinnacle-based linear accelerator IMRT treatment plans , 2014, Journal of medical physics.

[31]  H. Kjer,et al.  A voxel-based investigation for MRI-only radiotherapy of the brain using ultra short echo times , 2014, Physics in medicine and biology.

[32]  K. Smit,et al.  Relative dosimetry in a 1.5 T magnetic field: an MR-linac compatible prototype scanning water phantom , 2014, Physics in medicine and biology.

[33]  B. Erickson,et al.  Comprehensive MRI simulation methodology using a dedicated MRI scanner in radiation oncology for external beam radiation treatment planning. , 2014, Medical physics.

[34]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Adam Johansson,et al.  Accuracy of inverse treatment planning on substitute CT images derived from MR data for brain lesions , 2015, Radiation oncology.

[37]  Jason Dowling,et al.  A review of segmentation and deformable registration methods applied to adaptive cervical cancer radiation therapy treatment planning , 2015, Artif. Intell. Medicine.

[38]  Maria A Schmidt,et al.  Radiotherapy planning using MRI , 2015, Physics in medicine and biology.

[39]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[40]  J. Edmund,et al.  Patch-based generation of a pseudo CT from conventional MRI sequences for MRI-only radiotherapy of the brain. , 2015, Medical physics.

[41]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[42]  Charis Kontaxis,et al.  A new methodology for inter- and intrafraction plan adaptation for the MR-linac , 2015, Physics in medicine and biology.

[43]  H. Zaidi,et al.  Vision 20/20: Magnetic resonance imaging-guided attenuation correction in PET/MRI: Challenges, solutions, and opportunities. , 2016, Medical physics.

[44]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Rabih Hammoud,et al.  Characterization of 3D geometric distortion of magnetic resonance imaging scanners commissioned for radiation therapy planning. , 2016, Magnetic resonance imaging.

[46]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[47]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[48]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[49]  Tianyu Zhao,et al.  Online Magnetic Resonance Image Guided Adaptive Radiation Therapy: First Clinical Applications. , 2016, International journal of radiation oncology, biology, physics.

[50]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[51]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[52]  Leonard Wee,et al.  Intensity-based dual model method for generation of synthetic CT images from standard T2-weighted MR images - Generalized technique for four different MR scanners. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[53]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[54]  T. Nyholm,et al.  A review of substitute CT generation for MRI-only radiation therapy , 2017, Radiation oncology.

[55]  N. Kadoya Current Status of MR-Linac System. , 2017, Igaku butsuri : Nihon Igaku Butsuri Gakkai kikanshi = Japanese journal of medical physics : an official journal of Japan Society of Medical Physics.

[56]  Yasheng Chen,et al.  Attenuation Correction of PET/MR Imaging. , 2017, Magnetic resonance imaging clinics of North America.

[57]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  B. Stemkens,et al.  Towards fast online intrafraction replanning for free-breathing stereotactic body radiation therapy with the MR-linac , 2017, Physics in medicine and biology.

[59]  Andrew P. Leynes,et al.  Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI Attenuation Correction Using Deep Convolutional Neural Networks with Multiparametric MRI , 2017, The Journal of Nuclear Medicine.

[60]  Richard Kijowski,et al.  Deep convolutional neural network for segmentation of knee joint anatomy , 2018, Magnetic resonance in medicine.

[61]  Fang Liu,et al.  Technical Note: Deep learning based MRAC using rapid ultrashort echo time imaging , 2018, Medical physics.

[62]  Richard Kijowski,et al.  Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging , 2018, Magnetic resonance in medicine.

[63]  Stanley J Kruger,et al.  Pulmonary ventilation imaging in asthma and cystic fibrosis using oxygen‐enhanced 3D radial ultrashort echo time MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[64]  A. McMillan,et al.  Deep learning Mr imaging–based attenuation correction for PeT/Mr imaging 1 , 2017 .

[65]  Fang Liu,et al.  Bayesian convolutional neural network based MRI brain extraction on nonhuman primates , 2018, NeuroImage.

[66]  Fang Liu,et al.  SUSAN: segment unannotated image structure using adversarial network , 2018, Magnetic resonance in medicine.