Artificial intelligence in radiation oncology treatment planning: a brief overview

Among medical specialties, radiation oncology has long been an innovator and early adopter of therapeutic technologies. This specialty is now situated in prime position to be revolutionized by advances in artificial intelligence (AI), especially machine and deep learning. AI has been investigated by radiation oncologists and physicists in both general and niche radiotherapy planning tasks and has often demonstrated performance that is indistinguishable from human experts, while substantially shortening the time required to complete these tasks. We sought to review applications of AI to domains germane to radiation oncology, namely: image segmentation, treatment plan generation and optimization, normal tissue complication probability modeling, quality assurance (QA), and adaptive re-planning. We sought likewise to consider obstacles to AI adoption in the radiotherapy clinic, now primarily political, legal, and ethical rather than technical in nature.

[1]  H. Fred,et al.  Physician Burnout: Causes, Consequences, and (?) Cures. , 2018, Texas Heart Institute journal.

[2]  Frank Lohr,et al.  Expert system classifier for adaptive radiation therapy in prostate cancer , 2017, Australasian Physical & Engineering Sciences in Medicine.

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Gilmer Valdes,et al.  The application of artificial intelligence in the IMRT planning process for head and neck cancer. , 2018, Oral oncology.

[5]  Geraint Rees,et al.  Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy , 2018, ArXiv.

[6]  F Lohr,et al.  A machine learning tool for re-planning and adaptive RT: A multicenter cohort investigation. , 2016, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[7]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[8]  Cezary Szmigielski,et al.  Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images , 2018, International Journal of Computer Assisted Radiology and Surgery.

[9]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[10]  Fiona M. Callaghan,et al.  Use of internist's free time by ambulatory care Electronic Medical Record systems. , 2014, JAMA internal medicine.

[11]  Steve B. Jiang,et al.  Fully automated organ segmentation in male pelvic CT images , 2018, Physics in medicine and biology.

[12]  Steve B. Jiang,et al.  A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery , 2017, PloS one.

[13]  M. Knopp,et al.  NCTN Assessment on Current Applications of Radiomics in Oncology. , 2019, International journal of radiation oncology, biology, physics.

[14]  Y. Ge,et al.  Quantitative analysis of the factors which affect the interpatient organ-at-risk dose sparing variation in IMRT plans. , 2012, Medical physics.

[15]  Max Dahele,et al.  Using a knowledge-based planning solution to select patients for proton therapy. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[16]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[17]  Binbin Wu,et al.  An overlap-volume-histogram based method for rectal dose prediction and automated treatment planning in the external beam prostate radiotherapy following hydrogel injection. , 2012, Medical physics.

[18]  Timothy D. Solberg,et al.  Deep nets vs expert designed features in medical physics: An IMRT QA case study , 2018, Medical physics.

[19]  Jonathan J Wyatt,et al.  Systematic Review of Synthetic Computed Tomography Generation Methodologies for Use in Magnetic Resonance Imaging-Only Radiation Therapy. , 2018, International journal of radiation oncology, biology, physics.

[20]  Tian Liu,et al.  Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation , 2019, Medical physics.

[21]  Su Ruan,et al.  Multiorgan segmentation using distance-aware adversarial networks , 2019, Journal of medical imaging.

[22]  Fang-Fang Yin,et al.  Effect of machine learning methods on predicting NSCLC overall survival time based on Radiomics analysis , 2018, Radiation Oncology.

[23]  Xiaowei Liu,et al.  Comparison of the automatic segmentation of multiple organs at risk in CT images of lung cancer between deep convolutional neural network-based and atlas-based techniques , 2018, Acta oncologica.

[24]  H. Yue,et al.  A dosimetric evaluation of knowledge-based VMAT planning with simultaneous integrated boosting for rectal cancer patients. , 2016, Journal of applied clinical medical physics.

[25]  Hidetaka Arimura,et al.  Radiomics with artificial intelligence for precision medicine in radiation therapy , 2018, Journal of radiation research.

[26]  Bulat Ibragimov,et al.  Segmentation of parotid glands from registered CT and MR images. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[27]  Jonathan P. Weiner,et al.  Comment: "e-Iatrogenesis": The Most Critical Unintended Consequence of CPOE and other HIT , 2007, J. Am. Medical Informatics Assoc..

[28]  Aaron Babier,et al.  Knowledge‐based automated planning for oropharyngeal cancer , 2018, Medical physics.

[29]  Leixin Zhou,et al.  Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks , 2019, Medical physics.

[30]  Reid F Thompson,et al.  Artificial Intelligence in Radiation Oncology. , 2019, Hematology/oncology clinics of North America.

[31]  Lei Xing,et al.  Isodose feature‐preserving voxelization (IFPV) for radiation therapy treatment planning , 2018, Medical physics.

[32]  Dean F. Sittig,et al.  Prospective randomized double-blind study of atlas-based organ-at-risk autosegmentation-assisted radiation planning in head and neck cancer. , 2014, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[33]  Kuo Men,et al.  A feasibility study on an automated method to generate patient‐specific dose distributions for radiotherapy using deep learning , 2018, Medical physics.

[34]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[35]  Anselmo Cardoso de Paiva,et al.  Spinal cord detection in planning CT for radiotherapy through adaptive template matching, IMSLIC and convolutional neural networks , 2019, Comput. Methods Programs Biomed..

[36]  Giuseppe Baldazzi,et al.  A support vector machine tool for adaptive tomotherapy treatments: Prediction of head and neck patients criticalities. , 2015, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[37]  Yonggang Lu,et al.  A novel MRI segmentation method using CNN‐based correction network for MRI‐guided adaptive radiotherapy , 2018, Medical physics.

[38]  P. Lambin,et al.  Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach , 2014, Nature Communications.

[39]  D. Jaffray.  Image-guided radiotherapy: from current concept to future perspectives , 2012, Nature Reviews Clinical Oncology.

[40]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[41]  Deukwoo Kwon,et al.  Association of Radiomics and Metabolic Tumor Volumes in Radiation Treatment of Glioblastoma Multiforme. , 2017, International journal of radiation oncology, biology, physics.

[42]  Gilmer Valdes,et al.  Artificial Intelligence in Radiation Oncology Imaging. , 2018, International journal of radiation oncology, biology, physics.

[43]  Jiawei Fan,et al.  Automatic treatment planning based on three‐dimensional dose distribution predicted from deep learning technique , 2018, Medical physics.

[44]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[45]  Kumaradevan Punithakumar,et al.  A Fully Convolutional Deep Neural Network for Lung Tumor Boundary Tracking in MRI , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[46]  B. Heijmen,et al.  Automation in intensity modulated radiotherapy treatment planning-a review of recent innovations. , 2018, The British journal of radiology.

[47]  Philippe Lambin,et al.  Pre-treatment CT radiomics to predict 3-year overall survival following chemoradiotherapy of esophageal cancer , 2018, Acta oncologica.

[48]  Atallah Baydoun,et al.  Abdominal, multi-organ, auto-contouring method for online adaptive magnetic resonance guided radiotherapy: An intelligent, multi-level fusion approach , 2018, Artif. Intell. Medicine.

[49]  H. Abdollahi,et al.  Rectal wall MRI radiomics in prostate cancer patients: prediction of and correlation with early rectal toxicity , 2018, International journal of radiation biology.

[50]  Alberto Traverso,et al.  The radiation oncology ontology (ROO): Publishing linked data in radiation oncology using semantic web and ontology techniques , 2018, Medical physics.

[51]  Simukayi Mutasa,et al.  Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset , 2018, Journal of Digital Imaging.

[52]  Eduard Schreibmann,et al.  Heart chamber segmentation from CT using convolutional neural networks , 2018, Medical Imaging.

[53]  Odelin Charron,et al.  Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network , 2018, Comput. Biol. Medicine.

[54]  J. Fontenot,et al.  Impact of database quality in knowledge-based treatment planning for prostate cancer. , 2018, Practical radiation oncology.

[55]  Ali Selamat,et al.  Breast Cancer Detection Using Infrared Thermal Imaging and a Deep Learning Model , 2018, Sensors.

[56]  Andre Dekker,et al.  Standardizing Nomenclatures in Radiation Oncology , 2018 .

[57]  Kuo Men,et al.  Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks , 2017, Medical physics.

[58]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[59]  Xue-wen Chen,et al.  Big Data Deep Learning: Challenges and Perspectives , 2014, IEEE Access.

[60]  Massimo Esposito,et al.  Early prediction of radiotherapy-induced parotid shrinkage and toxicity based on CT radiomics and fuzzy classification , 2017, Artif. Intell. Medicine.

[61]  D Sarrut,et al.  Machine learning-based patient specific prompt-gamma dose monitoring in proton therapy , 2013, Physics in medicine and biology.

[62]  Robert A. Kowalski,et al.  Computational Logic and Human Thinking: How to Be Artificially Intelligent , 2011 .

[63]  Frederik Wenz,et al.  Knowledge-based radiation therapy (KBRT) treatment planning versus planning by experts: validation of a KBRT algorithm for prostate cancer treatment planning , 2015, Radiation oncology.

[64]  Jun Zhao,et al.  Automatic Delineation of the Clinical Target Volume in Rectal Cancer for Radiation Therapy using Three-dimensional Fully Convolutional Neural Networks , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[65]  E. Shortliffe,et al.  Clinical Decision Support in the Era of Artificial Intelligence. , 2018, JAMA.

[66]  T. Nyholm,et al.  A review of substitute CT generation for MRI-only radiation therapy , 2017, Radiation oncology.

[67]  Weigang Hu,et al.  Is it possible for knowledge-based planning to improve intensity modulated radiation therapy plan quality for planners with different planning experiences in left-sided breast cancer patients? , 2017, Radiation Oncology.

[68]  Jen-Tzung Chien,et al.  Radiation Therapy Outcomes Models in the Era of Radiomics and Radiogenomics: Uncertainties and Validation. , 2018, International journal of radiation oncology, biology, physics.

[69]  R. Steenbakkers,et al.  Parotid gland fat related Magnetic Resonance image biomarkers improve prediction of late radiation-induced xerostomia. , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[70]  Bassam Abdulkarim,et al.  Performance of Knowledge-Based Radiation Therapy Planning for the Glioblastoma Disease Site. , 2017, International journal of radiation oncology, biology, physics.

[71]  V. Grégoire,et al.  Analysis of EORTC-1219-DAHANCA-29 trial plans demonstrates the potential of knowledge-based planning to provide patient-specific treatment plan quality assurance. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[72]  Frederik Wenz,et al.  Performance of an atlas-based autosegmentation software for delineation of target volumes for radiotherapy of breast and anorectal cancer. , 2012, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[73]  Mechthild Krause,et al.  Radiation oncology in the era of precision medicine , 2016, Nature Reviews Cancer.

[74]  Geoffrey E. Hinton.  Deep Learning-A Technology With the Potential to Transform Health Care. , 2018, JAMA.

[75]  Clifton D Fuller,et al.  Deep Learning Algorithm for Auto-Delineation of High-Risk Oropharyngeal Clinical Target Volumes With Built-In Dice Similarity Coefficient Parameter Optimization Function. , 2018, International journal of radiation oncology, biology, physics.

[76]  D. Dong,et al.  Radiomics Features of Multiparametric MRI as Novel Prognostic Factors in Advanced Nasopharyngeal Carcinoma , 2017, Clinical Cancer Research.

[77]  Yang Lei,et al.  A learning-based automatic segmentation and quantification method on left ventricle in gated myocardial perfusion SPECT imaging: A feasibility study , 2019, Journal of Nuclear Cardiology.

[78]  Nuo Tong,et al.  Fully automatic multi‐organ segmentation for head and neck cancer radiotherapy using shape representation model constrained fully convolutional neural networks , 2018, Medical physics.

[79]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[80]  Erick Moreno-Centeno,et al.  Guided undersampling classification for automated radiation therapy quality assurance of prostate cancer treatment , 2018, Medical physics.

[81]  Satomi Shiraishi,et al.  Fully automated, comprehensive knowledge-based planning for stereotactic radiosurgery: Preclinical validation through blinded physician review. , 2017, Practical radiation oncology.

[82]  Kengo Ito,et al.  Automated prediction of dosimetric eligibility of patients with prostate cancer undergoing intensity-modulated radiation therapy using a convolutional neural network , 2018, Radiological Physics and Technology.

[83]  K. Brock,et al.  On the Fuzziness of Machine Learning, Neural Networks, and Artificial Intelligence in Radiation Oncology. , 2018, International journal of radiation oncology, biology, physics.

[84]  Thomas G. Purdie,et al.  Groupwise Conditional Random Forests for Automatic Shape Classification and Contour Quality Assessment in Radiotherapy Planning , 2013, IEEE Transactions on Medical Imaging.

[85]  Ying Zhuge,et al.  Brain tumor segmentation using holistically nested neural networks in MRI images , 2017, Medical physics.

[86]  J. Stockman.  Human Papillomavirus and Survival of Patients with Oropharyngeal Cancer , 2012 .

[87]  Andrew Bird,et al.  Clinical implementation of a knowledge based planning tool for prostate VMAT , 2017, Radiation Oncology.

[88]  Carlo Cavedon,et al.  Texture analysis of 3D dose distributions for predictive modelling of toxicity rates in radiotherapy. , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[89]  Xiaohui Xie,et al.  AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy , 2018, Medical physics.

[90]  Lubomir M. Hadjiiski,et al.  Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms , 2017, Physics in medicine and biology.

[91]  P. Lambin,et al.  Distributed learning: Developing a predictive model based on data from multiple hospitals without data leaving the hospital - A real life proof of concept. , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[92]  Jinkoo Kim,et al.  Development and evaluation of a clinical model for lung cancer patients using stereotactic body radiotherapy (SBRT) within a knowledge‐based algorithm for treatment planning , 2016, Journal of applied clinical medical physics.

[93]  Paul Aljabar,et al.  Autosegmentation for thoracic radiation treatment planning: A grand challenge at AAPM 2017 , 2018, Medical physics.

[94]  Eric C Ford,et al.  Deep learning for patient‐specific quality assurance: Identifying errors in radiotherapy delivery by radiomic analysis of gamma images with convolutional neural networks , 2019, Medical physics.

[95]  Gilmer Valdes,et al.  Use of TrueBeam developer mode for imaging QA , 2015, Journal of applied clinical medical physics.

[96]  Association of pre-treatment radiomic features with lung cancer recurrence following stereotactic body radiation therapy. , 2019, Physics in medicine and biology.

[97]  Antonio Criminisi,et al.  Autosegmentation of prostate anatomy for radiation treatment planning using deep decision forests of radiomic features , 2018, Physics in medicine and biology.

[98]  Carsten Brink,et al.  Automatic planning of head and neck treatment plans. , 2016, Journal of applied clinical medical physics.

[99]  Lijun Zhao,et al.  Automatic Nasopharyngeal Carcinoma Segmentation Using Fully Convolutional Networks with Auxiliary Paths on Dual-Modality PET-CT Images , 2019, Journal of Digital Imaging.

[100]  Eric C Ford,et al.  Bayesian network models for error detection in radiotherapy plans , 2015, Physics in medicine and biology.

[101]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[102]  C. Naylor,et al.  On the Prospects for a (Deep) Learning Health Care System , 2018, JAMA.

[103]  Timothy D. Solberg,et al.  IMRT QA using machine learning: A multi‐institutional validation , 2017, Journal of applied clinical medical physics.

[104]  J. Denny,et al.  Artificial intelligence, bias and clinical safety , 2019, BMJ Quality & Safety.

[105]  Issam El Naqa,et al.  The Role of Machine Learning in Knowledge-Based Response-Adapted Radiotherapy , 2018, Front. Oncol..

[106]  Laurence E. Court,et al.  The utility of quantitative CT radiomics features for improved prediction of radiation pneumonitis , 2018, Medical physics.

[107]  E. Medico,et al.  Predicting locally advanced rectal cancer response to neoadjuvant therapy with 18F-FDG PET and MRI radiomics features , 2019, European Journal of Nuclear Medicine and Molecular Imaging.

[108]  G. Reifenberger,et al.  The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary , 2016, Acta Neuropathologica.

[109]  Bulat Ibragimov,et al.  Combining deep learning with anatomical analysis for segmentation of the portal vein for liver SBRT planning , 2017, Physics in medicine and biology.

[110]  P. Lambin,et al.  Developing and Validating a Survival Prediction Model for NSCLC Patients Through Distributed Learning Across 3 Countries , 2017, International journal of radiation oncology, biology, physics.

[111]  A. Burgun,et al.  Big Data and machine learning in radiation oncology: State of the art and future prospects. , 2016, Cancer letters.

[112]  Bulat Ibragimov,et al.  Segmentation of organs‐at‐risks in head and neck CT images using convolutional neural networks , 2017, Medical physics.

[113]  Charles R. Thomas,et al.  The Future of Artificial Intelligence in Radiation Oncology. , 2018, International journal of radiation oncology, biology, physics.

[114]  H. Hricak.  2016 New Horizons Lecture: Beyond Imaging-Radiology of Tomorrow. , 2018, Radiology.

[115]  Mark Bangert,et al.  Design and Selection of Machine Learning Methods Using Radiomics and Dosiomics for Normal Tissue Complication Probability Modeling of Xerostomia , 2018, Front. Oncol..

[116]  Clifton D. Fuller,et al.  Exploring Applications of Radiomics in Magnetic Resonance Imaging of Head and Neck Cancer: A Systematic Review , 2018, Front. Oncol..

[117]  Max Dahele,et al.  Can knowledge-based DVH predictions be used for automated, individualized quality assurance of radiotherapy treatment plans? , 2015, Radiation Oncology.

[118]  Tao Zhang,et al.  Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[119]  Quynh-Thu Le,et al.  The use of texture-based radiomics CT analysis to predict outcomes in early-stage non-small cell lung cancer treated with stereotactic ablative radiotherapy. , 2019, The British journal of radiology.