Applications and limitations of machine learning in radiation oncology

Machine learning approaches to problem-solving are growing rapidly within healthcare, and radiation oncology is no exception. With the burgeoning interest in machine learning comes the significant risk of misaligned expectations as to what it can and cannot accomplish. This paper evaluates the role of machine learning and the problems it solves within the context of current clinical challenges in radiation oncology. The role of learning algorithms within the workflow for external beam radiation therapy are surveyed, considering simulation imaging, multimodal fusion, image segmentation, treatment planning, quality assurance, and treatment delivery and adaptation. For each aspect, the clinical challenges faced, the learning algorithms proposed, and the successes and limitations of various approaches are analyzed. It is observed that machine learning has largely thrived on reproducibly mimicking conventional human-driven solutions with more efficiency and consistency. On the other hand, since algorithms are generally trained using expert opinion as ground truth, machine learning is of limited utility where problems or ground truths are not well-defined, or if suitable measures of correctness are not available. As a result, machines may excel at replicating, automating and standardizing human behaviour on manual chores, meanwhile the conceptual clinical challenges relating to definition, evaluation, and judgement remain in the realm of human intelligence and insight.

[1]  Mannudeep K. Kalra,et al.  Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) , 2017, ArXiv.

[2]  Andre Dekker,et al.  Big Data in radiation therapy: challenges and opportunities , 2016, The British journal of radiology.

[3]  Fang-Fang Yin,et al.  Utilizing knowledge from prior plans in the evaluation of quality assurance , 2015, Physics in medicine and biology.

[4]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[5]  Martin J. Murphy,et al.  Machine Learning in Radiation Oncology , 2015 .

[6]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[7]  David A. Jaffray,et al.  Fully automated treatment planning for head and neck radiotherapy using a voxel-based dose prediction and dose mimicking method , 2016, Physics in medicine and biology.

[8]  A. Burgun,et al.  Big Data and machine learning in radiation oncology: State of the art and future prospects. , 2016, Cancer letters.

[9]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[10]  Dorin Comaniciu,et al.  An Artificial Agent for Robust Image Registration , 2016, AAAI.

[11]  Paul Aljabar,et al.  Comparative evaluation of autocontouring in clinical practice: A practical method using the Turing test , 2018, Medical physics.

[12]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[13]  Martha M. Matuszak,et al.  Using a Knowledge-Based Planning Model for Quality Assurance of Liver Stereotactic Body Radiation Therapy Plans , 2016 .

[14]  Sébastien Ourselin,et al.  Toward adaptive radiotherapy for head and neck patients: Uncertainties in dose warping due to the choice of deformable registration algorithm. , 2015, Medical physics.

[15]  Kianoush Nazarpour,et al.  Ensemble framework based real-time respiratory motion prediction for adaptive radiotherapy applications. , 2016, Medical engineering & physics.

[16]  Steve B. Jiang,et al.  A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning , 2017, Scientific Reports.

[17]  Benjamin E Nelms,et al.  Per-beam, planar IMRT QA passing rates do not predict clinically relevant patient dose errors. , 2011, Medical physics.

[18]  Jong Chul Ye,et al.  A deep convolutional neural network using directional wavelets for low‐dose X‐ray CT reconstruction , 2016, Medical physics.

[19]  Satomi Shiraishi,et al.  Knowledge-based prediction of three-dimensional dose distributions for external beam radiotherapy. , 2015, Medical physics.

[20]  T Kadir,et al.  TU-AB-202-10: How Effective Are Current Atlas Selection Methods for Atlas-Based Auto-Contouring in Radiotherapy Planning? , 2016, Medical physics.

[21]  Bulat Ibragimov,et al.  Segmentation of organs‐at‐risks in head and neck CT images using convolutional neural networks , 2017, Medical physics.

[22]  Peter R Seevinck,et al.  Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy , 2018, Physics in medicine and biology.

[23]  Kuo Men,et al.  Automatic segmentation of the clinical target volume and organs at risk in the planning CT for rectal cancer using deep dilated convolutional neural networks , 2017, Medical physics.

[24]  Sheng Xu,et al.  Adversarial Image Registration with Application for MR and TRUS Image Fusion , 2018, MLMI@MICCAI.

[25]  Martin J. Murphy.  Artificial Neural Networks to Emulate and Compensate Breathing Motion During Radiation Therapy , 2015 .

[26]  Carsten Brink,et al.  Automatic planning of head and neck treatment plans , 2016, Journal of applied clinical medical physics.

[27]  Alex Lallement,et al.  Survey on deep learning for radiotherapy , 2018, Comput. Biol. Medicine.

[28]  Gilmer Valdes,et al.  Clinical decision support of radiotherapy treatment planning: A data-driven machine learning strategy for patient-specific dosimetric decision making. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[29]  Issam El Naqa,et al.  The Role of Machine Learning in Knowledge-Based Response-Adapted Radiotherapy , 2018, Front. Oncol..

[30]  E W Korevaar,et al.  An automated, quantitative, and case-specific evaluation of deformable image registration in computed tomography images , 2018, Physics in medicine and biology.

[31]  Daniel Rueckert,et al.  Structured Decision Forests for Multi-modal Ultrasound Image Registration , 2015, MICCAI.

[32]  Frank Lohr,et al.  Expert system classifier for adaptive radiation therapy in prostate cancer , 2017, Australasian Physical & Engineering Sciences in Medicine.

[33]  Jen-Tzung Chien,et al.  Deep reinforcement learning for automated radiation adaptation in lung cancer , 2017, Medical physics.

[34]  Timothy C. Y. Chan,et al.  Automated Treatment Planning in Radiation Therapy using Generative Adversarial Networks , 2018, MLHC.

[35]  S. Ourselin,et al.  Evaluation of a multi-atlas CT synthesis approach for MRI-only radiotherapy treatment planning , 2017, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[36]  Steve B. Jiang,et al.  The management of imaging dose during image-guided radiotherapy: report of the AAPM Task Group 75. , 2007, Medical physics.

[37]  Elena Gallio,et al.  Evaluation of a commercial automatic treatment planning system for liver stereotactic body radiation therapy treatments. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[38]  Shunxing Bao,et al.  Fully convolutional neural networks improve abdominal organ segmentation , 2018, Medical Imaging.

[39]  Clifton D Fuller,et al.  Deep Learning Algorithm for Auto-Delineation of High-Risk Oropharyngeal Clinical Target Volumes With Built-In Dice Similarity Coefficient Parameter Optimization Function. , 2018, International journal of radiation oncology, biology, physics.

[40]  So-Yeon Park,et al.  A machine learning approach to the accurate prediction of multi-leaf collimator positional errors , 2016, Physics in medicine and biology.

[41]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[42]  Bernhard Schölkopf,et al.  Learning similarity measure for multi-modal 3D image registration , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  K. Brock,et al.  Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132 , 2017, Medical physics.

[44]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[45]  M. Wendling,et al.  Automated IMRT planning in Pinnacle , 2017, Strahlentherapie und Onkologie.

[46]  Weigang Hu,et al.  Is it possible for knowledge-based planning to improve intensity modulated radiation therapy plan quality for planners with different planning experiences in left-sided breast cancer patients? , 2017, Radiation Oncology.

[47]  Jin Liu,et al.  Discriminative Prior - Prior Image Constrained Compressed Sensing Reconstruction for Low-Dose CT Imaging , 2017, Scientific Reports.

[48]  Satomi Shiraishi,et al.  Fully automated, comprehensive knowledge-based planning for stereotactic radiosurgery: Preclinical validation through blinded physician review. , 2017, Practical radiation oncology.

[49]  Timothy D. Solberg,et al.  Machine Learning in Radiation Oncology: Opportunities, Requirements, and Needs , 2018, Front. Oncol..

[50]  Thomas G. Purdie,et al.  Groupwise Conditional Random Forests for Automatic Shape Classification and Contour Quality Assessment in Radiotherapy Planning , 2013, IEEE Transactions on Medical Imaging.

[51]  M Monz,et al.  Pareto navigation—algorithmic foundation of interactive multi-criteria IMRT planning , 2008, Physics in medicine and biology.

[52]  Jun Tan,et al.  Knowledge-based prediction of plan quality metrics in intracranial stereotactic radiosurgery. , 2015, Medical physics.

[53]  Higino Correia,et al.  Brain Tumour Segmentation based on Extremely Randomized Forest with high-level features , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[54]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[55]  Kanabu Nawa,et al.  Cone Beam Computed Tomography Image Quality Improvement Using a Deep Convolutional Neural Network , 2018, Cureus.

[56]  Li Zhang,et al.  Deep similarity learning for multimodal medical images , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[57]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[58]  Marcel van Herk,et al.  Errors and margins in radiotherapy. , 2004, Seminars in radiation oncology.

[59]  Dimos Baltas,et al.  Esophagus segmentation in CT via 3D fully convolutional neural network and random walk , 2017, Medical physics.

[60]  Maximilien Vermandel,et al.  Supervised machine learning-based classification scheme to segment the brainstem on MRI in multicenter brain tumor treatment context , 2015, International Journal of Computer Assisted Radiology and Surgery.

[61]  F Lohr,et al.  A machine learning tool for re-planning and adaptive RT: A multicenter cohort investigation. , 2016, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[62]  Jong Chul Ye,et al.  Deep Residual Learning for Compressed Sensing CT Reconstruction via Persistent Homology Analysis , 2016, ArXiv.

[63]  Steve B. Jiang,et al.  Dose Prediction with U-net: A Feasibility Study for Predicting Dose Distributions from Contours using Deep Learning on Prostate IMRT Patients , 2017, ArXiv.

[64]  G. Sharp,et al.  Vision 20/20: perspectives on automated image segmentation for radiotherapy. , 2014, Medical physics.

[65]  Ge Wang,et al.  Deep learning methods to guide CT image reconstruction and reduce metal artifacts , 2017, Medical Imaging.

[66]  Vorakarn Chanyavanich,et al.  Knowledge-based IMRT treatment planning for prostate cancer , 2010, Medical physics.

[67]  Fang Liu,et al.  Technical Note: Deep learning based MRAC using rapid ultrashort echo time imaging , 2018, Medical physics.

[68]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[69]  Nikos Paragios,et al.  Boosted metric learning for 3D multi-modal deformable registration , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[70]  Morteza Mardani,et al.  Deep-Learning Based Prediction of Achievable Dose for Personalizing Inverse Treatment Planning , 2016 .

[71]  David I Thwaites,et al.  Now you see it... Imaging in radiotherapy treatment planning and delivery. , 2007, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[72]  R. K. Ten Haken,et al.  Reinforcement Learning Strategies for Decision Making in Knowledge-Based Adaptive Radiation Therapy: Application in Liver Cancer , 2016 .

[73]  Hengyong Yu,et al.  Convolutional Neural Network Based Metal Artifact Reduction in X-Ray Computed Tomography , 2017, IEEE Transactions on Medical Imaging.

[74]  Quanzheng Li,et al.  Iterative Low-Dose CT Reconstruction With Priors Trained by Artificial Neural Network , 2017, IEEE Transactions on Medical Imaging.

[75]  M. Kessler.  Image registration and data fusion in radiation therapy. , 2006, The British journal of radiology.

[76]  Gilmer Valdes,et al.  An unsupervised convolutional neural network-based algorithm for deformable image registration , 2018, Physics in medicine and biology.

[77]  Gilmer Valdes,et al.  Artificial intelligence in radiation oncology: A specialty-wide disruptive transformation? , 2018, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[78]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[79]  Timothy D. Solberg,et al.  Deep nets vs expert designed features in medical physics: An IMRT QA case study , 2018, Medical physics.

[80]  Richard Speight,et al.  Automated, reference-free local error assessment of multimodal deformable image registration for radiotherapy in the head and neck. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[81]  David L Craft,et al.  Approximating convex pareto surfaces in multiobjective radiotherapy planning. , 2006, Medical physics.

[82]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[83]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2016, IEEE Transactions on Medical Imaging.

[84]  Yaozong Gao,et al.  Learning-Based Multimodal Image Registration for Prostate Cancer Radiation Therapy , 2016, MICCAI.

[85]  T. Roques,et al.  Patient selection and radiotherapy volume definition - can we improve the weakest links in the treatment chain? , 2014, Clinical oncology (Royal College of Radiologists (Great Britain)).

[86]  Margie Hunt,et al.  Dosimetric and workflow evaluation of first commercial synthetic CT software for clinical use in pelvis , 2017, Physics in medicine and biology.

[87]  Timothy D. Solberg,et al.  IMRT QA using machine learning: A multi‐institutional validation , 2017, Journal of applied clinical medical physics.

[88]  C. Njeh,et al.  Tumor delineation: The weakest link in the search for accuracy in radiotherapy , 2008, Journal of medical physics.

[89]  Max Dahele,et al.  Can knowledge-based DVH predictions be used for automated, individualized quality assurance of radiotherapy treatment plans? , 2015, Radiation Oncology.

[90]  Tao Zhang,et al.  Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[91]  Xun Jia,et al.  A DVH-guided IMRT optimization algorithm for automatic treatment planning and adaptive radiotherapy replanning. , 2014, Medical physics.

[92]  J. Conway,et al.  CT simulation for radiotherapy treatment planning. , 2002, The British journal of radiology.

[93]  Christopher Kurz,et al.  [OA127] Cone-beam CT intensity correction for adaptive radiotherapy of the prostate using deep learning , 2018 .

[94]  P. Noël,et al.  The evolution of image reconstruction for CT—from filtered back projection to artificial intelligence , 2018, European radiology.