Deep dose plugin: towards real-time Monte Carlo dose calculation through a deep learning-based denoising algorithm

Monte Carlo (MC) simulation is considered the gold standard method for radiotherapy dose calculation. However, achieving high precision requires a large number of simulation histories, which is time consuming. The use of computer graphics processing units (GPUs) has greatly accelerated MC simulation and allows dose calculation within a few minutes for a typical radiotherapy treatment plan. However, some clinical applications demand real time efficiency for MC dose calculation. To tackle this problem, we have developed a real time, deep learning based dose denoiser that can be plugged into a current GPU based MC dose engine to enable real time MC dose calculation. We used two different acceleration strategies to achieve this goal: 1) we applied voxel unshuffle and voxel shuffle operators to decrease the input and output sizes without any information loss, and 2) we decoupled the 3D volumetric convolution into a 2D axial convolution and a 1D slice convolution. In addition, we used a weakly supervised learning framework to train the network, which greatly reduces the size of the required training dataset and thus enables fast fine tuning based adaptation of the trained model to different radiation beams. Experimental results show that the proposed denoiser can run in as little as 39 ms, which is around 11.6 times faster than the baseline model. As a result, the whole MC dose calculation pipeline can be finished within 0.15 seconds, including both GPU MC dose calculation and deep learning based denoising, achieving the real time efficiency needed for some radiotherapy applications, such as online adaptive radiotherapy.

[1]  L. Xing,et al.  Aperture modulated arc therapy. , 2003, Physics in medicine and biology.

[2]  Holly Ning,et al.  Comparison of intensity-modulated radiotherapy, adaptive radiotherapy, proton radiotherapy, and adaptive proton radiotherapy for treatment of locally advanced head and neck cancer. , 2011, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  J. Sempau,et al.  DPM, a fast, accurate Monte Carlo code optimized for photon and electron radiotherapy treatment planning dose calculations , 2000 .

[5]  I. Kawrakow.  Accurate condensed history Monte Carlo simulation of electron transport. I. EGSnrc, the new EGS4 version. , 2000, Medical physics.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Linghong Zhou,et al.  Automatic treatment plan re-optimization for adaptive radiotherapy guided with the initial plan DVHs , 2013, Physics in medicine and biology.

[8]  Ali Amer,et al.  Online adaptive radiotherapy of the bladder: small bowel irradiated-volume reduction. , 2006, International journal of radiation oncology, biology, physics.

[9]  J. F. Briesmeister.  MCNP-A General Monte Carlo N-Particle Transport Code , 1993 .

[10]  Jan-Jakob Sonke,et al.  A fast algorithm for gamma evaluation in 3D. , 2007, Medical physics.

[11]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[12]  A. Dell'Acqua,et al.  Geant4 - A simulation toolkit , 2003 .

[13]  M Earl,et al.  Inverse planning for intensity-modulated arc therapy using direct aperture optimization , 2003, Physics in medicine and biology.

[14]  I Kawrakow,et al.  On the de-noising of Monte Carlo calculated dose distributions. , 2002, Physics in medicine and biology.

[15]  C. Ma,et al.  Clinical implementation of a Monte Carlo treatment planning system. , 1999, Medical physics.

[16]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Gang Yu,et al.  Cascaded Pyramid Network for Multi-person Pose Estimation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Lei Dong,et al.  Adaptive radiotherapy for head and neck cancer--dosimetric results from a prospective clinical trial. , 2013, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[19]  X Allen Li,et al.  Validation of an online replanning technique for prostate adaptive radiotherapy. , 2011, Physics in medicine and biology.

[20]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  K. Otto,et al.  A comparison of volumetric modulated arc therapy and conventional intensity-modulated radiotherapy for frontal and temporal high-grade gliomas. , 2010, International journal of radiation oncology, biology, physics.

[22]  Xavier Geets,et al.  Adaptive radiotherapy of head and neck cancer. , 2010, Seminars in radiation oncology.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[25]  Jan-Jakob Sonke,et al.  Adaptive radiotherapy for prostate cancer using kilovoltage cone-beam computed tomography: first clinical results. , 2008, International journal of radiation oncology, biology, physics.

[26]  P. Xia,et al.  Multileaf collimator leaf sequencing algorithm for intensity modulated beams with multiple static segments. , 1998, Medical physics.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  D. Shepard,et al.  A generalized inverse planning tool for volumetric-modulated arc therapy , 2009, Physics in medicine and biology.

[30]  D. Rogers,et al.  EGS4 code system , 1985 .

[31]  T. Kron,et al.  The outcome of a multi-centre feasibility study of online adaptive radiotherapy for muscle-invasive bladder cancer TROG 10.01 BOLART. , 2014, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[32]  David A Jaffray,et al.  Intensity-modulated arc therapy with dynamic multileaf collimation : an alternative to tomotherapy , 2002 .

[33]  Steve B. Jiang,et al.  Development of a GPU-based Monte Carlo dose calculation code for coupled electron–photon transport , 2009, Physics in medicine and biology.

[34]  Nicolas Holzschuch,et al.  A detail preserving neural network model for Monte Carlo denoising , 2020, Computational Visual Media.

[35]  Benoît Ozell,et al.  GPUMCD: A new GPU-oriented Monte Carlo dose calculation platform. , 2011, Medical physics.

[36]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[37]  Tianyu Zhao,et al.  A GPU-accelerated Monte Carlo dose calculation platform and its application toward validating an MRI-guided radiation therapy beam model. , 2016, Medical physics.

[38]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Steve B. Jiang,et al.  Validation of a GPU-based Monte Carlo code (gPMC) for proton radiation therapy: clinical cases study , 2015, Physics in medicine and biology.

[40]  T. Bortfeld,et al.  Methods of image reconstruction from projections applied to conformation radiotherapy. , 1990, Physics in medicine and biology.

[41]  J. Hesser,et al.  GMC: a GPU implementation of a Monte Carlo dose calculation based on Geant4 , 2012, Physics in medicine and biology.

[42]  Steve B Jiang,et al.  GPU-based ultra-fast dose calculation using a finite size pencil beam model. , 2009, Physics in medicine and biology.

[43]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Xun Jia,et al.  SCORE system for online adaptive radiotherapy , 2015 .

[45]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[46]  Bin Liu,et al.  Ternary Weight Networks , 2016, ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  D. Convery,et al.  The generation of intensity-modulated fields for conformal radiotherapy by dynamic collimation , 1992 .

[48]  Umair Javaid,et al.  Mitigating inherent noise in Monte Carlo dose distributions using dilated U-Net. , 2019, Medical physics.

[49]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[50]  S. Webb.  The physical basis of IMRT and inverse planning. , 2003, The British journal of radiology.

[51]  J. Sempau,et al.  PENELOPE-2006: A Code System for Monte Carlo Simulation of Electron and Photon Transport , 2009 .

[52]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[53]  Cedric X. Yu,et al.  Intensity-modulated arc therapy with dynamic multileaf collimation: an alternative to tomotherapy. , 1995, Physics in medicine and biology.

[54]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Rui Wang,et al.  Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation , 2019, ACM Trans. Graph..

[56]  T. Bortfeld,et al.  X-ray field compensation with multileaf collimators. , 1994, International journal of radiation oncology, biology, physics.

[57]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[58]  Xun Jia,et al.  GPU-based fast Monte Carlo simulation for radiotherapy dose calculation. , 2011, Physics in medicine and biology.

[59]  W Schlegel,et al.  Intensity modulation with the "step and shoot" technique using a commercial MLC: a planning study. Multileaf collimator. , 1999, International journal of radiation oncology, biology, physics.

[60]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  S. Webb.  Optimisation of conformal radiotherapy dose distributions by simulated annealing. , 1989, Physics in medicine and biology.

[62]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Hongming Shan,et al.  MCDNet – A Denoising Convolutional Neural Network to Accelerate Monte Carlo Radiation Transport Simulations: A Proof of Principle With Patient Dose From X-Ray CT Imaging , 2019, IEEE Access.

[64]  Ryan Neph,et al.  DeepMCDose: A Deep Learning Method for Efficient Monte Carlo Beamlet Dose Calculation by Predictive Denoising in MR-Guided Radiotherapy , 2019, AIRT@MICCAI.

[65]  Tomas Kron,et al.  Online adaptive radiotherapy for muscle-invasive bladder cancer: results of a pilot study. , 2011, International journal of radiation oncology, biology, physics.

[66]  Yichen Wei,et al.  Simple Baselines for Human Pose Estimation and Tracking , 2018, ECCV.

[67]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[68]  Steve B. Jiang,et al.  GPU-based fast Monte Carlo dose calculation for proton therapy , 2012, Physics in medicine and biology.

[69]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[70]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[71]  I. Kawrakow,et al.  3D electron dose calculation using a Voxel based Monte Carlo algorithm (VMC). , 1996, Medical physics.

[72]  C. Ma,et al.  A Monte Carlo dose calculation tool for radiotherapy treatment planning. , 2002, Physics in medicine and biology.

[73]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[74]  K Otto,et al.  Volumetric modulated Arc therapy and conventional intensity-modulated radiotherapy for simultaneous maximal intraprostatic boost: a planning comparison study. , 2009, Clinical oncology (Royal College of Radiologists (Great Britain)).

[75]  K. Otto,et al.  Volumetric modulated arc therapy for delivery of prostate radiotherapy: comparison with intensity-modulated radiotherapy and three-dimensional conformal radiotherapy. , 2008, International journal of radiation oncology, biology, physics.

[76]  R. Jeraj,et al.  Re-optimization in adaptive radiotherapy. , 2002, Physics in medicine and biology.