A unified spatio-temporal parallelization framework for accelerated Monte Carlo radiobiological modeling of electron tracks and subsequent radiation chemistry

Abstract Monte Carlo (MC) nano-scale modeling of the cellular damage is desirable but most times is prohibitive for large scaled systems due to their intensive computational cost. In this study a parallelized computational framework is presented, for accelerated MC simulations of both particle propagation and subsequent radiation chemistry at the subcellular level. Given the inherent parallelism of the electron tracks, the physical stage was “embarrassingly parallelized” into a number of independent tasks. For the chemical stage, the diffusion–reaction of the radical species was simulated with a time-driven kinetic Monte Carlo algorithm (KMC) based on the Smoluchowski formalism and the parallelization was realized by employing a spatio-temporal linked-list cell method based on a spatial subdivision with a uniform grid. The evaluation of our method was established on two metrics: speedup and efficiency. The results indicated a linear speedup ratio for the physical stage and a linear latency for shared- versus a distributed-memory system with a maximum of 3.6 ⋅ 10 − 3 % per electron track. For the chemical stage, a series of simulations were performed to show how the execution time per step was scaling with respect to the number of radical species and a 5.7× speedup was achieved when a larger number of reactants were simulated and eight processors were employed. The simulations were deployed on the Amazon EC2 infrastructure. It is also elucidated how the overhead started becoming significant as the number of reactant species decrease relative to the number of processors. The method reported here lays the methodological foundations for accelerated MC simulations and allows envisaging a future use for large-scale radiobiological modeling of multi-cellular systems involved into a clinical scenario.

[1]  Edward D. Lazowska,et al.  Speedup Versus Efficiency in Parallel Systems , 1989, IEEE Trans. Computers.

[2]  Kostas Kostarelos,et al.  A Monte Carlo track structure code for electrons (~10 eV-10 keV) and protons (~0.3-10 MeV) in water: partitioning of energy and collision events , 2000 .

[3]  K Y Sanbonmatsu,et al.  High performance computing in biology: multimillion atom simulations of nanoscale systems. , 2007, Journal of structural biology.

[4]  Rajiv K. Kalia,et al.  Performance Modeling, Analysis, and Optimization of Cell-List Based Molecular Dynamics , 2010, CSC.

[5]  Silas Boyd-Wickizer,et al.  A Software Approach to Unifying Multicore Caches , 2011 .

[6]  H. Paretzke,et al.  Calculation of electron impact ionization cross sections of DNA using the Deutsch–Märk and Binary–Encounter–Bethe formalisms , 2003 .

[7]  Jerome Spanier,et al.  Condensed history Monte Carlo methods for photon transport problems , 2007, J. Comput. Phys..

[8]  K. N. Joshipura,et al.  Theoretical calculations of the total and ionization cross sections for electron impact on some simple biomolecules , 2006 .

[9]  Sang Hyun Cho,et al.  Estimation of tumour dose enhancement due to gold nanoparticles during typical radiation treatments: a preliminary Monte Carlo study , 2005, Physics in medicine and biology.

[10]  A. V. van Duin,et al.  Reactive molecular dynamics study on the first steps of DNA damage by free hydroxyl radicals. , 2011, The journal of physical chemistry. A.

[11]  P. Hadjidoukas,et al.  Monte Carlo single-cell dosimetry of Auger-electron emitting radionuclides , 2010, Physics in medicine and biology.

[12]  L. Beckett,et al.  Nanomolecular HLA-DR10 antibody mimics: A potent system for molecular targeted therapy and imaging. , 2008, Cancer biotherapy & radiopharmaceuticals.

[13]  Weiqiang Wang,et al.  A Scalable Hierarchical Parallelization Framework for Molecular Dynamics Simulation on Multicore Clusters , 2009, PDPTA.

[14]  Constantinos Evangelinos,et al.  Cloud Computing for parallel Scientific HPC Applications: Feasibility of Running Coupled Atmosphere- , 2008 .

[15]  Dionisios G. Vlachos,et al.  Parallelization of tau-leap coarse-grained Monte Carlo simulations on GPUs , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[16]  Gui-Rong Liu,et al.  Improved neighbor list algorithm in molecular simulations using cell decomposition and data sorting method , 2004, Comput. Phys. Commun..

[17]  Nikil D. Dutt,et al.  An Efficient Simulation Environment for Modeling Large-Scale Cortical Processing , 2011, Front. Neuroinform..

[18]  Steve B. Jiang,et al.  Development of a GPU-based Monte Carlo dose calculation code for coupled electron–photon transport , 2009, Physics in medicine and biology.

[19]  Stephen J McMahon,et al.  Radiotherapy in the presence of contrast agents: a general figure of merit and its application to gold nanoparticles , 2008, Physics in medicine and biology.

[20]  P. Hadjidoukas,et al.  A Monte Carlo study of absorbed dose distributions in both the vapor and liquid phases of water by intermediate energy electrons based on different condensed-history transport schemes , 2008, Physics in medicine and biology.

[21]  Indrin J Chetty,et al.  Implementation of the DPM Monte Carlo code on a parallel architecture for treatment planning applications. , 2004, Medical physics.

[22]  M. El-Sayed,et al.  Nuclear targeting of gold nanoparticles in cancer cells induces DNA damage, causing cytokinesis arrest and apoptosis. , 2010, Journal of the American Chemical Society.

[23]  Christer Ericson,et al.  Real-Time Collision Detection , 2004 .

[24]  Sang Hyun Cho,et al.  Estimation of microscopic dose enhancement factor around gold nanoparticles by Monte Carlo calculations. , 2010, Medical physics.

[25]  C E deAlmeida,et al.  The invariance of the total direct DNA strand break yield. , 2011, Medical physics.

[26]  T. Kusama,et al.  Monte Carlo simulation of physicochemical processes of liquid water radiolysis , 1997 .

[27]  Z. Tan,et al.  A new calculation on spectrum of direct DNA damage induced by low-energy electrons , 2010, Radiation and environmental biophysics.

[28]  L. Pinsky,et al.  A database of frequency distributions of energy depositions in small-size targets by electrons and ions. , 2011, Radiation protection dosimetry.

[29]  Jeremy Kepner,et al.  'pMATLAB Parallel MATLAB Library' , 2007, Int. J. High Perform. Comput. Appl..

[30]  Peng Wang,et al.  Implementing molecular dynamics on hybrid high performance computers - short range forces , 2011, Comput. Phys. Commun..

[31]  Jonas S. Almeida,et al.  mGrid: A load-balanced distributed computing environment for the remote execution of the user-defined Matlab code , 2006, BMC Bioinformatics.

[32]  D. W. Vinter,et al.  Inhomogeneous deposition of radiopharmaceuticals at the cellular level: experimental evidence and dosimetric implications. , 1990, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[33]  Jochen Ditterich,et al.  Splash: A Software Tool for Stereotactic Planning of Recording Chamber Placement and Electrode Trajectories , 2011, Front. Neuroinform..

[34]  W. V. Prestwich,et al.  Modelling DNA damage induced by different energy photons and tritium beta-particles. , 1998, International journal of radiation biology.

[35]  Naga K. Govindaraju,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007 .

[36]  N. Papanikolaou,et al.  Dose-calculation algorithms in the context of inhomogeneity corrections for high energy photon beams. , 2009, Medical physics.

[37]  D. Emfietzoglou,et al.  Inelastic scattering of low-energy electrons in liquid water computed from optical-data models of the Bethe surface , 2012, International journal of radiation biology.

[38]  Lorenzo Dematté,et al.  GPU computing for systems biology , 2010, Briefings Bioinform..

[39]  Herwig G. Paretzke,et al.  Electron inelastic-scattering cross sections in liquid water , 1999 .

[40]  C Villagrasa,et al.  Comparison of GEANT4 very low energy cross section models with experimental data in water. , 2010, Medical physics.

[41]  Marty Humphrey,et al.  A quantitative analysis of high performance computing with Amazon's EC2 infrastructure: The death of the local cluster? , 2009, 2009 10th IEEE/ACM International Conference on Grid Computing.

[42]  Scott B. Baden,et al.  A large scale Monte Carlo simulator for cellular microphysiology , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[43]  Edward Walker,et al.  Benchmarking Amazon EC2 for High-Performance Scientific Computing , 2008, login Usenix Mag..

[44]  Laxmi N. Bhuyan,et al.  Software techniques to improve virtualized I/O performance on multi-core systems , 2008, ANCS '08.

[45]  Lei Xing,et al.  Toward real-time Monte Carlo simulation using a commercial cloud computing infrastructure , 2011, Physics in medicine and biology.

[46]  P. Andreo Monte Carlo techniques in medical radiation physics. , 1991, Physics in medicine and biology.

[47]  P O'Neill,et al.  Computational modelling of low-energy electron-induced DNA damage by early physical and chemical events. , 1997, International journal of radiation biology.

[48]  T. Kusama,et al.  Monte Carlo simulation of physicochemical processes of liquid water radiolysis. The effects of dissolved oxygen and OH scavenger. , 1997, Radiation and Environmental Biophysics.

[49]  Anne E. Trefethen,et al.  MultiMATLAB: MATLAB on Multiple Processors , 1996 .

[50]  F. A. Smith,et al.  Calculation of initial and primary yields in the radiolysis of water , 1994 .

[51]  Dimitris Emfietzoglou,et al.  Energy Loss of Hydrogen- and Helium-Ion Beams in DNA: Calculations Based on a Realistic Energy-Loss Function of the Target , 2011, Radiation research.

[52]  Yiannis Kaznessis,et al.  Accurate hybrid stochastic simulation of a system of coupled chemical or biochemical reactions. , 2005, The Journal of chemical physics.

[53]  Steve Plimpton,et al.  Fast parallel algorithms for short-range molecular dynamics , 1993 .

[54]  D. Weiss,et al.  Cytotoxicity, genotoxicity and intracellular distribution of the Auger electron emitter 65Zn in two human cell lines , 2004, Radiation and environmental biophysics.

[55]  R Kohno,et al.  Clinical implementation of a GPU-based simplified Monte Carlo method for a treatment planning system of proton beam therapy , 2011, Physics in medicine and biology.

[56]  A. Chatterjee,et al.  Energy deposition mechanisms and biochemical aspects of DNA strand breaks by ionizing radiation , 1991 .

[57]  Jeremy Kepner,et al.  MatlabMPI , 2004, J. Parallel Distributed Comput..

[58]  D. Gillespie Approximate accelerated stochastic simulation of chemically reacting systems , 2001 .

[59]  B. Lind,et al.  A Monte Carlo program for the analysis of low-energy electron tracks in liquid water , 2011, Physics in medicine and biology.

[60]  I. Petsalakis,et al.  Electron inelastic mean free paths in biological matter based on dielectric theory and local-field corrections , 2009 .

[61]  Shingo Matsumoto,et al.  Reconstruction for Time-Domain In Vivo EPR 3D Multigradient Oximetric Imaging—A Parallel Processing Perspective , 2009, Int. J. Biomed. Imaging.

[62]  M. Inokuti,et al.  The Bethe surface of liquid water , 1998, Radiation and environmental biophysics.

[63]  Amitava Majumdar Parallel performance study of Monte Carlo photon transport code on shared-, distributed-, and distributed-shared-memory architectures , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[64]  Dragan Mirkovic,et al.  A GPU implementation of a track-repeating algorithm for proton radiotherapy dose calculations , 2010, Physics in medicine and biology.

[65]  Trevor Mudge,et al.  Monte Carlo Photon Transport On Shared Memory and Distributed Memory Parallel Processors , 1987 .

[66]  John D. Owens,et al.  General Purpose Computation on Graphics Hardware , 2005, IEEE Visualization.

[67]  George Karypis,et al.  Introduction to Parallel Computing Solution Manual , 2003 .

[68]  I J Das,et al.  Gold microspheres: a selective technique for producing biologically effective dose enhancement. , 2000, International journal of radiation biology.

[69]  Ruichao Ren,et al.  Parallel Markov chain Monte Carlo simulations. , 2007, The Journal of chemical physics.

[70]  B. Lind,et al.  Limitations (and merits) of PENELOPE as a track-structure code , 2012, International journal of radiation biology.

[71]  K. Karava,et al.  Monte Carlo simulation of the energy loss of low-energy electrons in liquid water. , 2003, Physics in medicine and biology.

[72]  Yusa Muroya,et al.  Radiolysis of liquid water: An attempt to reconcile Monte-Carlo calculations with new experimental hydrated electron yield data at early times , 2002 .

[73]  Sadaf R. Alam,et al.  Characterization of Scientific Workloads on Systems with Multi-Core Processors , 2006, 2006 IEEE International Symposium on Workload Characterization.

[74]  Amith R. Mamidala,et al.  MPI Collectives on Modern Multicore Clusters: Performance Optimizations and Communication Characteristics , 2008, 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID).

[75]  Georgios Kalantzis,et al.  Hybrid stochastic simulations of intracellular reaction-diffusion systems , 2009, Comput. Biol. Chem..

[76]  Bin Ma,et al.  Qualitative Simulation of Photon Transport in Free Space Based on Monte Carlo Method and Its Parallel Implementation , 2010, Int. J. Biomed. Imaging.

[77]  M. Brechbiel,et al.  Development of radioimmunotherapeutic and diagnostic antibodies: an inside-out view. , 2007, Nuclear medicine and biology.

[78]  Radek Erban,et al.  STOCHSIMGPU: parallel stochastic simulation for the Systems Biology Toolbox 2 for MATLAB , 2011, Bioinform..

[79]  M. Terrissol Modelling of radiation damage by 125I on a nucleosome. , 1994, International journal of radiation biology.

[80]  Barry Halliwell,et al.  DNA damage by oxygen‐derived species Its mechanism and measurement in mammalian systems , 1991, FEBS letters.

[81]  R. H. Ritchie,et al.  Comparisons of Calculations with PARTRAC and NOREC: Transport of Electrons in Liquid Water , 2008, Radiation research.

[82]  Xun Jia,et al.  A GPU-based finite-size pencil beam algorithm with 3D-density correction for radiotherapy dose calculation. , 2011, Physics in medicine and biology.

[83]  Lei Xing,et al.  GPU computing in medical physics: a review. , 2011, Medical physics.

[84]  M G Stabin,et al.  Monte Carlo simulation of diffusion and reaction in water radiolysis – a study of reactant `jump through' and jump distances , 1998, Radiation and environmental biophysics.

[85]  D. Goodhead,et al.  Comparison and assessment of electron cross sections for Monte Carlo track structure codes. , 1999, Radiation research.

[87]  D. Emfietzoglou,et al.  A combined molecular dynamics and Monte Carlo simulation of the spatial distribution of energy deposition by proton beams in liquid water , 2011, Physics in medicine and biology.

[88]  Roger W Howell,et al.  Log normal distribution of cellular uptake of radioactivity: implications for biologic responses to radiopharmaceuticals. , 2006, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[89]  G. Bruce Berriman,et al.  Scientific workflow applications on Amazon EC2 , 2010, 2009 5th IEEE International Conference on E-Science Workshops.

[90]  J. Turner,et al.  Monte Carlo Simulations of Site-Specific Radical Attack to DNA Bases , 2008, Radiation research.

[91]  J. Robar,et al.  Generation and modelling of megavoltage photon beams for contrast-enhanced radiation therapy , 2006, Physics in medicine and biology.

[92]  P. Otto,et al.  Radiobiological characterization of post-lumpectomy focal brachytherapy with lipid nanoparticle-carried radionuclides , 2011, Physics in medicine and biology.

[93]  A. Krishnamurthy,et al.  Developing a Computational Science IDE for HPC Systems , 2007, Third International Workshop on Software Engineering for High Performance Computing Applications (SE-HPC '07).

[94]  Yuni K. Dewaraja,et al.  A parallel Monte Carlo code for planar and SPECT imaging: implementation, verification and applications in 131I SPECT , 2002, Comput. Methods Programs Biomed..

[95]  Guido Germano,et al.  Efficiency of linked cell algorithms , 2010, Comput. Phys. Commun..

[96]  R. Stolarski,et al.  Analytic models of electron impact excitation cross sections , 1972 .

[97]  M. King,et al.  Accelerated SPECT Monte Carlo Simulation Using Multiple Projection Sampling and Convolution-Based Forced Detection , 2006, IEEE Transactions on Nuclear Science.

[98]  T. Mandal,et al.  Engineered nanoparticles in cancer therapy. , 2007, Recent patents on drug delivery & formulation.