Multiscale Simulation of Cardiovascular flows on the IBM Bluegene/P: Full Heart-Circulation System at Red-Blood Cell Resolution

We present the first large-scale simulation of blood flow in the coronary artieries and other vessels supplying blood to the heart muscle, with a realistic description of human arterial geometry at spatial resolutions from centimeters down to 10 microns (near the size of red blood cells). This multiscale simulation resolves the fluid into a billion volume units, embedded in a bounding space of 300 billion voxels, coupled with the concurrent motion of 300 million red blood cells, which interact with one another and with the surrounding fluid. The level of detail is sufficient to describe phenomena of potential physiological and clinical significance, such as the development of atherosclerotic plaques. The simulation achieves excellent scalability on up to 294, 912 Blue Gene/P computational cores.

[1]  Thomas Zeiser,et al.  Performance evaluation of a parallel sparse lattice Boltzmann solver , 2008, J. Comput. Phys..

[2]  M. Dupin,et al.  Modeling the flow of dense suspensions of deformable particles in three dimensions. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  J. Boon.  The Lattice Boltzmann Equation for Fluid Dynamics and Beyond , 2003 .

[4]  C. Ross Ethier,et al.  Computational modeling of arterial biomechanics , 2001, Comput. Sci. Eng..

[5]  R. Benzi,et al.  The lattice Boltzmann equation: theory and applications , 1992 .

[6]  Massimo Bernaschi,et al.  MUPHY: A parallel MUlti PHYsics/scale code for high performance bio-fluidic simulations , 2009, Comput. Phys. Commun..

[7]  Ulrich Rüde,et al.  Coupling multibody dynamics and computational fluid dynamics on 8192 processor cores , 2010, Parallel Comput..

[8]  J. McWhirter,et al.  Flow-induced clustering and alignment of vesicles and red blood cells in microcapillaries , 2009, Proceedings of the National Academy of Sciences.

[9]  George Biros,et al.  A numerical method for simulating the dynamics of 3D axisymmetric vesicles suspended in viscous flows , 2009, J. Comput. Phys..

[10]  Q. Zou,et al.  On pressure and velocity boundary conditions for the lattice Boltzmann BGK model , 1995, comp-gas/9611001.

[11]  George Em Karniadakis,et al.  Simulation of the human intracranial arterial tree. , 2009, Philosophical transactions. Series A, Mathematical, physical, and engineering sciences.

[12]  George Em Karniadakis,et al.  Accurate coarse-grained modeling of red blood cells. , 2008, Physical review letters.

[13]  B. Leimkuhler,et al.  Symplectic splitting methods for rigid body molecular dynamics , 1997 .

[14]  Jan Westerholm,et al.  An efficient swap algorithm for the lattice Boltzmann method , 2007, Comput. Phys. Commun..

[15]  Alfio Quarteroni,et al.  Mathematical and Numerical Modeling of Solute Dynamics in Blood Flow and Arterial Walls , 2001, SIAM J. Numer. Anal..

[16]  George L.-T. Chiu,et al.  Overview of the Blue Gene/L system architecture , 2005, IBM J. Res. Dev..

[17]  Peter V. Coveney,et al.  HemeLB: A high performance parallel lattice-Boltzmann code for large scale fluid flow in complex geometries , 2008, Comput. Phys. Commun..

[18]  Cyrus K. Aidun,et al.  Parallel performance of a lattice-Boltzmann/finite element cellular blood flow solver on the IBM Blue Gene/P architecture , 2010, Comput. Phys. Commun..

[19]  B. Berne.  Modification of the overlap potential to mimic a linear site-site potential , 1981 .

[20]  G. Karniadakis,et al.  Blood flow velocity effects and role of activation delay time on growth and form of platelet thrombi , 2006, Proceedings of the National Academy of Sciences.

[21]  R H Smallwood,et al.  The application of multiscale modelling to the process of development and prevention of stenosis in a stented coronary artery , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  Laxmikant V. Kalé,et al.  NAMD: Biomolecular Simulation on Thousands of Processors , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[23]  François Pellegrini,et al.  PT-Scotch: A tool for efficient parallel graph ordering , 2008, Parallel Comput..

[24]  Massimo Bernaschi,et al.  A flexible high‐performance Lattice Boltzmann GPU code for the simulations of fluid flows in complex geometries , 2010, Concurr. Comput. Pract. Exp..

[25]  B. Chopard,et al.  Lattice Boltzmann Simulations of Blood Flow: Non-Newtonian Rheology and Clotting Processes , 2005 .

[26]  Massimo Bernaschi,et al.  Hydrokinetic approach to large-scale cardiovascular blood flow , 2010, Comput. Phys. Commun..