Towards blood flow in the virtual human: efficient self-coupling of HemeLB

Many scientific and medical researchers are working towards the creation of a virtual human—a personalized digital copy of an individual—that will assist in a patient’s diagnosis, treatment and recovery. The complex nature of living systems means that the development of this remains a major challenge. We describe progress in enabling the HemeLB lattice Boltzmann code to simulate 3D macroscopic blood flow on a full human scale. Significant developments in memory management and load balancing allow near linear scaling performance of the code on hundreds of thousands of computer cores. Integral to the construction of a virtual human, we also outline the implementation of a self-coupling strategy for HemeLB. This allows simultaneous simulation of arterial and venous vascular trees based on human-specific geometries.

[1]  Dhabaleswar K. Panda,et al.  HAND: A Hybrid Approach to Accelerate Non-contiguous Data Movement Using MPI Datatypes on GPU Clusters , 2014, 2014 43rd International Conference on Parallel Processing.

[2]  Jean-Frédéric Gerbeau,et al.  Kinetic scheme for arterial and venous blood flow, and application to partial hepatectomy modeling. , 2017 .

[3]  P. Coveney,et al.  Modeling Patient-Specific Magnetic Drug Targeting Within the Intracranial Vasculature , 2018, Front. Physiol..

[4]  Peter V. Coveney,et al.  Multiscale computing for science and engineering in the era of exascale performance , 2019, Philosophical Transactions of the Royal Society A.

[5]  Andrea Tagliasacchi,et al.  Mean Curvature Skeletons , 2012, Comput. Graph. Forum.

[6]  K. C. Watts,et al.  Computational simulation of blood flow in human systemic circulation incorporating an external force field , 2006, Medical and Biological Engineering and Computing.

[7]  Niels Kuster,et al.  The Virtual Family—development of surface-based anatomical models of two adults and two children for dosimetric simulations , 2010, Physics in medicine and biology.

[8]  Derek Groen,et al.  Impact of blood rheology on wall shear stress in a model of the middle cerebral artery , 2012, Interface Focus.

[9]  Martin Jones,et al.  Computer simulations reveal complex distribution of haemodynamic forces in a mouse retina model of angiogenesis , 2013, Journal of The Royal Society Interface.

[10]  Mette S Olufsen,et al.  Structured tree outflow condition for blood flow in larger systemic arteries. , 1999, American journal of physiology. Heart and circulatory physiology.

[11]  Fabian Klemens,et al.  OpenLB - Open source lattice Boltzmann code , 2020, Comput. Math. Appl..

[12]  Ümit V. Çatalyürek,et al.  The Zoltan and Isorropia parallel toolkits for combinatorial scientific computing: Partitioning, ordering and coloring , 2012, Sci. Program..

[13]  D. Noble,et al.  Systems biology and the virtual physiological human , 2009, Molecular systems biology.

[14]  Alfio Quarteroni,et al.  A vision and strategy for the virtual physiological human in 2010 and beyond , 2010, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[15]  Peter V. Coveney,et al.  Validation of Patient-Specific Cerebral Blood Flow Simulation Using Transcranial Doppler Measurements , 2018, Front. Physiol..

[16]  Erik W. Draeger,et al.  Massively parallel simulations of hemodynamics in the primary large arteries of the human vasculature , 2015, J. Comput. Sci..

[17]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[18]  Derek Groen,et al.  Choice of boundary condition for lattice-Boltzmann simulation of moderate-Reynolds-number flow in complex domains. , 2012, Physical review. E, Statistical, nonlinear, and soft matter physics.

[19]  Bernd Mohr,et al.  The Scalasca performance toolset architecture , 2010, Concurr. Comput. Pract. Exp..

[20]  J. C. Cajas,et al.  Alya Red CCM: HPC-Based Cardiac Computational Modelling , 2015 .

[21]  Pierre Alliez,et al.  CGAL - The Computational Geometry Algorithms Library , 2011 .

[22]  Peter V. Coveney,et al.  Analysing and modelling the performance of the HemeLB lattice-Boltzmann simulation environment , 2012, J. Comput. Sci..

[23]  Yong Sook Lee,et al.  Visible Korean Human: Improved serially sectioned images of the entire body , 2005, IEEE Transactions on Medical Imaging.

[24]  Thomas Desaive,et al.  Virtual patients and virtual cohorts: a new way to think about the design and implementation of personalized ICU treatments , 2016 .

[25]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[26]  Niels Kuster,et al.  Development of a new generation of high-resolution anatomical models for medical device evaluation: the Virtual Population 3.0 , 2014, Physics in medicine and biology.

[27]  Mette S Olufsen,et al.  Numerical simulation of blood flow and pressure drop in the pulmonary arterial and venous circulation , 2014, Biomechanics and modeling in mechanobiology.

[28]  Peter V. Coveney,et al.  HemeLB: A high performance parallel lattice-Boltzmann code for large scale fluid flow in complex geometries , 2008, Comput. Phys. Commun..

[29]  B. Shi,et al.  Discrete lattice effects on the forcing term in the lattice Boltzmann method. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Robert Latham,et al.  Extending the MPI-2 Generalized Request Interface , 2007, PVM/MPI.

[31]  Alfio Quarteroni,et al.  A vision and strategy for the virtual physiological human: 2012 update , 2013, Interface Focus.

[32]  Robert Latham,et al.  To INT_MAX... and Beyond! Exploring Large-Count Support in MPI , 2014, 2014 Workshop on Exascale MPI at Supercomputing Conference.