Load balancing of parallel cell-based blood flow simulations

Abstract The non-homogeneous distribution of computational costs is often challenging to handle in highly parallel applications. Using a methodology based on fractional overheads, we studied the fractional load imbalance overhead in a high-performance biofluid simulation aiming to accurately resolve blood flow on a cellular level. In general, the concentration of particles in such a suspension flow is not homogeneous. Usually, there is a depletion of cells close to walls, and a higher concentration towards the centre of the flow domain. We perform parallel simulations of such suspension flows. The emerging non-homogeneous cell distributions might lead to strong load imbalance, resulting in deterioration of the parallel performance. We formulate a model for the fractional load imbalance overhead, validate it by measuring this overhead in parallel lattice Boltzmann based cell-based blood flow simulations, and compare the arising load imbalance with other sources of overhead, in particular the communication overhead. We find a good agreement between the measurements and our load imbalance model. We also find that in our test cases, the communication overhead was higher than the load imbalance overhead. However, for larger systems, we expect load imbalance overhead to be dominant. Thus, efficient load balancing strategies should be developed.

[1]  G. Karniadakis,et al.  Systematic coarse-graining of spectrin-level red blood cell models. , 2010, Computer Methods in Applied Mechanics and Engineering.

[2]  C. Peskin The immersed boundary method , 2002, Acta Numerica.

[3]  L. Mountrakis,et al.  Transport of blood cells studied with fully resolved models , 2015 .

[4]  A G Hoekstra,et al.  Where do the platelets go? A simulation study of fully resolved blood flow through aneurysmal vessels , 2013, Interface Focus.

[5]  Alfons G. Hoekstra,et al.  Scaling of shear-induced diffusion and clustering in a blood-like suspension , 2016 .

[6]  George Em Karniadakis,et al.  Accurate coarse-grained modeling of red blood cells. , 2008, Physical review letters.

[7]  G. Karniadakis,et al.  Blood Flow and Cell‐Free Layer in Microvessels , 2010, Microcirculation.

[8]  J. Boon The Lattice Boltzmann Equation for Fluid Dynamics and Beyond , 2003 .

[9]  Oguz K. Baskurt,et al.  Handbook of hemorheology and hemodynamics , 2007 .

[10]  Thomas Zeiser,et al.  Performance evaluation of a parallel sparse lattice Boltzmann solver , 2008, J. Comput. Phys..

[11]  Hiroshi Noguchi,et al.  Multiscale modeling of blood flow: from single cells to blood rheology , 2014, Biomechanics and modeling in mechanobiology.

[12]  T. G. M. Ven,et al.  The flow of suspensions , 1985 .

[13]  Petia M. Vlahovska,et al.  Continuum- and particle-based modeling of shapes and dynamics of red blood cells in health and disease. , 2013, Soft matter.

[14]  George Em Karniadakis,et al.  Blood flow in small tubes: quantifying the transition to the non-continuum regime , 2013, Journal of Fluid Mechanics.

[15]  Cyrus K. Aidun,et al.  Parallel performance of a lattice-Boltzmann/finite element cellular blood flow solver on the IBM Blue Gene/P architecture , 2010, Comput. Phys. Commun..

[16]  Orestis Malaspinas,et al.  Parallel performance of an IB-LBM suspension simulation framework , 2015, J. Comput. Sci..