Towards realistic HPC models of the neuromuscular system

Realistic simulations of detailed, biophysics-based, multi-scale models require very high resolution and, thus, large-scale compute facilities. Existing simulation environments, especially for biomedical applications, are designed to allow for a high flexibility and generality in model development. Flexibility and model development, however, are often a limiting factor for large-scale simulations. Therefore, new models are typically tested and run on small-scale compute facilities. By using a detailed biophysics-based, chemo-electromechanical skeletal muscle model and the international open-source software library OpenCMISS as an example, we present an approach to upgrade an existing muscle simulation framework from a moderately parallel version towards a massively parallel one that scales both in terms of problem size and in terms of the number of parallel processes. For this purpose, we investigate different modeling, algorithmic and implementational aspects. We present improvements addressing both numerical and parallel scalability. In addition, our approach includes a novel visualization environment, which is based on the MegaMol environment capable of handling large amounts of simulated data. It offers a platform for fast visualization prototyping, distributed rendering, and advanced visualization techniques. We present results of a variety of scaling studies at the Tier-1 supercomputer HazelHen at the High Performance Computing Center Stuttgart (HLRS). We improve the overall runtime by a factor of up to 2.6 and achieved good scalability on up to 768 cores, where the previous implementation used only 4 cores.

[1]  Oliver Röhrle,et al.  Modeling the Chemoelectromechanical Behavior of Skeletal Muscle Using the Parallel Open-Source Software Library OpenCMISS , 2013, Comput. Math. Methods Medicine.

[2]  D. Winter,et al.  Models of recruitment and rate coding organization in motor-unit pools. , 1993, Journal of neurophysiology.

[3]  Andrew J. Pullan,et al.  Mathematically Modelling the Electrical Activity of the Heart: From Cell to Body Surface and Back Again , 2005 .

[4]  John B. Davidson,et al.  A mathematical model of fatigue in skeletal muscle force contraction , 2007, Journal of Muscle Research and Cell Motility.

[5]  S. Delp,et al.  A 3D model of muscle reveals the causes of nonuniform strains in the biceps brachii. , 2005, Journal of biomechanics.

[6]  Onkar Sahni,et al.  Controlling Unstructured Mesh Partitions for Massively Parallel Simulations , 2010, SIAM J. Sci. Comput..

[7]  D. Farina,et al.  Decorrelation of cortical inputs and motoneuron output. , 2011, Journal of neurophysiology.

[8]  M. Karhanek,et al.  An allosteric model of the molecular interactions of excitation- contraction coupling in skeletal muscle , 1993, The Journal of general physiology.

[9]  John B. Davidson,et al.  Bridging Scales: A Three-Dimensional Electromechanical Finite Element Model of Skeletal Muscle , 2008, SIAM J. Sci. Comput..

[10]  R. Blickhan,et al.  A finite-element model for the mechanical analysis of skeletal muscles. , 2000, Journal of theoretical biology.

[11]  A. Pullan,et al.  Three-dimensional finite element modelling of muscle forces during mastication. , 2007, Journal of biomechanics.

[12]  A. Huxley,et al.  The variation in isometric tension with sarcomere length in vertebrate muscle fibres , 1966, The Journal of physiology.

[13]  William Gropp,et al.  Efficient Management of Parallelism in Object-Oriented Numerical Software Libraries , 1997, SciTools.

[14]  F. Zajac Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. , 1989, Critical reviews in biomedical engineering.

[15]  G. Golub,et al.  Gmres: a Generalized Minimum Residual Algorithm for Solving , 2022 .

[16]  K B Campbell,et al.  Stiffness-distortion sarcomere model for muscle simulation. , 1999, Journal of applied physiology.

[17]  B Calvo,et al.  A 3D electro-mechanical continuum model for simulating skeletal muscle contraction. , 2013, Journal of theoretical biology.

[18]  Viatcheslav Gurev,et al.  A high-resolution computational model of the deforming human heart , 2015, Biomechanics and modeling in mechanobiology.

[19]  Thomas Ertl,et al.  MegaMol—A Prototyping Framework for Particle-Based Visualization , 2015, IEEE Transactions on Visualization and Computer Graphics.

[20]  Oliver Röhrle,et al.  POD-DEIM reduction of computational EMG models , 2017, J. Comput. Sci..

[21]  Peter Lindstrom,et al.  Fixed-Rate Compressed Floating-Point Arrays , 2014, IEEE Transactions on Visualization and Computer Graphics.

[22]  N. Dubin Mathematical Model , 2022 .

[23]  Dario Farina,et al.  An analytical model for surface EMG generation in volume conductors with smooth conductivity variations , 2006, IEEE Transactions on Biomedical Engineering.

[24]  Oliver Röhrle,et al.  A multi-scale continuum model of skeletal muscle mechanics predicting force enhancement based on actin–titin interaction , 2016, Biomechanics and modeling in mechanobiology.

[25]  Todd A. Kuiken,et al.  A multiple-layer finite-element model of the surface EMG signal , 2002, IEEE Transactions on Biomedical Engineering.

[26]  M D Binder,et al.  Computer simulation of the steady-state input-output function of the cat medial gastrocnemius motoneuron pool. , 1991, Journal of neurophysiology.

[27]  Patrick Amestoy,et al.  A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling , 2001, SIAM J. Matrix Anal. Appl..

[28]  W Herzog,et al.  Length dependence of active force production in skeletal muscle. , 1999, Journal of applied physiology.

[29]  Peter J. Hunter,et al.  FieldML: concepts and implementation , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[30]  Stefan Schamberger,et al.  Partitioning finite element meshes using space-filling curves , 2005, Future Gener. Comput. Syst..

[31]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[32]  Rogério Rodrigues Lima Cisi,et al.  Simulation system of spinal cord motor nuclei and associated nerves and muscles, in a Web-based architecture , 2008, Journal of Computational Neuroscience.

[33]  Patrick Amestoy,et al.  Hybrid scheduling for the parallel solution of linear systems , 2006, Parallel Comput..

[34]  Catherine M Lloyd,et al.  CellML: its future, present and past. , 2004, Progress in biophysics and molecular biology.

[35]  Markus Böl,et al.  Micromechanical modelling of skeletal muscles based on the finite element method , 2008, Computer Methods in Biomechanics and Biomedical Engineering.

[36]  Heye Zhang,et al.  OpenCMISS: a multi-physics & multi-scale computational infrastructure for the VPH/Physiome project. , 2011, Progress in biophysics and molecular biology.

[37]  Oliver Röhrle,et al.  A multiscale chemo-electro-mechanical skeletal muscle model to analyze muscle contraction and force generation for different muscle fiber arrangements , 2014, Front. Physiol..

[38]  Aaron Knoll,et al.  OSPRay - A CPU Ray Tracing Framework for Scientific Visualization , 2017, IEEE Transactions on Visualization and Computer Graphics.

[39]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[40]  Dario Farina,et al.  A novel approach for precise simulation of the EMG signal detected by surface electrodes , 2001, IEEE Trans. Biomed. Eng..

[41]  Dario Farina,et al.  An integrated model of the neuromuscular system , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[42]  Michele Harms,et al.  Skeletal Muscle — Form and Function , 1997 .

[43]  Thomas Klotz,et al.  A continuum-mechanical skeletal muscle model including actin-titin interaction predicts stable contractions on the descending limb of the force-length relation , 2017, PLoS Comput. Biol..

[44]  N. Dimitrova,et al.  Precise and fast calculation of the motor unit potentials detected by a point and rectangular plate electrode. , 1998, Medical engineering & physics.

[45]  J. B. Davidson,et al.  A Physiologically Based, Multi-Scale Model of Skeletal Muscle Structure and Function , 2012, Front. Physio..

[46]  Gary L. Miller,et al.  Automatic Mesh Partitioning , 1992 .

[47]  Fan Zhang,et al.  In Situ Processing , 2012, High Performance Visualization.

[48]  D Hawkins,et al.  A comprehensive approach for studying muscle-tendon mechanics. , 1994, Journal of biomechanical engineering.

[49]  Oliver Röhrle,et al.  Predicting electromyographic signals under realistic conditions using a multiscale chemo–electro–mechanical finite element model , 2015, Interface Focus.