Bayesian Uncertainty Quantification for Data-Driven Applications in Engineering and Life Sciences

The world is quickly moving towards using computer modeling and simulations in various branches of science and engineering. The computational methods found their way to biology, chemistry, physics, social sciences and many other fields. Simulations allow the researchers to study phenomena which can not be directly observed in the experiments. In order to be trustworthy, the simulations must go through a thorough process of data-driven validation. Often, an adjustment of the free parameters of the computational model is required for it to match the experimental observations. One of the possibilities to find the right model parameters is an optimization with respect to the quantities of interest which one wishes to reproduce in the simulation. The amount of information which can be extracted from this deterministic approach is, however, very limited. The parameters sensitivity or their mutual correlations, which are crucial for making robust predictions, are not available in this approach. The solution to this problem lies in the area of probability, specifically in the Bayesian methodology. The value of the Bayesian approach in the modern computational science can hardly be overestimated. It updates the prior beliefs about the model parameters, expressed in the form of a probability distribution, using the available experimental data. This approach takes into account all the uncertainties arising in the inference process: the experimental, the modeling and the simulation ones, thus providing a reliable result. Blending together the simulation and the experiment outcomes, it constructs a conditional distribution of the model parameters given the observational data. From this distribution, one can extract the uncertainty bounds, the correlations, the sensitivities etc. for each of the parameters, as well as for any quantity of interest which can be computed using the calibrated model. All these useful features, however, come with a price of an excessive computa-

[1]  Guillermo Rus-Carlborg,et al.  Approximate Bayesian Computation by Subset Simulation , 2014, SIAM J. Sci. Comput..

[2]  M. Vakilzadeh,et al.  Manifold Metropolis adjusted Langevin algorithm for high-dimensional Bayesian FE , 2014 .

[3]  Pascal Pernot,et al.  Calibration of forcefields for molecular simulation: Sequential design of computer experiments for building cost‐efficient kriging metamodels , 2014, J. Comput. Chem..

[4]  F. Feroz,et al.  MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics , 2008, 0809.3437.

[5]  Gerhard Gompper,et al.  White blood cell margination in microcirculation. , 2014, Soft matter.

[6]  Thomas L. Griffiths,et al.  Neural Implementation of Hierarchical Bayesian Inference by Importance Sampling , 2009, NIPS.

[7]  M Scott Shell,et al.  The relative entropy is fundamental to multiscale and inverse thermodynamic problems. , 2008, The Journal of chemical physics.

[8]  Chwee Teck Lim,et al.  Connections between single-cell biomechanics and human disease states: gastrointestinal cancer and malaria. , 2005, Acta biomaterialia.

[9]  Steve Plimpton,et al.  Fast parallel algorithms for short-range molecular dynamics , 1993 .

[10]  Christian P Robert,et al.  Lack of confidence in approximate Bayesian computation model choice , 2011, Proceedings of the National Academy of Sciences.

[11]  E. Evans Bending elastic modulus of red blood cell membrane derived from buckling instability in micropipet aspiration tests. , 1983, Biophysical journal.

[12]  George Em Karniadakis,et al.  A multiscale red blood cell model with accurate mechanics, rheology, and dynamics. , 2010, Biophysical journal.

[13]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[14]  J. A. Barker,et al.  Liquid argon: Monte carlo and molecular dynamics calculations , 1971 .

[15]  Pascal Pernot,et al.  Statistical approaches to forcefield calibration and prediction uncertainty in molecular simulation. , 2011, The Journal of chemical physics.

[16]  Kurt Kremer,et al.  Multiscale simulation of soft matter systems. , 2010, Faraday discussions.

[17]  J. Tenenbaum,et al.  Bayesian Special Section Learning Overhypotheses with Hierarchical Bayesian Models , 2022 .

[18]  Harald Kruggel-Emden,et al.  A study on tangential force laws applicable to the discrete element method (DEM) for materials with viscoelastic or plastic behavior , 2008 .

[19]  Brandon M. Turner,et al.  Journal of Mathematical Psychology a Tutorial on Approximate Bayesian Computation , 2022 .

[20]  Bo Qiu,et al.  Quantifying Uncertainty in Multiscale Heat Conduction Calculations , 2012 .

[21]  R. Wilkinson Approximate Bayesian computation (ABC) gives exact results under the assumption of model error , 2008, Statistical applications in genetics and molecular biology.

[22]  Costas Papadimitriou,et al.  Exploiting Task-Based Parallelism in Bayesian Uncertainty Quantification , 2015, Euro-Par.

[23]  Janet E. Jones On the determination of molecular fields. III.—From crystal measurements and kinetic theory data , 1924 .

[24]  Costas Papadimitriou,et al.  Fusing heterogeneous data for the calibration of molecular dynamics force fields using hierarchical Bayesian models. , 2016, The Journal of chemical physics.

[25]  E. Merrill,et al.  Influence of flow properties of blood upon viscosity-hematocrit relationships. , 1962, The Journal of clinical investigation.

[26]  J. Beck,et al.  Updating Models and Their Uncertainties. I: Bayesian Statistical Framework , 1998 .

[27]  P. Koumoutsakos,et al.  X-TMCMC: Adaptive kriging for Bayesian inverse modeling , 2015 .

[28]  Paul Marjoram,et al.  Markov chain Monte Carlo without likelihoods , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[30]  M. Beaumont,et al.  Coestimation of recombination, substitution and molecular adaptation rates by approximate Bayesian computation , 2013, Heredity.

[31]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[32]  M. Hobson,et al.  Efficient Bayesian inference for multimodal problems in cosmology , 2007, astro-ph/0701867.

[33]  Costas Papadimitriou,et al.  Bayesian uncertainty quantification and propagation in molecular dynamics simulations: a high performance computing framework. , 2012, The Journal of chemical physics.

[34]  Mark A. Girolami,et al.  BioBayes: A software package for Bayesian inference in systems biology , 2008, Bioinform..

[35]  L. Excoffier,et al.  Efficient Approximate Bayesian Computation Coupled With Markov Chain Monte Carlo Without Likelihood , 2009, Genetics.

[36]  V. Goder,et al.  Molecular mechanism of signal sequence orientation in the endoplasmic reticulum , 2003, The EMBO journal.

[37]  O. Cappé,et al.  Population Monte Carlo , 2004 .

[38]  Costas Papadimitriou,et al.  Π4U: A high performance computing framework for Bayesian uncertainty quantification of complex models , 2015, J. Comput. Phys..

[39]  F. Montel,et al.  Molecular dynamics comparative study of Lennard-Jones -6 and exponential -6 potentials: application to real simple fluids (viscosity and pressure). , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  D. Boal,et al.  Simulations of the erythrocyte cytoskeleton at large deformation. II. Micropipette aspiration. , 1998, Biophysical journal.

[41]  J. Beck,et al.  Bayesian Model Updating Using Hybrid Monte Carlo Simulation with Application to Structural Dynamic Models with Many Uncertain Parameters , 2009 .

[42]  S. E. Ahmed,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 2008, Technometrics.

[43]  D. Krige A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[44]  Magalie Faivre,et al.  Swinging of red blood cells under shear flow. , 2007, Physical review letters.

[45]  Radford M. Neal Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[46]  S Wu,et al.  A hierarchical Bayesian framework for force field selection in molecular dynamics simulations , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Subra Suresh,et al.  Lipid bilayer and cytoskeletal interactions in a red blood cell , 2013, Proceedings of the National Academy of Sciences.

[48]  M. Stein Statistical Interpolation of Spatial Data: Some Theory for Kriging , 1999 .

[49]  Richard G. Everitt,et al.  Likelihood-free estimation of model evidence , 2011 .

[50]  Andrea Rau,et al.  Reverse engineering gene regulatory networks using approximate Bayesian computation , 2012, Stat. Comput..

[51]  Connie Y. Wang,et al.  Structurally detailed coarse-grained model for Sec-facilitated co-translational protein translocation and membrane integration , 2017, PLoS Comput. Biol..

[52]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[53]  J. Tinsley Oden,et al.  Selection, calibration, and validation of coarse-grained models of atomistic systems , 2015 .

[54]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[55]  Michael H. Moys,et al.  Experimental study of oblique impacts with initial spin , 2006 .

[56]  A. Rambaut,et al.  BEAST: Bayesian evolutionary analysis by sampling trees , 2007, BMC Evolutionary Biology.

[57]  J. Koelman,et al.  Simulating microscopic hydrodynamic phenomena with dissipative particle dynamics , 1992 .

[58]  Michael D Lee,et al.  BayesSDT: Software for Bayesian inference with signal detection theory , 2008, Behavior research methods.

[59]  L. Katafygiotis,et al.  Tangential‐projection algorithm for manifold representation in unidentifiable model updating problems , 2002 .

[60]  J. Kestin,et al.  Equilibrium and transport properties of the noble gases and their mixtures at low density , 1984 .

[61]  F. RIZZI,et al.  Uncertainty Quantification in MD Simulations. Part I: Forward Propagation , 2012, Multiscale Model. Simul..

[62]  R. Tran-Son-Tay,et al.  Determination of red blood cell membrane viscosity from rheoscopic observations of tank-treading motion. , 1984, Biophysical journal.

[63]  Duncan Temple Lang,et al.  Programming With Models: Writing Statistical Algorithms for General Model Structures With NIMBLE , 2015, 1505.05093.

[64]  Mark Girolami,et al.  Statistical analysis of nonlinear dynamical systems using differential geometric sampling methods , 2011, Interface Focus.

[65]  M. Socol,et al.  Full dynamics of a red blood cell in shear flow , 2012, Proceedings of the National Academy of Sciences.

[66]  J. A. White,et al.  Lennard-Jones as a model for argon and test of extended renormalization group calculations , 1999 .

[67]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[68]  M. Kalos,et al.  Monte Carlo methods , 1986 .

[69]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[70]  Costas Papadimitriou,et al.  Data driven, predictive molecular dynamics for nanoscale flow simulations under uncertainty. , 2013, The journal of physical chemistry. B.

[71]  E. Merrill,et al.  Rheology of human blood, near and at zero flow. Effects of temperature and hematocrit level. , 1963, Biophysical journal.

[72]  T. Junne,et al.  Orientation of internal signal-anchor sequences at the Sec61 translocon. , 2012, Journal of molecular biology.

[73]  Costas Papadimitriou,et al.  Bayesian uncertainty quantification and propagation for discrete element simulations of granular materials , 2014 .

[74]  Gerhard Gompper,et al.  Predicting human blood viscosity in silico , 2011, Proceedings of the National Academy of Sciences.

[75]  Simon J. Godsill,et al.  An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo , 2007, Proceedings of the IEEE.

[76]  J. Ching,et al.  Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating, Model Class Selection, and Model Averaging , 2007 .

[77]  D. Nicholson,et al.  Monte Carlo grand canonical ensemble calculation in a gas-liquid transition region for 12-6 Argon , 1975 .

[78]  Jun Yu,et al.  Bugs for a Bayesian Analysis of Stochastic Volatility Models , 2000 .

[79]  Thomas M Fischer,et al.  Tank-tread frequency of the red cell membrane: dependence on the viscosity of the suspending medium. , 2007, Biophysical journal.

[80]  Steven J. Plimpton,et al.  Peridynamics with LAMMPS : a user guide. , 2008 .

[81]  S. V. Sciver Special Topics in Helium Cryogenics , 2012 .

[82]  Gordon Luikart,et al.  Comparative Evaluation of a New Effective Population Size Estimator Based on Approximate Bayesian Computation , 2004, Genetics.

[83]  M. Shiga,et al.  Rapid estimation of elastic constants by molecular dynamics simulation under constant stress , 2004 .

[84]  Dmitry A. Fedosov,et al.  Multiscale Modeling of Blood Flow and Soft Matter , 2010 .

[85]  Iason Papaioannou,et al.  Transitional Markov Chain Monte Carlo: Observations and Improvements , 2016 .

[86]  W. Gilks Markov Chain Monte Carlo , 2005 .

[87]  M. Beaumont Approximate Bayesian Computation in Evolution and Ecology , 2010 .

[88]  Lasting Contacts in Molecular Dynamics Simulations , 1998 .

[89]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[90]  Ronald L. Graham,et al.  Bounds on Multiprocessing Timing Anomalies , 1969, SIAM Journal of Applied Mathematics.

[91]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[92]  K. Walters Parameter estimation for an immortal model of colonic stem cell division using approximate Bayesian computation. , 2012, Journal of theoretical biology.

[93]  Costas Papadimitriou,et al.  Approximate Bayesian Computation for Granular and Molecular Dynamics Simulations , 2016, PASC.

[94]  Andrew D. Martin,et al.  MCMCpack: Markov chain Monte Carlo in R , 2011 .

[95]  A. Gelman,et al.  Stan , 2015 .

[96]  Jukka Corander,et al.  Approximate Bayesian Computation , 2013, PLoS Comput. Biol..

[97]  Perry de Valpine,et al.  An approximate Bayesian computation approach to parameter estimation in a stochastic stage-structured population model. , 2014, Ecology.

[98]  M. Feldman,et al.  Population growth of human Y chromosomes: a study of Y chromosome microsatellites. , 1999, Molecular biology and evolution.

[99]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[100]  P. Español,et al.  Statistical Mechanics of Dissipative Particle Dynamics. , 1995 .

[101]  J. Beck,et al.  Estimation of Small Failure Probabilities in High Dimensions by Subset Simulation , 2001 .

[102]  E. Evans,et al.  Molecular maps of red cell deformation: hidden elasticity and in situ connectivity. , 1994, Science.

[103]  T. Fischer On the energy dissipation in a tank-treading human red blood cell. , 1980, Biophysical journal.

[104]  Iason Papaioannou,et al.  Bayesian Updating with Structural Reliability Methods , 2015 .

[105]  P. B. Warren,et al.  DISSIPATIVE PARTICLE DYNAMICS : BRIDGING THE GAP BETWEEN ATOMISTIC AND MESOSCOPIC SIMULATION , 1997 .

[106]  S. Suresh,et al.  Spectrin-level modeling of the cytoskeleton and optical tweezers stretching of the erythrocyte. , 2005, Biophysical journal.

[107]  Benjamin D. Redelings,et al.  BAli-Phy: simultaneous Bayesian inference of alignment and phylogeny , 2006, Bioinform..

[108]  Costas Papadimitriou,et al.  Bayesian Annealed Sequential Importance Sampling: An Unbiased Version of Transitional Markov Chain Monte Carlo , 2018 .

[109]  H Schmid-Schönbein,et al.  The red cell as a fluid droplet: tank tread-like motion of the human erythrocyte membrane in shear flow. , 1978, Science.

[110]  George Em Karniadakis,et al.  Single-particle hydrodynamics in DPD: A new formulation , 2008 .

[111]  George Em Karniadakis,et al.  Accurate coarse-grained modeling of red blood cells. , 2008, Physical review letters.

[112]  N. S. Gingrich,et al.  The Diffraction of X-Rays by Argon in the Liquid, Vapor, and Critical Regions , 1942 .

[113]  Aneesur Rahman,et al.  Correlations in the Motion of Atoms in Liquid Argon , 1964 .

[114]  Nenad Bićanić,et al.  Discrete Element Methods , 2004 .