Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models

This paper studies the numerical computation of integrals, representing estimates or predictions, over the output $f(x)$ of a computational model with respect to a distribution $p(\mathrm{d}x)$ over uncertain inputs $x$ to the model. For the functional cardiac models that motivate this work, neither $f$ nor $p$ possess a closed-form expression and evaluation of either requires $\approx$ 100 CPU hours, precluding standard numerical integration methods. Our proposal is to treat integration as an estimation problem, with a joint model for both the a priori unknown function $f$ and the a priori unknown distribution $p$. The result is a posterior distribution over the integral that explicitly accounts for dual sources of numerical approximation error due to a severely limited computational budget. This construction is applied to account, in a statistically principled manner, for the impact of numerical errors that (at present) are confounding factors in functional cardiac model assessment.

[1]  Patrick R. Conrad,et al.  Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations , 2014, 1402.1694.

[2]  David Duvenaud,et al.  Optimally-Weighted Herding is Bayesian Quadrature , 2012, UAI.

[3]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[4]  Arthur Gretton,et al.  Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families , 2015, NIPS.

[5]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[6]  Francis R. Bach,et al.  On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions , 2015, J. Mach. Learn. Res..

[7]  V. Pawlowsky-Glahn,et al.  Bayes Hilbert Spaces , 2014 .

[8]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[9]  M. Stone The Generalized Weierstrass Approximation Theorem , 1948 .

[10]  E. Novak,et al.  Tractability of Multivariate Problems , 2008 .

[11]  Gary R. Mirams,et al.  Uncertainty and variability in computational and mathematical models of cardiac physiology , 2016, The Journal of physiology.

[12]  Kenji Fukumizu,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2010, J. Mach. Learn. Res..

[13]  Simo Särkkä,et al.  Fully symmetric kernel quadrature , 2017, SIAM J. Sci. Comput..

[14]  Samuel N. Cohen Data-driven nonlinear expectations for statistical uncertainty in decisions , 2016, 1609.06545.

[15]  Frances Y. Kuo,et al.  High-dimensional integration: The quasi-Monte Carlo way*† , 2013, Acta Numerica.

[16]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, IFIP Working Conference on Database Semantics.

[17]  Robert B. Gramacy,et al.  Adaptive Design and Analysis of Supercomputer Experiments , 2008, Technometrics.

[18]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[19]  J. Sethuraman A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[20]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[21]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[22]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions for non-i.i.d. observations , 2007, 0708.0491.

[23]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[24]  Kenji Fukumizu,et al.  Convergence guarantees for kernel-based quadrature rules in misspecified settings , 2016, NIPS.

[25]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[26]  Costas Papadimitriou,et al.  Bayesian uncertainty quantification and propagation in molecular dynamics simulations: a high performance computing framework. , 2012, The Journal of chemical physics.

[27]  Francis R. Bach,et al.  On the Equivalence between Quadrature Rules and Random Features , 2015, ArXiv.

[28]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[29]  F. Pillichshammer,et al.  Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration , 2010 .

[30]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[31]  B. Delyon,et al.  Integral approximation by kernel smoothing , 2014, 1409.0733.

[32]  John P. Cunningham,et al.  Bayesian Learning of Kernel Embeddings , 2016, UAI.

[33]  P. McCullagh,et al.  A theory of statistical models for Monte Carlo integration , 2003 .

[34]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[35]  E. Novak,et al.  Tractability of Multivariate Problems Volume II: Standard Information for Functionals , 2010 .

[36]  H. Muller,et al.  Functional data analysis for density functions by transformation to a Hilbert space , 2016, 1601.02869.

[37]  D. Blackwell Conditional Expectation and Unbiased Sequential Estimation , 1947 .

[38]  Carl E. Rasmussen,et al.  Bayesian Monte Carlo , 2002, NIPS.

[39]  B. Goodwin Oscillatory behavior in enzymatic control processes. , 1965, Advances in enzyme regulation.

[40]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[41]  Michael Goldstein,et al.  Bayesian Forecasting for Complex Systems Using Computer Simulators , 2001 .

[42]  Michael A. Osborne,et al.  Probabilistic Integration: A Role for Statisticians in Numerical Analysis? , 2015 .

[43]  S. Gupta,et al.  Statistical decision theory and related topics IV , 1988 .

[44]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[45]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[46]  Harald Bergstriim Mathematical Theory of Probability and Statistics , 1966 .

[47]  Michael V. Truong,et al.  Biophysical Modeling to Determine the Optimization of Left Ventricular Pacing Site and AV/VV Delays in the Acute and Chronic Phase of Cardiac Resynchronization Therapy , 2017, Journal of cardiovascular electrophysiology.

[48]  Anthony O'Hagan,et al.  Monte Carlo is fundamentally unsound , 1987 .

[49]  Milan Lukić,et al.  Stochastic processes with sample paths in reproducing kernel Hilbert spaces , 2001 .

[50]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[51]  T. Ferguson BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS , 1983 .

[52]  Michael A. Osborne,et al.  Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees , 2015, NIPS.

[53]  Roman Garnett,et al.  Bayesian Quadrature for Ratios , 2012, AISTATS.

[54]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[55]  H. Ishwaran,et al.  Exact and approximate sum representations for the Dirichlet process , 2002 .

[56]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[57]  Mark Girolami,et al.  The Controlled Thermodynamic Integral for Bayesian Model Evidence Evaluation , 2016 .

[58]  Ryan P. Adams,et al.  The Gaussian Process Density Sampler , 2008, NIPS.

[59]  J. Skilling Bayesian Solution of Ordinary Differential Equations , 1992 .

[60]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[61]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[62]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[63]  Mark A. Girolami,et al.  On the Sampling Problem for Kernel Quadrature , 2017, ICML.

[64]  Mark A. Girolami,et al.  Bayesian Probabilistic Numerical Methods , 2017, SIAM Rev..