Bayesian Probabilistic Numerical Methods

Over forty years ago average-case error was proposed in the applied mathematics literature as an alternative criterion with which to assess numerical methods. In contrast to worst-case error, this ...

[1]  Tomasz Dyakowski,et al.  Using electrical impedance tomography for controlling hydrocyclone underflow discharge , 1995 .

[2]  A. Beskos,et al.  On the stability of sequential Monte Carlo methods in high dimensions , 2011, 1103.3965.

[3]  Zoubin Ghahramani,et al.  Practical probabilistic programming with monads , 2015, Haskell.

[4]  Van Der Vaart,et al.  The Bernstein-Von-Mises theorem under misspecification , 2012 .

[5]  E. T. Jaynes,et al.  [Optimal Information Processing and Bayes's Theorem]: Comment , 1988 .

[6]  Ben Calderhead,et al.  Probabilistic Linear Multistep Methods , 2016, NIPS.

[7]  Sudipto Chakraborty,et al.  Studies on the performance of a hydrocyclone and modeling for flow characterization in presence and absence of air core , 2007 .

[8]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[9]  F. M. Larkin.  Optimal approximation in Hilbert spaces with reproducing kernel functions , 1970 .

[10]  Marcos A. Capistrán,et al.  Bayesian Analysis of ODEs: Solver Optimal Accuracy and Bayes Factors , 2013, SIAM/ASA J. Uncertain. Quantification.

[11]  Xiao-Li Meng,et al.  FURTHER EXPLORATIONS OF LIKELIHOOD THEORY FOR MONTE CARLO INTEGRATION , 2007 .

[12]  Andrew M. Stuart,et al.  Statistical analysis of differential equations: introducing probability measures on numerical solutions , 2016, Statistics and Computing.

[13]  D. Richards,et al.  Understanding uncertainty , 2012, Evidence-Based Dentistry.

[14]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[15]  A. M. Stuart,et al.  Sparse deterministic approximation of Bayesian inverse problems , 2011, 1103.4522.

[16]  Diana Verseghy,et al.  The reliability of single precision computations in the simulation of deep soil heat diffusion in a land surface model , 2016, Climate Dynamics.

[17]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[18]  F. Lindsten,et al.  Divide-and-Conquer With Sequential Monte Carlo , 2014, 1406.4993.

[19]  Andrew M. Stuart,et al.  Geometric MCMC for infinite-dimensional inverse problems , 2016, J. Comput. Phys..

[20]  S. Gupta,et al.  Statistical decision theory and related topics IV , 1988 .

[21]  G. Box,et al.  Scientific Inference, Data Analysis and Robustness. , 1985 .

[22]  Majnu John,et al.  Confidence intervals for finite difference solutions , 2017, Commun. Stat. Simul. Comput..

[23]  Christopher J. Roy,et al.  Verification and Validation in Scientific Computing , 2010 .

[24]  Roman Garnett,et al.  Bayesian Quadrature for Ratios , 2012, AISTATS.

[25]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[26]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[27]  Tiangang Cui,et al.  Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction , 2015, J. Comput. Phys..

[28]  P. McCullagh,et al.  A theory of statistical models for Monte Carlo integration , 2003 .

[29]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[30]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[31]  Philipp Hennig,et al.  Probabilistic Line Searches for Stochastic Optimization , 2015, NIPS.

[32]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[33]  Paris Perdikaris,et al.  Numerical Gaussian Processes for Time-Dependent and Nonlinear Partial Differential Equations , 2017, SIAM J. Sci. Comput..

[34]  P. Pfanzagl,et al.  CONDITIONAL DISTRIBUTIONS AS DERIVATIVES , 1979 .

[35]  Pierre Del Moral,et al.  Sequential Monte Carlo for rare event estimation , 2012, Stat. Comput..

[36]  J. Mockus.  Bayesian Approach to Global Optimization: Theory and Applications , 1989 .

[37]  Philipp Hennig,et al.  Active Uncertainty Calibration in Bayesian ODE Solvers , 2016, UAI.

[38]  Zhiqiang Tan,et al.  On a Likelihood Approach for Monte Carlo Integration , 2004 .

[39]  Ivo Babuska,et al.  On Roundoff Error Growth in Elliptic Problems , 2018, ACM Trans. Math. Softw..

[40]  R. Nickl,et al.  On the Bernstein–von Mises phenomenon for nonparametric Bayes procedures , 2013, 1310.2484.

[41]  A. Doucet,et al.  Sequential auxiliary particle belief propagation , 2005, 2005 7th International Conference on Information Fusion.

[42]  Jaroslav Tišer,et al.  Vitali covering theorem in Hilbert space , 2003 .

[43]  Houman Owhadi,et al.  Bayesian Numerical Homogenization , 2014, Multiscale Model. Simul..

[44]  Matthew M. Dunlop,et al.  The Bayesian Formulation of EIT: Analysis and Algorithms , 2015, 1508.04106.

[45]  Michael Horstein,et al.  Sequential transmission using noiseless feedback , 1963, IEEE Trans. Inf. Theory.

[46]  Philipp Hennig,et al.  Probabilistic Interpretation of Linear Solvers , 2014, SIAM J. Optim..

[47]  Houman Owhadi,et al.  Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games , 2015, SIAM Rev..

[48]  Martin Kiefel,et al.  Quasi-Newton Methods: A New Direction , 2012, ICML.

[49]  Vaja Tarieladze,et al.  Disintegration of Gaussian measures and average-case optimal algorithms , 2007, J. Complex..

[50]  David Lee,et al.  Approximation of linear functionals on a banach space with a Gaussian measure , 1986, J. Complex..

[51]  Mark Girolami,et al.  Probabilistic Integration and Intractable Distributions , 2016 .

[52]  Pier Giovanni Bissiri,et al.  A general framework for updating belief distributions , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[53]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[54]  C. Schwab,et al.  Scaling limits in computational Bayesian inversion , 2014 .

[55]  M. Girolami,et al.  Bayesian Solution Uncertainty Quantification for Differential Equations , 2013 .

[56]  Sebastian Mosbach,et al.  A quantitative probabilistic investigation into the accumulation of rounding errors in numerical ODE solution , 2009, Comput. Math. Appl..

[57]  J. Kadane.  Principles of Uncertainty , 2011 .

[58]  Simon W. Funke,et al.  Deflation Techniques for Finding Distinct Solutions of Nonlinear Partial Differential Equations , 2014, SIAM J. Sci. Comput..

[59]  Michèle Giry,et al.  A categorical approach to probability theory , 1982 .

[60]  Gunther Uhlmann,et al.  Electrical impedance tomography and Calderón's problem , 2009 .

[61]  David A. Freedman,et al.  Frequency Properties of Bayes Rules , 1983 .

[62]  A. Stuart,et al.  Besov priors for Bayesian inverse problems , 2011, 1105.0889.

[63]  Travis V. Anderson.  Efficient, Accurate, and Non-Gaussian Error Propagation Through Nonlinear, Closed-Form, Analytical System Models , 2011 .

[64]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[65]  Andreas Krause,et al.  Advances in Neural Information Processing Systems (NIPS) , 2014 .

[66]  Sophia Blau,et al.  Analysis Of The Finite Element Method , 2016 .

[67]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[68]  Mark Girolami,et al.  The Controlled Thermodynamic Integral for Bayesian Model Evidence Evaluation , 2016 .

[69]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[70]  Mark A. Girolami,et al.  Probabilistic Numerical Methods for PDE-constrained Bayesian Inverse Problems , 2017, ArXiv.

[71]  B. Kendall.  Nonlinear Dynamics and Chaos , 2001 .

[72]  Robert G. Aykroyd,et al.  Spatial-temporal modeling for electrical impedance imaging of a mixing process , 2005 .

[73]  Peter I. Frazier,et al.  Bisection Search with Noisy Responses , 2013, SIAM J. Control. Optim..

[74]  D. Isaacson,et al.  Electrode models for electric current computed tomography , 1989, IEEE Transactions on Biomedical Engineering.

[75]  T. J. Sullivan,et al.  Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors , 2016, 1605.05898.

[76]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[77]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[78]  Houman Owhadi,et al.  On the Brittleness of Bayesian Inference , 2013, SIAM Rev..

[79]  Arnaud Doucet,et al.  An adaptive sequential Monte Carlo method for approximate Bayesian computation , 2011, Statistics and Computing.

[80]  Mark A. Girolami,et al.  Probabilistic Meshless Methods for Partial Differential Equations and Bayesian Inverse Problems , 2016, ArXiv.

[81]  Judith Rousseau,et al.  Combining expert opinions in prior elicitation , 2010 .

[82]  Le Cam,et al.  On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates , 1953 .

[83]  Mark A. Girolami,et al.  Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models , 2016, NIPS.

[84]  Andrew M. Stuart,et al.  Approximation of Bayesian Inverse Problems for PDEs , 2009, SIAM J. Numer. Anal..

[85]  Isaac Siwale.  ON GLOBAL OPTIMIZATION , 2015 .

[86]  David Duvenaud,et al.  Probabilistic ODE Solvers with Runge-Kutta Means , 2014, NIPS.

[87]  J. Richard Swenson,et al.  Tests of probabilistic models for propagation of roundoff errors , 1966, CACM.

[88]  Simo Särkkä,et al.  Fully symmetric kernel quadrature , 2017, SIAM J. Sci. Comput..

[89]  William Kahan,et al.  The improbability of probabilistic error analyses for numerical computations , 1996 .

[90]  F. M. Larkin,et al.  Weak probability distributions on reproducing kernel Hilbert spaces , 1972 .

[91]  F. M. Larkin.  Probabilistic Error Estimates in Spline Interpolation and Quadrature , 1974, IFIP Congress.

[92]  Norman Ramsey,et al.  Exact Bayesian inference by symbolic disintegration , 2017, POPL.

[93]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[94]  Mark A. Girolami,et al.  A Bayesian approach to multiscale inverse problems with on-the-fly scale determination , 2016, J. Comput. Phys..

[95]  Simo Särkkä,et al.  A probabilistic model for the numerical solution of initial value problems , 2016, Statistics and Computing.

[96]  F. M. Larkin.  A modification of the secant rule derived from a maximum likelihood principle , 1979 .

[97]  David A. McAllester,et al.  Particle Belief Propagation , 2009, AISTATS.

[98]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[99]  D. Freedman.  On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case , 1963 .

[100]  J. Skilling.  Bayesian Solution of Ordinary Differential Equations , 1992 .

[101]  Simon French,et al.  Aggregating expert judgement , 2011 .

[102]  G. Wahba,et al.  SPLINE FUNCTIONS AND STOCHASTIC PROCESSES. , 1969 .

[103]  Daniel M. Roy,et al.  On computability and disintegration , 2017, Math. Struct. Comput. Sci..

[104]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .

[105]  Yee Whye Teh,et al.  Expectation Particle Belief Propagation , 2015, NIPS.

[106]  A. Calderón,et al.  On an inverse boundary value problem , 2006 .

[107]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[108]  Miguel de Carvalho,et al.  On the Geometry of Bayesian Inference , 2017, Bayesian Analysis.

[109]  F. Pillichshammer,et al.  Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration , 2010 .

[110]  Frank D. Wood,et al.  Inference Networks for Sequential Monte Carlo in Graphical Models , 2016, ICML.

[111]  Lloyd N. Trefethen,et al.  Is Gauss Quadrature Better than Clenshaw-Curtis? , 2008, SIAM Rev..

[112]  E. Somersalo,et al.  Existence and uniqueness for electrode models for electric current computed tomography , 1992 .

[113]  Philipp Hennig,et al.  Probabilistic Approximate Least-Squares , 2016, AISTATS.

[114]  Christopher J. Roy,et al.  Review of Discretization Error Estimators in Scientific Computing , 2010 .

[115]  Michael A. Osborne,et al.  Probabilistic Integration: A Role for Statisticians in Numerical Analysis? , 2015 .

[116]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[117]  F. M. Larkin.  Estimation of a non-negative function , 1969 .

[118]  Michael I. Jordan.  Graphical Models , 1998 .

[119]  Andrew M. Stuart,et al.  Sequential Monte Carlo methods for Bayesian elliptic inverse problems , 2014, Stat. Comput..

[120]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[121]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[122]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[123]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[124]  Grzegorz W. Wasilkowski,et al.  Average Case ϵ-Complexity in Computer Science: A Bayesian View , 1983 .

[125]  Joseph T. Chang,et al.  Conditioning as disintegration , 1997 .

[126]  Thomas Augustin,et al.  Foundations of Probability , 2011, International Encyclopedia of Statistical Science.

[127]  H. Poincaré.  Calcul des Probabilités , 1912 .

[128]  Kenji Fukumizu,et al.  Convergence guarantees for kernel-based quadrature rules in misspecified settings , 2016, NIPS.

[129]  P. Diaconis.  Bayesian Numerical Analysis , 1988 .

[130]  Gregory E. Fasshauer,et al.  Solving differential equations with radial basis functions: multilevel methods and smoothing , 1999, Adv. Comput. Math..

[131]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[132]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[133]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[134]  Andrew M. Stuart,et al.  MAP estimators for piecewise continuous inversion , 2015, 1509.03136.

[135]  Anthony O'Hagan,et al.  Monte Carlo is fundamentally unsound , 1987 .

[136]  Carl E. Rasmussen,et al.  Bayesian Monte Carlo , 2002, NIPS.

[137]  Dirk P. Kroese,et al.  Efficient Monte Carlo simulation via the generalized splitting method , 2012, Stat. Comput..

[138]  A. Stuart,et al.  The Bayesian Approach to Inverse Problems , 2013, 1302.6989.

[139]  Nicolas Garcia Trillos,et al.  Gradient Flows: Applications to Classification, Image Denoising, and Riemannian MCMC , 2017 .

[140]  R. Tweedie,et al.  Exponential convergence of Langevin distributions and their discrete approximations , 1996 .

[141]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[142]  F. M. Larkin.  Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[143]  Roman Fric,et al.  A Categorical Approach to Probability Theory , 2010, Stud Logica.

[144]  Lawrence Mitchell,et al.  Simulating Human Cardiac Electrophysiology on Clinical Time-Scales , 2011, Front. Physio..

[145]  Paul A. Jenkins,et al.  Inference and rare event simulation for stopped Markov processes via reverse-time sequential Monte Carlo , 2016, Stat. Comput..

[146]  Johannes T. N. Krebs.  Consistency and asymptotic normality of stochastic Euler schemes for ordinary differential equations , 2016, 1609.06880.