Symmetry exploits for Bayesian cubature methods

Bayesian cubature provides a flexible framework for numerical integration, in which a priori knowledge on the integrand can be encoded and exploited. This additional flexibility, compared to many classical cubature methods, comes at a computational cost which is cubic in the number of evaluations of the integrand. It has been recently observed that fully symmetric point sets can be exploited in order to reduce - in some cases substantially - the computational cost of the standard Bayesian cubature method. This work identifies several additional symmetry exploits within the Bayesian cubature framework. In particular, we go beyond earlier work in considering non-symmetric measures and, in addition to the standard Bayesian cubature method, present exploits for the Bayes-Sard cubature method and the multi-output Bayesian cubature method.

[1]  Ondrej Straka,et al.  Student-t process quadratures for filtering of non-linear systems with heavy-tailed noise , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[2]  Fred J. Hickernell,et al.  Fast automatic Bayesian cubature using lattice sampling , 2018, Statistics and Computing.

[3]  F. M. Larkin.  Probabilistic Error Estimates in Spline Interpolation and Quadrature , 1974, IFIP Congress.

[4]  Kenji Fukumizu,et al.  Convergence guarantees for kernel-based quadrature rules in misspecified settings , 2016, NIPS.

[5]  Alan Genz,et al.  Fully symmetric interpolatory rules for multiple integrals , 1986 .

[6]  Mark A. Girolami,et al.  Bayesian Quadrature for Multiple Related Integrals , 2018, ICML.

[7]  K. Ritter,et al.  Simple Cubature Formulas with High Polynomial Exactness , 1999 .

[8]  Lester W. Mackey,et al.  Stein Points , 2018, ICML.

[9]  Francis R. Bach,et al.  On the Equivalence between Herding and Conditional Gradient Algorithms , 2012, ICML.

[10]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[11]  Marc Kennedy,et al.  Bayesian quadrature with non-normal approximating functions , 1998, Stat. Comput..

[12]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[13]  Mark A. Girolami,et al.  Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models , 2016, NIPS.

[14]  Martin Ehler,et al.  Optimal Monte Carlo integration on closed manifolds , 2017, Statistics and Computing.

[15]  Roman Garnett,et al.  An Improved Bayesian Framework for Quadrature of Constrained Integrands , 2018, ArXiv.

[16]  A. Genz,et al.  Fully symmetric interpolatory rules for multiple integrals over infinite regions with Gaussian weight , 1996 .

[17]  D. Xiu,et al.  Modeling uncertainty in flow simulations via generalized polynomial chaos , 2003 .

[18]  C. R. Dietrich,et al.  Fast and Exact Simulation of Stationary Gaussian Processes through Circulant Embedding of the Covariance Matrix , 1997, SIAM J. Sci. Comput..

[19]  Robert Schaback,et al.  Error estimates and condition numbers for radial basis function interpolation , 1995, Adv. Comput. Math..

[20]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[21]  Ronald Cools,et al.  Constructing cubature formulae: the science behind the art , 1997, Acta Numerica.

[22]  Ian H. Sloan,et al.  Why Are High-Dimensional Finance Problems Often of Low Effective Dimension? , 2005, SIAM J. Sci. Comput..

[23]  David L. Darmofal,et al.  Higher-Dimensional Integration with Gaussian Weight for Applications in Probabilistic Design , 2005, SIAM J. Sci. Comput..

[24]  Philippe Bekaert,et al.  Advanced global illumination , 2006 .

[25]  Christian Bouville,et al.  A Bayesian Monte Carlo Approach to Global Illumination , 2009, Comput. Graph. Forum.

[26]  Erik Strumbelj,et al.  An Efficient Explanation of Individual Classifications using Game Theory , 2010, J. Mach. Learn. Res..

[27]  Michael A. Osborne,et al.  Probabilistic Integration: A Role in Statistical Computation? , 2015, Statistical Science.

[28]  Luís Paulo Santos,et al.  A Spherical Gaussian Framework for Bayesian Monte Carlo Rendering of Glossy Surfaces , 2013, IEEE Transactions on Visualization and Computer Graphics.

[29]  J. McNamee,et al.  Construction of fully symmetric numerical integration formulas of fully symmetric numerical integration formulas , 1967 .

[30]  A. Y. Bezhaev,et al.  Cubature formulae on scattered meshes , 1991 .

[31]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[32]  Ian H. Sloan,et al.  QMC designs: Optimal order Quasi Monte Carlo integration schemes on the sphere , 2012, Math. Comput..

[33]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[34]  Francis R. Bach,et al.  Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression , 2016, J. Mach. Learn. Res..

[35]  Francis R. Bach,et al.  On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions , 2015, J. Mach. Learn. Res..

[36]  Simo Särkkä,et al.  A Bayes-Sard Cubature Method , 2018, NeurIPS.

[37]  K. Ritter,et al.  On an interpolatory method for high dimensional integration , 1999 .

[38]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[39]  Simo Särkkä,et al.  Fully symmetric kernel quadrature , 2017, SIAM J. Sci. Comput..

[40]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[41]  Roman Garnett,et al.  An Improved Bayesian Framework for Quadrature , 2017 .

[42]  Mark A. Girolami,et al.  On the Sampling Problem for Kernel Quadrature , 2017, ICML.

[43]  Markus Holtz,et al.  Sparse Grid Quadrature in High Dimensions with Applications in Finance and Insurance , 2010, Lecture Notes in Computational Science and Engineering.

[44]  Alvise Sommariva,et al.  Numerical Cubature on Scattered Data by Radial Basis Functions , 2005, Computing.

[45]  Mark A. Girolami,et al.  Bayesian Probabilistic Numerical Methods , 2017, SIAM Rev..

[46]  Carl E. Rasmussen,et al.  Sparse Spectrum Gaussian Process Regression , 2010, J. Mach. Learn. Res..

[47]  L. Pronzato,et al.  Bayesian quadrature and energy minimization for space-filling design , 2018, 1808.10722.

[48]  Roman Garnett,et al.  Bayesian Quadrature for Ratios , 2012, AISTATS.

[49]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[50]  Philip Rabinowitz,et al.  Methods of Numerical Integration , 1985 .

[51]  Florian Schäfer,et al.  Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity , 2017, Multiscale Model. Simul..

[52]  Kenji Fukumizu,et al.  Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings , 2017, Foundations of Computational Mathematics.

[53]  F. M. Larkin.  Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[54]  Wolfgang Hackbusch,et al.  A Sparse Matrix Arithmetic Based on H-Matrices. Part I: Introduction to H-Matrices , 1999, Computing.

[55]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[56]  Simo Särkkä,et al.  Classical quadrature rules via Gaussian processes , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[57]  Ronald A. DeVore,et al.  Computing a Quantity of Interest from Observational Data , 2018, Constructive Approximation.

[58]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[59]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[60]  Y. Marzouk,et al.  Uncertainty quantification in chemical systems , 2009 .

[61]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[62]  Frank Stenger,et al.  Con-struction of fully symmetric numerical integration formulas , 1967 .

[63]  Michael A. Osborne,et al.  Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees , 2015, NIPS.

[64]  M. Urner.  Scattered Data Approximation , 2016 .

[65]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[66]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[67]  Arno Solin,et al.  Variational Fourier Features for Gaussian Processes , 2016, J. Mach. Learn. Res..

[68]  Luís Paulo Santos,et al.  Efficient Quadrature Rules for Illumination Integrals: From Quasi Monte Carlo to Bayesian Monte Carlo , 2015, Efficient Quadrature Rules for Illumination Integrals: From Quasi Monte Carlo to Bayesian Monte Carlo.