On the positivity and magnitudes of Bayesian quadrature weights

This article reviews and studies the properties of Bayesian quadrature weights, which strongly affect stability and robustness of the quadrature rule. Specifically, we investigate conditions that are needed to guarantee that the weights are positive or to bound their magnitudes. First, it is shown that the weights are positive in the univariate case if the design points locally minimise the posterior integral variance and the covariance kernel is totally positive (e.g., Gaussian and Hardy kernels). This suggests that gradient-based optimisation of design points may be effective in constructing stable and robust Bayesian quadrature rules. Secondly, we show that magnitudes of the weights admit an upper bound in terms of the fill distance and separation radius if the RKHS of the kernel is a Sobolev space (e.g., Matern kernels), suggesting that quasi-uniform points should be used. A number of numerical examples demonstrate that significant generalisations and improvements appear to be possible, manifesting the need for further research.

[1]  Nira Richter-Dyn Minimal Interpolation and Approximation in Hilbert Spaces , 1971 .

[2]  Simo Särkkä,et al.  On the use of gradient information in Gaussian process quadratures , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[3]  F. M. Larkin Optimal approximation in Hilbert spaces with reproducing kernel functions , 1970 .

[4]  F. M. Larkin Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[5]  Borislav Bojanov On the existence of optimal quadrature formulae for smooth functions , 1979 .

[6]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[7]  David L. Barrow On multiple node Gaussian quadrature formulae , 1978 .

[8]  W. Gautschi Orthogonal Polynomials: Computation and Approximation , 2004 .

[9]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[10]  J. Cooper TOTAL POSITIVITY, VOL. I , 1970 .

[11]  Simo Särkkä,et al.  Fully symmetric kernel quadrature , 2017, SIAM J. Sci. Comput..

[12]  Robert Schaback,et al.  Stability of kernel-based interpolation , 2010, Adv. Comput. Math..

[13]  Alvise Sommariva,et al.  Meshless cubature by Green's formula , 2006, Appl. Math. Comput..

[14]  Nira Richter-Dyn PROPERTIES OF MINIMAL INTEGRATION RULES. II , 1971 .

[15]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  W. J. Studden,et al.  Tchebycheff Systems: With Applications in Analysis and Statistics. , 1967 .

[17]  M. Urner Scattered Data Approximation , 2016 .

[18]  H. L. Loeb,et al.  Multiple zeros and applications to optimal linear functionals , 1975 .

[19]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[20]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[21]  J. Cooper SINGULAR INTEGRALS AND DIFFERENTIABILITY PROPERTIES OF FUNCTIONS , 1973 .

[22]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[23]  Jonathan W. Pillow,et al.  Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature. , 2017, 1704.00060.

[24]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[25]  Meishe Isakovich Levin,et al.  Optimal quadrature formulas , 1979 .

[26]  David Duvenaud,et al.  Optimally-Weighted Herding is Bayesian Quadrature , 2012, UAI.

[27]  Mark A. Girolami,et al.  On the Sampling Problem for Kernel Quadrature , 2017, ICML.

[28]  Mark A. Girolami,et al.  Bayesian Probabilistic Numerical Methods , 2017, SIAM Rev..

[29]  Simo Särkkä,et al.  Gaussian kernel quadrature at scaled Gauss–Hermite nodes , 2018, BIT Numerical Mathematics.

[30]  Simo Särkkä,et al.  Classical quadrature rules via Gaussian processes , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[31]  Kenji Fukumizu,et al.  Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings , 2017, Foundations of Computational Mathematics.

[32]  Manuel Gräf,et al.  Points on manifolds with asymptotically optimal covering radius , 2016, J. Complex..

[33]  Lloyd N. Trefethen,et al.  Impossibility of Fast Stable Approximation of Analytic Functions from Equispaced Samples , 2011, SIAM Rev..

[34]  Jacob Burbea Total positivity of certain reproducing kernels. , 1976 .

[35]  C. W. Clenshaw,et al.  A method for numerical integration on an automatic computer , 1960 .

[36]  Michael I. Jordan,et al.  Gradient Descent Only Converges to Minimizers , 2016, COLT.

[37]  Gregory E. Fasshauer,et al.  Meshfree Approximation Methods with Matlab , 2007, Interdisciplinary Mathematical Sciences.

[38]  Tobin A. Driscoll,et al.  Polynomials and Potential Theory for Gaussian Radial Basis Function Interpolation , 2005, SIAM J. Numer. Anal..

[39]  Begnaud Francis Hildebrand,et al.  Introduction to numerical analysis: 2nd edition , 1987 .

[40]  Fred J. Hickernell,et al.  Fast automatic Bayesian cubature using lattice sampling , 2018, Statistics and Computing.

[41]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[42]  Holger Wendland,et al.  Approximate Interpolation with Applications to Selecting Smoothing Parameters , 2005, Numerische Mathematik.

[43]  Alvise Sommariva,et al.  Numerical Cubature on Scattered Data by Radial Basis Functions , 2005, Computing.

[44]  Robert Schaback,et al.  Stability constants for kernel-based interpolation processes , 2008 .

[45]  Mark A. Girolami,et al.  Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models , 2016, NIPS.

[46]  Nira Richter Properties of Minimal Integration Rules , 1970 .

[47]  Hrushikesh Narhar Mhaskar,et al.  Spherical Marcinkiewicz-Zygmund inequalities and positive quadrature , 2001, Math. Comput..

[48]  Michael A. Osborne,et al.  Probabilistic Integration: A Role in Statistical Computation? , 2015, Statistical Science.

[49]  Klaus-Jürgen Förster,et al.  Variance in Quadrature — a Survey , 1993 .

[50]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[51]  María Cruz López de Silanes,et al.  An extension of a bound for functions in Sobolev spaces, with applications to (m, s)-spline interpolation and smoothing , 2007, Numerische Mathematik.

[52]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[53]  Roman Garnett,et al.  An Improved Bayesian Framework for Quadrature of Constrained Integrands , 2018, ArXiv.

[54]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[55]  H. L. Loeb,et al.  On the existence of optimal integration formulas for analytic functions , 1974 .

[56]  Simo Särkkä,et al.  A Bayes-Sard Cubature Method , 2018, NeurIPS.

[57]  Kenji Fukumizu,et al.  Convergence guarantees for kernel-based quadrature rules in misspecified settings , 2016, NIPS.

[58]  Erich Novak Intractability Results for Positive Quadrature Formulas and Extremal Problems for Trigonometric Polynomials , 1999, J. Complex..

[59]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[60]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[61]  Carl E. Rasmussen,et al.  Bayesian Monte Carlo , 2002, NIPS.