Fully symmetric kernel quadrature

Kernel quadratures and other kernel-based approximation methods typically suffer from prohibitive cubic time and quadratic space complexity in the number of function evaluations. The problem arises because a system of linear equations needs to be solved. In this article we show that the weights of a kernel quadrature rule can be computed efficiently and exactly for up to tens of millions of nodes if the kernel, integration domain, and measure are fully symmetric and the node set is a union of fully symmetric sets. This is based on the observations that in such a setting there are only as many distinct weights as there are fully symmetric sets and that these weights can be solved from a linear system of equations constructed out of row sums of certain submatrices of the full kernel matrix. We present several numerical examples that show feasibility, both for a large number of nodes and in high dimensions, of the developed fully symmetric kernel quadrature rules. Most prominent of the fully symmetric kernel...

[1]  I. Error Coefficients.  Symmetric Integration Rules for Hypercubes , 2016 .

[2]  Kenji Fukumizu,et al.  Convergence guarantees for kernel-based quadrature rules in misspecified settings , 2016, NIPS.

[3]  F. M. Larkin.  Optimal approximation in Hilbert spaces with reproducing kernel functions , 1970 .

[4]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[5]  Shmuel Rippa,et al.  An algorithm for selecting a good value for the parameter c in radial basis function interpolation , 1999, Adv. Comput. Math..

[6]  Simo Särkkä,et al.  Classical quadrature rules via Gaussian processes , 2017, 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP).

[7]  Mark A. Girolami,et al.  On the Sampling Problem for Kernel Quadrature , 2017, ICML.

[8]  Markus Holtz,et al.  Sparse Grid Quadrature in High Dimensions with Applications in Finance and Insurance , 2010, Lecture Notes in Computational Science and Engineering.

[9]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[10]  David L. Darmofal,et al.  Higher-Dimensional Integration with Gaussian Weight for Applications in Probabilistic Design , 2005, SIAM J. Sci. Comput..

[11]  John F. Monahan,et al.  A stochastic algorithm for high-dimensional integrals over unbounded regions with Gaussian weight , 1999 .

[12]  Kenji Fukumizu,et al.  Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings , 2017, Foundations of Computational Mathematics.

[13]  Mark A. Girolami,et al.  Bayesian Probabilistic Numerical Methods , 2017, SIAM Rev..

[14]  Alan Genz,et al.  Fully symmetric interpolatory rules for multiple integrals , 1986 .

[15]  E. Novak,et al.  Tractability of Multivariate Problems Volume II: Standard Information for Functionals , 2010 .

[16]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[17]  Dongbin Xiu,et al.  Stochastic Collocation Methods on Unstructured Grids in High Dimensions via Interpolation , 2012, SIAM J. Sci. Comput..

[18]  K. Ritter,et al.  The Curse of Dimension and a Universal Method For Numerical Integration , 1997 .

[19]  E. Novak,et al.  Tractability of Multivariate Problems , 2008 .

[20]  K. Ritter,et al.  On an interpolatory method for high dimensional integration , 1999 .

[21]  C. D. Boor,et al.  On multivariate polynomial interpolation , 1990 .

[22]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[23]  Michael A. Osborne,et al.  Probabilistic Integration: A Role for Statisticians in Numerical Analysis? , 2015 .

[24]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[25]  A. Genz,et al.  Fully symmetric interpolatory rules for multiple integrals over infinite regions with Gaussian weight , 1996 .

[26]  S. Gupta,et al.  Statistical decision theory and related topics IV , 1988 .

[27]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[28]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[29]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[30]  K. Ritter,et al.  High dimensional integration of smooth functions over cubes , 1996 .

[31]  Jouni Hartikainen,et al.  On the relation between Gaussian process quadratures and sigma-point methods , 2015, 1504.05994.

[32]  Frank Stenger,et al.  Con-struction of fully symmetric numerical integration formulas , 1967 .

[33]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[34]  Michael A. Osborne,et al.  Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees , 2015, NIPS.

[35]  David Duvenaud,et al.  Optimally-Weighted Herding is Bayesian Quadrature , 2012, UAI.

[36]  Jeremy Levesley,et al.  Quasi-interpolation on a sparse grid with Gaussian , 2016 .

[37]  J. N. Lyness.  Symmetric integration rules for hypercubes. I. Error coefficients , 1965 .

[38]  Jeremy Levesley,et al.  Fast multilevel sparse Gaussian kernels for high-dimensional approximation and integration , 2015, 1501.03296.

[39]  Carl E. Rasmussen,et al.  Bayesian Monte Carlo , 2002, NIPS.

[40]  S. Tezuka,et al.  Toward real-time pricing of complex financial derivatives , 1996 .

[41]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[42]  A. Y. Bezhaev,et al.  Cubature formulae on scattered meshes , 1991 .

[43]  Ondřej Straka,et al.  Gaussian Process Quadrature Moment Transform , 2017, IEEE Transactions on Automatic Control.

[44]  Ondrej Straka,et al.  Student-t process quadratures for filtering of non-linear systems with heavy-tailed noise , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[45]  Jean-François Richard,et al.  Methods of Numerical Integration , 2000 .

[46]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[47]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[48]  Thomas Gerstner,et al.  Numerical integration using sparse grids , 2004, Numerical Algorithms.

[49]  F. M. Larkin.  Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[50]  Alvise Sommariva,et al.  Numerical Cubature on Scattered Data by Radial Basis Functions , 2005, Computing.

[51]  R. Caflisch.  Monte Carlo and quasi-Monte Carlo methods , 1998, Acta Numerica.

[52]  Barbara I. Wohlmuth,et al.  Algorithm 847: Spinterp: piecewise multilinear hierarchical sparse grid interpolation in MATLAB , 2005, TOMS.

[53]  Jochen Garcke,et al.  A dimension adaptive sparse grid combination technique for machine learning , 2007 .

[54]  M. Urner.  Scattered Data Approximation , 2016 .

[55]  Carl E. Rasmussen,et al.  Active Learning of Model Evidence Using Bayesian Quadrature , 2012, NIPS.

[56]  John Monahan,et al.  Stochastic Integration Rules for Infinite Regions , 1998, SIAM J. Sci. Comput..

[57]  Fazli Subhan,et al.  Multilevel Sparse Kernel-Based Interpolation , 2012, SIAM J. Sci. Comput..

[58]  Jeremy Levesley,et al.  Multilevel quasi-interpolation on a sparse grid with the Gaussian , 2017, Numerical Algorithms.

[59]  K. Ritter,et al.  Simple Cubature Formulas with High Polynomial Exactness , 1999 .

[60]  Ronald Cools,et al.  Constructing cubature formulae: the science behind the art , 1997, Acta Numerica.

[61]  Gregory E. Fasshauer,et al.  On choosing “optimal” shape parameters for RBF approximation , 2007, Numerical Algorithms.