Positive definite kernels: past, present and future

Positive definite kernels play an increasingly prominent role in many applications such as scattered data fitting, numerical solution of PDEs, computer experiments, machine learning, rapid prototyping and computer graphics. We discuss some of the historical and current developments of the theory and applications of positive definite kernels – always with an eye toward the mathematics of Göttingen in general and Robert Schaback in particular. A few comments concerning the future of the field are also provided.

[1]  E. Schmidt.  Über die Auflösung linearer Gleichungen mit Unendlich vielen unbekannten , 1908 .

[2]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[3]  M. Mathias.  Über positive Fourier-Integrale , 1923 .

[4]  R. Courant,et al.  Über die partiellen Differenzengleichungen der mathematischen Physik , 1928 .

[5]  A. Khintchine.  Korrelationstheorie der stationären stochastischen Prozesse , 1934 .

[6]  I. J. Schoenberg.  Metric spaces and completely monotone functions , 1938 .

[7]  S. Bergman.  The kernel function and conformal mapping , 1950 .

[8]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[9]  S. Bochner,et al.  Vorlesungen über Fouriersche Integrale , 1952 .

[10]  J. Riley.  Solving systems of linear equations with a positive definite, symmetric, but possibly ill-conditioned matrix , 1955 .

[11]  J. C. Mairhuber.  ON HAAR'S THEOREM CONCERNING CHEBYCHEV APPROXIMATION PROBLEMS HAVING UNIQUE SOLUTIONS' , 1956 .

[12]  Michael Golomb,et al.  OPTIMAL APPROXIMATIONS AND ERROR BOUNDS , 1958 .

[13]  Philip C. Curtis.  $n$-parameter families and best approximation. , 1959 .

[14]  Herbert Meschkowski,et al.  Hilbertsche Räume mit Kernfunktion , 1962 .

[15]  G. Matheron.  Les variables régionalisées et leur estimation : une application de la théorie de fonctions aléatoires aux sciences de la nature , 1965 .

[16]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[17]  J. Wells,et al.  Embeddings and Extensions in Analysis , 1975 .

[18]  J. Stewart.  Positive definite functions and generalizations, an historical survey , 1976 .

[19]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[20]  C. Berg,et al.  Harmonic Analysis on Semigroups , 1984 .

[21]  E. Kansa.  Application of Hardy's multiquadric interpolation to hydrodynamics , 1985 .

[22]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[23]  D. Braess.  Nonlinear Approximation Theory , 1986 .

[24]  C. Micchelli.  Interpolation of scattered data: Distance matrices and conditionally positive definite functions , 1986 .

[25]  A. Krall.  Applied Analysis , 1986 .

[26]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[27]  E. Kansa.  MULTIQUADRICS--A SCATTERED DATA APPROXIMATION SCHEME WITH APPLICATIONS TO COMPUTATIONAL FLUID-DYNAMICS-- II SOLUTIONS TO PARABOLIC, HYPERBOLIC AND ELLIPTIC PARTIAL DIFFERENTIAL EQUATIONS , 1990 .

[28]  M. Buhmann.  Multivariate cardinal interpolation with radial-basis functions , 1990 .

[29]  G. Wahba.  Spline models for observational data , 1990 .

[30]  Zongmin Wu,et al.  Hermite-Birkhoff interpolation of scattered data by radial basis functions , 1992, Approximation Theory and its Applications.

[31]  W. Madych,et al.  Bounds on multivariate polynomials and exponential error estimates for multiquadratic interpolation , 1992 .

[32]  R. Schaback.  Comparison of Radial Basis Function Interpolants , 1993 .

[33]  Zongmin Wu,et al.  Local error estimates for radial basis function interpolation of scattered data , 1993 .

[34]  Noel A Cressie,et al.  Statistics for Spatial Data, Revised Edition. , 1994 .

[35]  F. J. Narcowich,et al.  Generalized Hermite interpolation via matrix-valued conditionally positive definite functions , 1994 .

[36]  Robert Schaback,et al.  Error estimates and condition numbers for radial basis function interpolation , 1995, Adv. Comput. Math..

[37]  R. Schaback.  Creating Surfaces from Scattered Data Using Radial Basis Functions , 1995 .

[38]  R. Schaback.  Multivariate Interpolation and Approximation by Translates of a Basis Function the Space of Functions (1.1) Often , 1995 .

[39]  Holger Wendland,et al.  Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree , 1995, Adv. Comput. Math..

[40]  Robert Schaback,et al.  Operators on radial functions , 1996 .

[41]  Bengt Fornberg,et al.  A practical guide to pseudospectral methods: Introduction , 1996 .

[42]  T. Gutzmer.  Interpolation by positive definite functions on locally compact groups with application to SO (3) , 1996 .

[43]  N. Sivakumar,et al.  On Shifted Cardinal Interpolation by Gaussians and Multiquadrics , 1996 .

[44]  齋藤 三郎.  Integral transforms, reproducing kernels and their applications , 1997 .

[45]  M. Buhmann.  Radial functions on compact support , 1998 .

[46]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[47]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[48]  Carsten Franke,et al.  Solving partial differential equations by collocation using radial basis functions , 1998, Appl. Math. Comput..

[49]  Fred J. Hickernell,et al.  Radial basis function approximations as smoothing splines , 1999, Appl. Math. Comput..

[50]  Shmuel Rippa,et al.  An algorithm for selecting a good value for the parameter c in radial basis function interpolation , 1999, Adv. Comput. Math..

[51]  R. Schaback.  Native Hilbert Spaces for Radial Basis Functions I , 1999 .

[52]  Ward Cheney,et al.  A course in approximation theory , 1999 .

[53]  Not Available Not Available.  Meshfree particle methods , 2000 .

[54]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[55]  R. Schaback.  A unified theory of radial basis functions Native Hilbert spaces for radial basis functions II , 2000 .

[56]  Philip Crotwell.  Constructive Approximation on the Sphere , 2000 .

[57]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[58]  L. Trefethen.  Spectral Methods in MATLAB , 2000 .

[59]  Ted Chang,et al.  Introduction to Geostatistics: Applications in Hydrogeology , 2001, Technometrics.

[60]  Richard K. Beatson,et al.  Fast Solution of the Radial Basis Function Interpolation Equations: Domain Decomposition Methods , 2000, SIAM J. Sci. Comput..

[61]  Holger Wendland.  Gaussian interpolation revisited , 2001 .

[62]  Robert Schaback,et al.  On unsymmetric collocation by radial basis functions , 2001, Appl. Math. Comput..

[63]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[64]  Andrew D. Back,et al.  Radial Basis Functions , 2001 .

[65]  Robert Schaback,et al.  Adaptive Interpolation by Scaled Multiquadrics , 2002, Adv. Comput. Math..

[66]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[67]  Guirong Liu.  Mesh Free Methods: Moving Beyond the Finite Element Method , 2002 .

[68]  S. Atluri,et al.  The meshless local Petrov-Galerkin (MLPG) method , 2002 .

[69]  T. Driscoll,et al.  Interpolation in the limit of increasingly flat radial basis functions , 2002 .

[70]  B. Fornberg,et al.  Theoretical and computational aspects of multivariate interpolation with increasingly flat radial basis functions , 2003 .

[71]  B. Fornberg,et al.  A numerical study of some radial basis function based solution methods for elliptic PDEs , 2003 .

[72]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[73]  A. U.S.,et al.  Stable Computation of Multiquadric Interpolants for All Values of the Shape Parameter , 2003 .

[74]  Guirong Liu,et al.  Smoothed Particle Hydrodynamics: A Meshfree Particle Method , 2003 .

[75]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[76]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[77]  Armin Iske,et al.  Multiresolution Methods in Scattered Data Modelling , 2004, Lecture Notes in Computational Science and Engineering.

[78]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[79]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[80]  S. Atluri.  The meshless method (MLPG) for domain & BIE discretizations , 2004 .

[81]  Holger Wendland,et al.  Near-optimal data-independent point locations for radial basis function interpolation , 2005, Adv. Comput. Math..

[82]  Tobin A. Driscoll,et al.  Polynomials and Potential Theory for Gaussian Radial Basis Function Interpolation , 2005, SIAM J. Numer. Anal..

[83]  A. Cheng,et al.  Direct Solution Of Certain Ill-posed BoundaryValue Problems By The Collocation Method , 2005 .

[84]  Svenja Lowitzsch,et al.  Matrix-valued radial basis functions: stability estimates and applications , 2005, Adv. Comput. Math..

[85]  Carl de Boor.  On interpolation by radial polynomials , 2006, Adv. Comput. Math..

[86]  Holger Wendland,et al.  Kernel techniques: From machine learning to meshless methods , 2006, Acta Numerica.

[87]  Elisabeth Larsson,et al.  A new class of oscillatory radial basis functions , 2006, Comput. Math. Appl..

[88]  Roland Opfer,et al.  Multiscale kernels , 2006, Adv. Comput. Math..

[89]  Robert Schaback,et al.  Convergence of Unsymmetric Kernel-Based Meshless Collocation Methods , 2007, SIAM J. Numer. Anal..

[90]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[91]  Gregory E. Fasshauer,et al.  Meshfree Approximation Methods with Matlab , 2007, Interdisciplinary Mathematical Sciences.

[92]  Ding-Xuan Zhou,et al.  Learning Theory: An Approximation Theory Viewpoint , 2007 .

[93]  Bengt Fornberg,et al.  A Stable Algorithm for Flat Radial Basis Functions on a Sphere , 2007, SIAM J. Sci. Comput..

[94]  Wayne L. Tabor,et al.  Global and local optimization using radial basis function response surface models , 2007 .

[95]  Gregory E. Fasshauer,et al.  Iterated Approximate Moving Least Squares Approximation , 2007 .

[96]  Gregory E. Fasshauer,et al.  On choosing “optimal” shape parameters for RBF approximation , 2007, Numerical Algorithms.

[97]  Bengt Fornberg,et al.  The Runge phenomenon and spatially variable shape parameters in RBF interpolation , 2007, Comput. Math. Appl..

[98]  Gang Joon Yoon,et al.  Convergence of Increasingly Flat Radial Basis Interpolants to Polynomial Interpolants , 2007, SIAM J. Math. Anal..

[99]  Ramani Duraiswami,et al.  Fast Radial Basis Function Interpolation via Preconditioned Krylov Iteration , 2007, SIAM J. Sci. Comput..

[100]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[101]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[102]  E FasshauerG,et al.  Tutorial on Meshfree Approximation Methods with Matlab , 2008 .

[103]  Robert Schaback,et al.  Sampling and Stability , 2008, MMCS.

[104]  Marc Duflot,et al.  Meshless methods: A review and computer implementation aspects , 2008, Math. Comput. Simul..

[105]  Robert Schaback,et al.  Solvability of partial differential equations by meshless kernel methods , 2008, Adv. Comput. Math..

[106]  Robert Schaback,et al.  Limit problems for interpolation by analytic radial basis functions , 2008 .

[107]  Robert Schaback,et al.  Stable and Convergent Unsymmetric Meshless Collocation Methods , 2008, SIAM J. Numer. Anal..

[108]  C. Pipper,et al.  [''R"--project for statistical computing]. , 2008, Ugeskrift for laeger.

[109]  Series kernels and their approximation properties , 2009 .

[110]  Robert Schaback,et al.  A Newton basis for Kernel spaces , 2009, J. Approx. Theory.

[111]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[112]  Robert Schaback,et al.  Nonstandard Kernels and their Applications , 2009 .

[113]  Stefan Müller,et al.  Komplexität und Stabilität von kernbasierten Rekonstruktionsmethoden , 2009 .

[114]  Lei Wang,et al.  An analytic approximation to the cardinal functions of Gaussian radial basis functions on an infinite lattice , 2009, Appl. Math. Comput..

[115]  Robert Schaback,et al.  On convergent numerical algorithms for unsymmetric collocation , 2009, Adv. Comput. Math..

[116]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[117]  Barbara Zwicknagl,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Power Series Kernels Power Series Kernels , 2022 .

[118]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[119]  D. Hilbert.  Grundzuge Einer Allgemeinen Theorie Der Linearen Integralgleichungen , 2009 .

[120]  Ken-ichi Anjyo,et al.  Scattered data interpolation and approximation for computer graphics , 2010, SA '10.

[121]  Christian Rieger,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Sampling Inequalities for Infinitely Smooth Functions, with Applications to Interpolation and Machine Learning Sampling Inequalities for Infinitely Smooth Functions, with Applications to Interpolation and Machine Learning , 2022 .

[122]  Robert Schaback,et al.  Unsymmetric meshless methods for operator equations , 2010, Numerische Mathematik.

[123]  Robert Schaback,et al.  Stability of kernel-based interpolation , 2010, Adv. Comput. Math..

[124]  Elisabeth Larsson,et al.  Stable Computations with Gaussian Radial Basis Functions , 2011, SIAM J. Sci. Comput..

[125]  Peter Z. G. Qian,et al.  Accurate emulators for large-scale computer experiments , 2011, 1203.2433.

[126]  Ben Niu.  MONTE CARLO SIMULATION OF INFINITE-DIMENSIONAL INTEGRALS , 2011 .

[127]  M. Mongillo.  Choosing Basis Functions and Shape Parameters for Radial Basis Function Methods , 2011 .

[128]  Michael Scheuerer,et al.  An alternative procedure for selecting a good value for the parameter c in RBF-interpolation , 2011, Adv. Comput. Math..

[129]  Robert Schaback,et al.  Bases for kernel-based spaces , 2011, J. Comput. Appl. Math..

[130]  G. Fasshauer,et al.  STABLE EVALUATION OF GAUSSIAN RBF INTERPOLANTS , 2011 .

[131]  L. Bos,et al.  On Multivariate Newton Interpolation at Discrete Leja Points , 2011 .

[132]  Holger Wendland,et al.  A high-order, analytically divergence-free discretization method for Darcy's problem , 2010, Math. Comput..

[133]  Fred J. Hickernell,et al.  On Dimension-independent Rates of Convergence for Function Approximation with Gaussian Kernels , 2012, SIAM J. Numer. Anal..

[134]  F. J. Narcowich,et al.  Cardinal Interpolation with Gaussian Kernels , 2010, 1008.3168.

[135]  Fred J. Hickernell,et al.  Multivariate interpolation with increasingly flat radial basis functions of finite smoothness , 2012, Adv. Comput. Math..

[136]  Grady B. Wright,et al.  Scattered Data Interpolation on Embedded Submanifolds with Restricted Positive Definite Kernels: Sobolev Error Estimates , 2010, SIAM J. Numer. Anal..