Reproducing Kernel Hilbert Spaces, Polynomials, and the Classical Moment Problem

We show that polynomials do not belong to the reproducing kernel Hilbert space of infinitely differentiable translation-invariant kernels whose spectral measures have moments corresponding to a determinate moment problem. Our proof is based on relating this question to the problem of the best linear estimation in continuous time one-parameter regression models with a stationary error process defined by the kernel. In particular, we show that the existence of a sequence of estimators with variances converging to 0 implies that the regression function cannot be an element of the reproducing kernel Hilbert space. This question is then related to the determinacy of the Hamburger moment problem for the spectral measure corresponding to the kernel. AMS Subject Classification: 46E22, 62J05, 44A60

[1]  Anatoly Zhigljavsky,et al.  Bayesian and High-Dimensional Global Optimization , 2021 .

[2]  A. Zhigljavsky,et al.  The BLUE in continuous-time regression models with correlated errors , 2019, The Annals of Statistics.

[3]  G. A. Young,et al.  High‐dimensional Statistics: A Non‐asymptotic Viewpoint, Martin J.Wainwright, Cambridge University Press, 2019, xvii 552 pages, £57.99, hardback ISBN: 978‐1‐1084‐9802‐9 , 2020, International Statistical Review.

[4]  Andreas Krause,et al.  A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions , 2016, bioRxiv.

[5]  W. J. Studden,et al.  On an extremal problem of Feje´r , 1988 .

[6]  Toni Karvonen.  On non-inclusion of certain functions in reproducing kernel Hilbert spaces , 2021 .

[7]  Holger Dette,et al.  On a new characterization of the classical orthogonal polynomials , 1992 .

[8]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[9]  Allan Gut,et al.  The Moment Problem , 2002, Encyclopedia of Special Functions: The Askey-Bateman Project.

[10]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[11]  M. Urner.  Scattered Data Approximation , 2016 .

[12]  V. Paulsen,et al.  An Introduction to the Theory of Reproducing Kernel Hilbert Spaces , 2016 .

[13]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[14]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[15]  Robert Schaback,et al.  Interpolation of spatial data – A stochastic or a deterministic problem? , 2013, European Journal of Applied Mathematics.

[16]  Stamatis Cambanis,et al.  Sampling designs for the detection of signals in noise , 1983, IEEE Trans. Inf. Theory.

[17]  J. Norris.  Appendix: probability and measure , 1997 .

[18]  Luc Pronzato,et al.  Bayesian Quadrature, Energy Minimization, and Space-Filling Design , 2020, SIAM/ASA J. Uncertain. Quantification.

[19]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[20]  Jordan Stoyanov,et al.  Counterexamples in Probability , 1989 .

[21]  Stefan Rolewicz,et al.  On a problem of moments , 1968 .

[22]  Michael L. Stein,et al.  Maximum Likelihood Estimation for a Smooth Gaussian Random Field Model , 2017, SIAM/ASA J. Uncertain. Quantification.

[23]  Michael A. Osborne,et al.  Probabilistic Integration: A Role in Statistical Computation? , 2015, Statistical Science.

[24]  Yang Chen,et al.  Small eigenvalues of large Hankel matrices: The indeterminate case , 1999 .

[25]  Dino Sejdinovic,et al.  Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences , 2018, ArXiv.

[26]  J. Joseph,et al.  Fourier transforms , 2012 .

[27]  E. Parzen.  An Approach to Time Series Analysis , 1961 .

[28]  A. Yaglom.  Correlation Theory of Stationary and Related Random Functions I: Basic Results , 1987 .

[29]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[30]  SOME REMARKS ON THE MOMENT PROBLEM (II) , 1963 .

[31]  Holger Dette,et al.  The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis , 1997 .

[32]  Don R. Hush,et al.  An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels , 2006, IEEE Transactions on Information Theory.

[33]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[34]  Ding-Xuan Zhou,et al.  Reproducing Kernel Hilbert Spaces Associated with Analytic Translation-Invariant Mercer Kernels , 2008 .

[35]  Antonio Candelieri,et al.  Bayesian Optimization and Data Science , 2019, SpringerBriefs in Optimization.

[36]  Radakovič.  The theory of approximation , 1932 .

[37]  G. D. Lin.  Recent developments on the moment problem , 2017, 1703.01027.

[38]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[39]  Simo Särkkä,et al.  Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions , 2020, SIAM/ASA J. Uncertain. Quantification.

[40]  Bing Li,et al.  Variable selection via additive conditional independence , 2016 .

[41]  K. Ritter,et al.  MULTIVARIATE INTEGRATION AND APPROXIMATION FOR RANDOM FIELDS SATISFYING SACKS-YLVISAKER CONDITIONS , 1995 .

[42]  N. Akhiezer,et al.  The Classical Moment Problem and Some Related Questions in Analysis , 2020 .