Universal Kernels

In this paper we investigate conditions on the features of a continuous kernel so that it may approximate an arbitrary continuous target function uniformly on any compact subset of the input space. A number of concrete examples are given of kernels with this universal approximating property.

[1]  T. J. Rivlin.  The Chebyshev polynomials , 1974 .

[2]  Raymond M. Redheffer,et al.  Completeness of sets of complex exponentials , 1977 .

[3]  Hongwei Sun,et al.  Mercer theorem for RKHS on noncompact sets , 2005, J. Complex..

[4]  Alexander J. Smola,et al.  Learning the Kernel with Hyperkernels , 2005, J. Mach. Learn. Res..

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[6]  Johan A. K. Suykens,et al.  Advances in learning theory : methods, models and applications , 2003 .

[7]  Gunnar Rätsch,et al.  A General and Efficient Multiple Kernel Learning Algorithm , 2005, NIPS.

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  C. Micchelli,et al.  Functions that preserve families of positive semidefinite matrices , 1995 .

[10]  E. Stein,et al.  Introduction to Fourier Analysis on Euclidean Spaces. , 1971 .

[11]  Charles A. Micchelli,et al.  Learning Convex Combinations of Continuously Parameterized Basic Kernels , 2005, COLT.

[13]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[14]  Charles A. Micchelli,et al.  Feature space perspectives for learning the kernel , 2006, Machine Learning.

[15]  L. Galway.  Spline Models for Observational Data , 1991 .

[16]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[17]  Mahesan Niranjan,et al.  Uncertainty in geometric computations , 2002 .

[18]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[19]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[20]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[21]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[22]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[23]  Charles A. Micchelli,et al.  A DC-programming algorithm for kernel selection , 2006, ICML.

[24]  Paul Malliavin,et al.  On the closure of characters and the zeros of entire functions , 1967 .

[25]  I. J. Schoenberg.  Metric spaces and completely monotone functions , 1938 .

[26]  Rene F. Swarttouw,et al.  Orthogonal polynomials , 2020, NIST Handbook of Mathematical Functions.

[27]  Charles A. Micchelli,et al.  A Function Representation for Learning in Banach Spaces , 2004, COLT.

[28]  Gabriele Steidl,et al.  SVM-Based Feature Selection by Direct Objective Minimisation , 2004, DAGM-Symposium.

[29]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[30]  I. J. Schoenberg.  Positive definite functions on spheres , 1942 .

[31]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[32]  S. Bochner,et al.  Lectures on Fourier integrals : with an author's supplement on monotonic functions, Stieltjes integrals, and harmonic analysis , 1959 .

[33]  J. Dicapua.  Chebyshev Polynomials , 2019, Fibonacci and Lucas Numbers With Applications.