Refinement of Reproducing Kernels

We continue our recent study on constructing a refinement kernel for a given kernel so that the reproducing kernel Hilbert space associated with the refinement kernel contains that with the original kernel as a subspace. To motivate this study, we first develop a refinement kernel method for learning, which gives an efficient algorithm for updating a learning predictor. Several characterizations of refinement kernels are then presented. It is shown that a nontrivial refinement kernel for a given kernel always exists if the input space has an infinite cardinal number. Refinement kernels for translation invariant kernels and Hilbert-Schmidt kernels are investigated. Various concrete examples are provided.

[1]  Stéphane Canu,et al.  Frames, Reproducing Kernels, Regularization and Learning , 2005, J. Mach. Learn. Res..

[2]  Golub Gene H. Et.Al.  Matrix Computations, 3rd Edition , 2007 .

[3]  Charles A. Micchelli,et al.  Learning the Kernel Function via Regularization , 2005, J. Mach. Learn. Res..

[4]  Ingo Steinwart,et al.  Fast rates for support vector machines using Gaussian kernels , 2007, 0708.1838.

[5]  Yuesheng Xu,et al.  On the matrix completion problem for multivariate filter bank construction , 2007, Adv. Comput. Math..

[6]  Charles A. Micchelli,et al.  Regularity of multiwavelets , 1997, Adv. Comput. Math..

[7]  Yuesheng Xu,et al.  Universal Kernels , 2006, J. Mach. Learn. Res..

[8]  C. Micchelli,et al.  Functions that preserve families of positive semidefinite matrices , 1995 .

[9]  Yuesheng Xu,et al.  Refinable Kernels , 2007, J. Mach. Learn. Res..

[10]  G. Wahba.  Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[11]  A. Ron,et al.  Strictly positive definite functions on spheres in Euclidean spaces , 1994, Math. Comput..

[12]  S. Smale,et al.  ESTIMATING THE APPROXIMATION ERROR IN LEARNING THEORY , 2003 .

[13]  R. Cooke.  Real and Complex Analysis , 2011 .

[14]  Yang Wang,et al.  Wavelets, tiling, and spectral sets , 2002 .

[15]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[16]  E. Stein,et al.  Introduction to Fourier Analysis on Euclidean Spaces. , 1971 .

[17]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[18]  Yiming Ying,et al.  Learnability of Gaussians with Flexible Variances , 2007, J. Mach. Learn. Res..

[19]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[20]  R. Jia.  Characterization of Smoothness of Multivariate Refinable Functions in Sobolev Spaces , 1999 .

[21]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[22]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[23]  C. Gasquet,et al.  Fourier analysis and applications , 1998 .

[24]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[25]  S. Bochner,et al.  Lectures on Fourier integrals : with an author's supplement on monotonic functions, Stieltjes integrals, and harmonic analysis , 1959 .

[26]  Bernhard Schölkopf,et al.  Implicit Surface Modelling with a Globally Regularised Basis of Compact Support , 2006, Comput. Graph. Forum.

[27]  Yuesheng Xu,et al.  Multivariate Filter Banks Having Matrix Factorizations , 2003, SIAM J. Matrix Anal. Appl..

[28]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[29]  C. Micchelli,et al.  Using the Matrix Refinement Equation for the Construction of Wavelets on Invariant Sets , 1994 .

[30]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[31]  T. J. Rivlin.  The Chebyshev polynomials , 1974 .

[32]  E. C. Titchmarsh,et al.  The Laplace Transform , 1991, Heat Transfer 1.

[33]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[34]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[35]  S. L. Lee,et al.  WAVELETS OF MULTIPLICITY r , 1994 .

[36]  V. Hutson.  Integral Equations , 1967, Nature.

[37]  J. Dicapua.  Chebyshev Polynomials , 2019, Fibonacci and Lucas Numbers With Applications.

[38]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[39]  Roland Opfer,et al.  Multiscale kernels , 2006, Adv. Comput. Math..

[40]  Rong-Qing Jia,et al.  Smoothness of Multiple Refinable Functions and Multiple Wavelets , 1999, SIAM J. Matrix Anal. Appl..

[41]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[42]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[43]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[44]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[45]  L. Grafakos.  Classical and modern Fourier analysis , 2003 .

[46]  I. J. Schoenberg.  Metric spaces and completely monotone functions , 1938 .

[47]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[48]  Hongwei Sun,et al.  Mercer theorem for RKHS on noncompact sets , 2005, J. Complex..