Non‐parametric regression with wavelet kernels

This paper introduces a method to construct a reproducing wavelet kernel Hilbert spaces for non-parametric regression estimation when the sampling points are not equally spaced. Another objective is to make high-dimensional wavelet estimation problems tractable. It then provides a theoretical foundation to build reproducing kernel from operators and a practical technique to obtain reproducing kernel Hilbert spaces spanned by a set of wavelets. A multiscale approximation technique that aims at taking advantage of the multiresolution structure of wavelets is also described. Examples on toy regression and a real-world problem illustrate the effectiveness of these wavelet kernels. Copyright © 2005 John Wiley & Sons, Ltd.

[1]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[2]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[3]  R. Tibshirani,et al.  Generalized additive models for medical research , 1986, Statistical methods in medical research.

[4]  P. Mikusinski,et al.  Introduction to Hilbert spaces with applications , 1990 .

[5]  G. Wahba.  Spline models for observational data , 1990 .

[6]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[7]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[8]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[9]  Noga Alon,et al.  Scale-sensitive dimensions, uniform convergence, and learnability , 1997, JACM.

[10]  Qinghua Zhang,et al.  Using wavelet network in nonparametric estimation , 1997, IEEE Trans. Neural Networks.

[11]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[12]  Arne Kovac,et al.  Extending the Scope of Wavelet Regression Methods by Coefficient-Dependent Thresholding , 2000 .

[13]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[14]  E. Candès.  Harmonic Analysis of Neural Networks , 1999 .

[15]  Massimiliano Pontil,et al.  On the Vgamma Dimension for Regression in Reproducing Kernel Hilbert Spaces , 1999, ALT.

[16]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[17]  Emmanuel J. Candès.  Ridgelets and the Representation of Mutilated Sobolev Functions , 2001, SIAM J. Math. Anal..

[18]  Jianqing Fan,et al.  Regularization of Wavelet Approximations , 2001 .

[19]  Guy P. Nason.  Choice of wavelet smoothness, primary resolution and threshold in wavelet shrinkage , 2002, Stat. Comput..