Nonparametric regression using needlet kernels for spherical data

Needlets have been recognized as state-of-the-art tools to tackle spherical data, due to their excellent localization properties in both spacial and frequency domains. This paper considers developing kernel methods associated with the needlet kernel for nonparametric regression problems whose predictor variables are defined on a sphere. Due to the localization property in the frequency domain, we prove that the regularization parameter of the kernel ridge regression associated with the needlet kernel can decrease arbitrarily fast. A natural consequence is that the regularization term for the kernel ridge regression is not necessary in the sense of rate optimality. Based on the excellent localization property in the spacial domain further, we also prove that all the $l^{q}$ $(01\leq q < \infty)$ kernel regularization estimates associated with the needlet kernel, including the kernel lasso estimate and the kernel bridge estimate, possess almost the same generalization capability for a large range of regularization parameters in the sense of rate optimality. This finding tentatively reveals that, if the needlet kernel is utilized, then the choice of $q$ might not have a strong impact in terms of the generalization capability in some modeling contexts. From this perspective, $q$ can be arbitrarily specified, or specified merely by other no generalization criteria like smoothness, computational complexity, sparsity, etc..

[1]  Bruno Pelletier.  Non-parametric regression estimation on closed Riemannian manifolds , 2006 .

[2]  Hongzhi Tong,et al.  Least Square Regression with lp-Coefficient Regularization , 2010, Neural Computation.

[3]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[4]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[5]  Pencho Petrushev,et al.  Decomposition of Besov and Triebel–Lizorkin spaces on the sphere , 2006 .

[6]  P. Bickel,et al.  Local polynomial regression on unknown manifolds , 2007, 0708.0983.

[7]  Ronald A. DeVore,et al.  Approximation Methods for Supervised Learning , 2006, Found. Comput. Math..

[8]  Pencho Petrushev,et al.  Localized Tight Frames on Spheres , 2006, SIAM J. Math. Anal..

[9]  Hrushikesh Narhar Mhaskar,et al.  Approximation properties of zonal function networks using scattered data on the sphere , 1999, Adv. Comput. Math..

[10]  S. Smale,et al.  Learning Theory Estimates via Integral Operators and Their Approximations , 2007 .

[11]  A. Caponnetto,et al.  Optimal Rates for the Regularized Least-Squares Algorithm , 2007, Found. Comput. Math..

[12]  Hrushikesh Narhar Mhaskar,et al.  On the representation of smooth functions on the sphere using finitely many bits , 2005 .

[13]  Kurt Jetter,et al.  Approximation with polynomial kernels and SVM classifiers , 2006, Adv. Comput. Math..

[14]  Hrushikesh Narhar Mhaskar,et al.  Spherical Marcinkiewicz-Zygmund inequalities and positive quadrature , 2001, Math. Comput..

[15]  Vitaly Maiorov,et al.  Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary , 2006, Adv. Comput. Math..

[16]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[17]  Zongben Xu,et al.  Learning rates of regularized regression on the unit sphere , 2013 .

[18]  Ted Chang,et al.  Regression techniques in plate tectonics , 2000 .

[19]  Hrushikesh Narhar Mhaskar,et al.  L BERNSTEIN ESTIMATES AND APPROXIMATION BY SPHERICAL BASIS FUNCTIONS , 2010 .

[20]  Ding-Xuan Zhou,et al.  Distributed Kernel-Based Gradient Descent Algorithms , 2018 .

[21]  J. Mackenzie,et al.  The estimation of an orientation relationship , 1957 .

[22]  R. Wilson.  Modern Cosmology , 2004 .

[23]  Hrushikesh Narhar Mhaskar,et al.  Polynomial operators and local smoothness classes on the unit interval , 2004, J. Approx. Theory.

[24]  S. Smale,et al.  Reproducing kernel hilbert spaces in learning theory , 2006 .

[25]  Ding-Xuan Zhou,et al.  Concentration estimates for learning with ℓ1-regularizer and data dependent hypothesis spaces , 2011 .

[26]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[27]  Feng Dai,et al.  Approximation of smooth functions on compact two-point homogeneous spaces , 2005, math/0510007.

[28]  Ian H. Sloan,et al.  Filtered hyperinterpolation: a constructive polynomial approximation on the sphere , 2012 .

[29]  Zen-Chung Shih,et al.  All-frequency precomputed radiance transfer using spherical radial basis functions and clustered tensor approximation , 2006, ACM Trans. Graph..

[30]  Felipe Cucker,et al.  Best Choices for Regularization Parameters in Learning Theory: On the Bias—Variance Problem , 2002, Found. Comput. Math..

[31]  Jean-Baptiste Monnier.  Nonparametric regression on the hyper-sphere with uniform design , 2011 .

[32]  P. Baldi,et al.  Asymptotics for spherical needlets , 2006, math/0606599.

[33]  Quoc Thong Le Gia,et al.  Localized Linear Polynomial Operators and Quadrature Formulas on the Sphere , 2008, SIAM J. Numer. Anal..

[34]  Philip Crotwell.  Constructive Approximation on the Sphere , 2000 .

[35]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[36]  Luoqing Li.  Regularized Least Square Regression with Spherical Polynomial Kernels , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[37]  Gerard Kerkyacharian,et al.  Concentration inequalities and confidence bands for needlet density estimators on compact homogeneous manifolds , 2011, 1102.2450.

[38]  J. Fadili,et al.  CMB data analysis and sparsity , 2008, 0804.1295.

[39]  Felipe Cucker,et al.  Learning Theory: An Approximation Theory Viewpoint: Index , 2007 .

[40]  C. Taylor,et al.  Nonparametric Regression for Spherical Data , 2014 .

[41]  Joel Ratsaby,et al.  On the Degree of Approximation by Manifolds of Finite Pseudo-Dimension , 1999 .

[42]  Zongben Xu,et al.  Learning Rates of lq Coefficient Regularization Learning with Gaussian Kernel , 2013, Neural Computation.

[43]  Stefan Kunis,et al.  Probabilistic spherical Marcinkiewicz-Zygmund inequalities , 2009, J. Approx. Theory.

[44]  Ding-Xuan Zhou,et al.  Learning with sample dependent hypothesis spaces , 2008, Comput. Math. Appl..