Linear and nonlinear approximation of spherical radial basis function networks

In this paper, the center-selection strategy of spherical radial basis function networks (SRBFNs) is considered. To approximate functions in the Bessel-potential Sobolev classes, we provide two lower bounds of nonlinear SRBFN approximation. In the first one, we prove that, up to a logarithmic factor, the lower bound of SRBFN approximation coincides with the Kolmogorov n -width. In the other one, we prove that if a pseudo-dimension assumption is imposed on the activation function, then the logarithmic factor can even be omitted. These results together with the well known Jackson-type inequality of SRBFN approximation imply that the center-selection strategy does not affect the approximation capability of SRBFNs very much, provided the target function belongs to the Bessel-potential Sobolev classes. Thus, we can choose centers only for the algorithmic factor. Hence, a linear SRBFN approximant whose centers are specified before the training is recommended.

[1]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[2]  Zongben Xu,et al.  Essential rate for approximation by spherical neural networks , 2011, Neural Networks.

[3]  Marian Neamtu,et al.  Error Bounds for Solving Pseudodifferential Equations on Spheres by Collocation with Zonal Kernels , 2002, J. Approx. Theory.

[4]  Zongben Xu,et al.  Jackson-type inequalities for spherical neural networks with doubling weights , 2015, Neural Networks.

[5]  Marcello Sanguineti,et al.  Comparison of worst case errors in linear and neural network approximation , 2002, IEEE Trans. Inf. Theory.

[6]  Dai Feng,et al.  Kolmogorov Width of Classes of Smooth Functions on the Sphere Sd-1 , 2002, J. Complex..

[7]  Vitaly Maiorov,et al.  Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary , 2006, Adv. Comput. Math..

[8]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[9]  Hrushikesh Narhar Mhaskar,et al.  Approximation properties of zonal function networks using scattered data on the sphere , 1999, Adv. Comput. Math..

[10]  A. Barron,et al.  Approximation and learning by greedy algorithms , 2008, 0803.1718.

[11]  E. Saff,et al.  Distributing many points on a sphere , 1997 .

[12]  Ron Meir,et al.  Lower bounds for multivariate approximation by affine-invariant dictionaries , 2001, IEEE Trans. Inf. Theory.

[13]  Xingping Sun,et al.  Fundamental sets of continuous functions on spheres , 1997 .

[14]  Heping Wang,et al.  Probabilistic and Average Widths of Sobolev Spaces on Compact Two-Point Homogeneous Spaces Equipped with a Gaussian Measure , 2014 .

[15]  Martin Burger,et al.  Error Bounds for Approximation with Neural Networks , 2001, J. Approx. Theory.

[16]  Hrushikesh Narhar Mhaskar,et al.  L BERNSTEIN ESTIMATES AND APPROXIMATION BY SPHERICAL BASIS FUNCTIONS , 2010 .

[17]  Zen-Chung Shih,et al.  All-frequency precomputed radiance transfer using spherical radial basis functions and clustered tensor approximation , 2006, ACM Trans. Graph..

[18]  Allan Pinkus,et al.  Lower bounds for approximation by MLP neural networks , 1999, Neurocomputing.

[19]  V. Maiorov On Best Approximation by Ridge Functions , 1999 .

[20]  Joseph D. Ward,et al.  Scattered Data Interpolation on Spheres: Error Estimates and Locally Supported Basis Functions , 2002, SIAM J. Math. Anal..

[21]  V. A. Menegatto,et al.  A necessary and sufficient condition for strictly positive definite functions on spheres , 2003 .

[22]  David Haussler,et al.  Sphere Packing Numbers for Subsets of the Boolean n-Cube with Bounded Vapnik-Chervonenkis Dimension , 1995, J. Comb. Theory, Ser. A.

[23]  Holger Wendland,et al.  Direct and Inverse Sobolev Error Estimates for Scattered Data Interpolation via Spherical Basis Functions , 2007, Found. Comput. Math..

[24]  Zhenzhong Chen,et al.  Spherical basis functions and uniform distribution of points on spheres , 2008, J. Approx. Theory.

[25]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[26]  A. Pinkus n-Widths in Approximation Theory , 1985 .

[27]  Vitaly Maiorov Almost Optimal Estimates for Best Approximation by Translates on a Torus , 2005 .

[28]  Philip Crotwell Constructive Approximation on the Sphere , 2000 .

[29]  Vitaly Maiorov,et al.  On best approximation of classes by radial functions , 2003, J. Approx. Theory.

[30]  Zongben Xu,et al.  A general radial quasi-interpolation operator on the sphere , 2012, J. Approx. Theory.

[31]  Joel Ratsaby,et al.  On the Degree of Approximation by Manifolds of Finite Pseudo-Dimension , 1999 .

[32]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[33]  Jeremy Levesley,et al.  Approximation in rough native spaces by shifts of smooth kernels on spheres , 2005, J. Approx. Theory.

[34]  Volker Michel,et al.  A non-linear approximation method on the sphere , 2014 .

[35]  Holger Wendland,et al.  Continuous and discrete least-squares approximation by radial basis functions on spheres , 2006, J. Approx. Theory.

[36]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[37]  E. Saff,et al.  Asymptotics for minimal discrete energy on the sphere , 1995 .