Essential rate for approximation by spherical neural networks

We consider the optimal rate of approximation by single hidden feed-forward neural networks on the unit sphere. It is proved that there exists a neural network with n neurons, and an analytic, strictly increasing, sigmoidal activation function such that the deviation of a Sobolev class W²(2r)(S(d)) from the class of neural networks Φ(n)(ϕ), behaves asymptotically as n(-2r/d-1). Namely, we prove that the essential rate of approximation by spherical neural networks is n(-2r/d-1).

[1]  Vitaly Maiorov,et al.  On best approximation of classes by radial functions , 2003, J. Approx. Theory.

[2]  Y. Makovoz.  Uniform Approximation by Neural Networks , 1998 .

[3]  Holger Wendland,et al.  Direct and Inverse Sobolev Error Estimates for Scattered Data Interpolation via Spherical Basis Functions , 2007, Found. Comput. Math..

[4]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[5]  Ron Meir,et al.  Approximation bounds for smooth functions in C(Rd) by neural and mixture networks , 1998, IEEE Trans. Neural Networks.

[6]  Hrushikesh Narhar Mhaskar,et al.  Zonal function network frames on the sphere , 2003, Neural Networks.

[7]  V. Maiorov.  On Best Approximation by Ridge Functions , 1999 .

[8]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[9]  Willi Freeden,et al.  Multiscale Gravitational Field Recovery from GPS-Satellite-to-Satellite Tracking , 1999 .

[10]  Nahmwoo Hahm,et al.  Degree of approximation by neural networks , 1996 .

[11]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[12]  Dany Leviatan,et al.  Approximation of Sobolev classes by polynomials and ridge functions , 2009, J. Approx. Theory.

[13]  P. Petrushev.  Approximation by ridge functions and neural networks , 1999 .

[14]  Allan Pinkus,et al.  Lower bounds for approximation by MLP neural networks , 1999, Neurocomputing.

[15]  Shin Suzuki,et al.  Constructive function-approximation by three-layer artificial neural networks , 1998, Neural Networks.

[16]  Hrushikesh Narhar Mhaskar,et al.  Approximation properties of zonal function networks using scattered data on the sphere , 1999, Adv. Comput. Math..

[17]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[18]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[19]  C. Chui,et al.  Approximation by ridge functions and neural networks with one hidden layer , 1992 .

[20]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[21]  Willi Freeden,et al.  Satellite-to-satellite tracking and satellite gravity gradiometry (Advanced techniques for high-resolution geopotential field determination) , 2002 .

[22]  Dany Leviatan,et al.  Approximation by polynomials and ridge functions of classes of s-monotone radial functions , 2008, J. Approx. Theory.

[23]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[24]  C. Micchelli,et al.  Degree of Approximation by Neural and Translation Networks with a Single Hidden Layer , 1995 .

[25]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[26]  Willi Freeden,et al.  Spherical Tikhonov regularization wavelets in satellite gravity gradiometry with random noise , 2001 .

[27]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[28]  H. N. Mhaskar,et al.  Neural Networks for Optimal Approximation of Smooth and Analytic Functions , 1996, Neural Computation.

[29]  Abhay B. Bulsari,et al.  Some analytical solutions to the general approximation problem for feedforward neural networks , 1993, Neural Networks.

[30]  Feng Dai,et al.  Jackson inequality for Banach spaces on the sphere , 2008 .

[31]  Feilong Cao,et al.  The errors in simultaneous approximation by feed-forward neural networks , 2010, Neurocomputing.

[32]  П. И. ЛИЗОРКИН,et al.  A theorem concerning approximation on the sphere , 1983 .

[33]  Quoc Thong Le Gia,et al.  Localized Linear Polynomial Operators and Quadrature Formulas on the Sphere , 2008, SIAM J. Numer. Anal..

[34]  Philip Crotwell.  Constructive Approximation on the Sphere , 2000 .

[35]  Xingping Sun,et al.  Fundamental sets of continuous functions on spheres , 1997 .

[36]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[37]  XUZongben,et al.  The essential order of approximation for neural networks , 2004 .

[38]  Siegfried Pawelke,et al.  ÜBER DIE APPROXIMATIONSORDNUNG BEI KUGELFUNKTIONEN UND ALGEBRAISCIHEN POLYNOMEN , 1972 .

[39]  Hrushikesh Narhar Mhaskar,et al.  Weighted quadrature formulas and approximation by zonal function networks on the sphere , 2006, J. Complex..