The errors of simultaneous approximation of multivariate functions by neural networks

There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth multivariate function and all of its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated.

[1]  XUZongben,et al.  The essential order of approximation for neural networks , 2004 .

[2]  Zongben Xu,et al.  Simultaneous Lp-approximation order for neural networks , 2005, Neural Networks.

[3]  George A. Anastassiou,et al.  Rate of convergence of some neural network operators to the unit-univariate case, revisited , 2013 .

[4]  Ron Meir,et al.  Approximation bounds for smooth functions in C(Rd) by neural and mixture networks , 1998, IEEE Trans. Neural Networks.

[5]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[6]  G. Lorentz.  Approximation of Functions , 1966 .

[7]  Gilles Pagès,et al.  Approximations of Functions by a Multilayer Perceptron: a New Approach , 1997, Neural Networks.

[8]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[9]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[10]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[11]  C. Chui,et al.  Approximation by ridge functions and neural networks with one hidden layer , 1992 .

[12]  Y. Makovoz.  Uniform Approximation by Neural Networks , 1998 .

[13]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[14]  Pierre Cardaliaguet,et al.  Approximation of a function and its derivative with a neural network , 1992, Neural Networks.

[15]  George A. Anastassiou.  Rate of Convergence of Basic Multivariate Neural Network Operators to the Unit , 2000 .

[16]  P. Heywood.  Trigonometric Series , 1968, Nature.

[17]  Ward Cheney,et al.  A course in approximation theory , 1999 .

[18]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[19]  Xin Li,et al.  Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer , 1996, Neurocomputing.

[20]  Feilong Cao,et al.  The errors in simultaneous approximation by feed-forward neural networks , 2010, Neurocomputing.

[21]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[22]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[23]  George A. Anastassiou,et al.  Rate of Convergence of Basic Neural Network Operators to the Unit-Univariate Case , 1997 .

[24]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[25]  Le Baron O. Ferguson,et al.  Approximation by Polynomials with Integral Coe cients , 1980 .

[26]  Halbert White,et al.  On learning the derivatives of an unknown mapping with multilayer feedforward networks , 1992, Neural Networks.

[27]  Zongben Xu,et al.  The estimate for approximation error of neural networks: A constructive approach , 2008, Neurocomputing.

[28]  C. Micchelli,et al.  Degree of Approximation by Neural and Translation Networks with a Single Hidden Layer , 1995 .

[29]  Xin Li,et al.  On simultaneous approximations by radial basis function neural networks , 1998, Appl. Math. Comput..

[30]  Nahmwoo Hahm,et al.  Degree of approximation by neural networks , 1996 .

[31]  Shin Suzuki,et al.  Constructive function-approximation by three-layer artificial neural networks , 1998, Neural Networks.

[32]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[33]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.