The errors in simultaneous approximation by feed-forward neural networks

There have been many studies on the simultaneous approximation capability of feed-forward neural networks (FNNs). Most of these, however, are only concerned with the density or feasibility of performing simultaneous approximations. This paper considers the simultaneous approximation of algebraic polynomials, employing Taylor expansion and an algebraic constructive approach, to construct a class of FNNs which realize the simultaneous approximation of any smooth univariate function and all its derivatives. We also present an upper bound on the approximation accuracy of the FNNs, expressed in terms of the modulus of continuity of the functions to be approximated. The results obtained in this paper reveal the complexity of simultaneous approximation by FNNs.

[1]  Zongben Xu,et al.  Simultaneous Lp-approximation order for neural networks , 2005, Neural Networks.

[2]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[3]  George A. Anastassiou,et al.  Rate of Convergence of Basic Neural Network Operators to the Unit-Univariate Case , 1997 .

[4]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[5]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[6]  C. Chui,et al.  Approximation by ridge functions and neural networks with one hidden layer , 1992 .

[7]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[8]  Xin Li,et al.  On simultaneous approximations by radial basis function neural networks , 1998, Appl. Math. Comput..

[9]  C. Micchelli,et al.  Degree of Approximation by Neural and Translation Networks with a Single Hidden Layer , 1995 .

[10]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[11]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[12]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[13]  H. N. Mhaskar,et al.  Neural Networks for Optimal Approximation of Smooth and Analytic Functions , 1996, Neural Computation.

[14]  Ron Meir,et al.  Approximation bounds for smooth functions in C(Rd) by neural and mixture networks , 1998, IEEE Trans. Neural Networks.

[15]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[16]  Zongben Xu,et al.  The essential order of approximation for neural networks , 2004, Science in China Series F: Information Sciences.

[17]  Y. Makovoz.  Uniform Approximation by Neural Networks , 1998 .

[18]  Shin Suzuki,et al.  Constructive function-approximation by three-layer artificial neural networks , 1998, Neural Networks.

[19]  Bum Il Hong,et al.  An approximation by neural networkswith a fixed weight , 2004 .

[20]  Halbert White,et al.  On learning the derivatives of an unknown mapping with multilayer feedforward networks , 1992, Neural Networks.

[21]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[22]  Xin Li,et al.  Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer , 1996, Neurocomputing.

[23]  Pierre Cardaliaguet,et al.  Approximation of a function and its derivative with a neural network , 1992, Neural Networks.

[24]  George A. Anastassiou.  Rate of Convergence of Basic Multivariate Neural Network Operators to the Unit , 2000 .

[25]  Zongben Xu,et al.  The estimate for approximation error of neural networks: A constructive approach , 2008, Neurocomputing.

[26]  George A. Anastassiou,et al.  Univariate fuzzy-random neural network approximation operators , 2004 .