The construction and approximation of feedforward neural network with hyperbolic tangent function

In this paper, we discuss some analytic properties of hyperbolic tangent function and estimate some approximation errors of neural network operators with the hyperbolic tangent activation function. Firstly, an equation of partitions of unity for the hyperbolic tangent function is given. Then, two kinds of quasi-interpolation type neural network operators are constructed to approximate univariate and bivariate functions, respectively. Also, the errors of the approximation are estimated by means of the modulus of continuity of function. Moreover, for approximated functions with high order derivatives, the approximation errors of the constructed operators are estimated.

[1]  Y. Makovoz.  Uniform Approximation by Neural Networks , 1998 .

[2]  George A. Anastassiou,et al.  Multivariate hyperbolic tangent neural network approximation , 2011, Comput. Math. Appl..

[3]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[4]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[5]  Yuan Xu,et al.  Univariant approximation by superpositions of a sigmoidal function , 1993 .

[6]  Feilong Cao,et al.  The approximation operators with sigmoidal functions , 2009, Comput. Math. Appl..

[7]  Renato Spigler,et al.  Approximation results for neural network operators activated by sigmoidal functions , 2013, Neural Networks.

[8]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[9]  Chen Debao.  Degree of approximation by superpositions of a sigmoidal function , 1993, Approximation Theory and its Applications.

[10]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[11]  Hong Chen,et al.  Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks , 1993, IEEE Trans. Neural Networks.

[12]  Renato Spigler,et al.  Multivariate neural network operators with sigmoidal activation functions , 2013, Neural Networks.

[13]  Hong Chen,et al.  Approximation capability in C(R¯n) by multilayer feedforward networks and related problems , 1995, IEEE Trans. Neural Networks.

[14]  George A. Anastassiou,et al.  Intelligent Systems: Approximation by Artificial Neural Networks , 2011, Intelligent Systems Reference Library.

[15]  George A. Anastassiou,et al.  Rate of Convergence of Basic Neural Network Operators to the Unit-Univariate Case , 1997 .

[16]  D. Costarelli,et al.  Constructive Approximation by Superposition of Sigmoidal Functions , 2013 .

[17]  Zongben Xu,et al.  The estimate for approximation error of neural networks: A constructive approach , 2008, Neurocomputing.

[18]  George A. Anastassiou,et al.  Multivariate sigmoidal neural network approximation , 2011, Neural Networks.

[19]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[20]  Danilo Costarelli,et al.  Interpolation by neural network operators activated by ramp functions , 2014 .

[21]  Allan Pinkus,et al.  Approximation theory of the MLP model in neural networks , 1999, Acta Numerica.

[22]  George A. Anastassiou,et al.  Univariate hyperbolic tangent neural network approximation , 2011, Math. Comput. Model..

[23]  C. Chui,et al.  Approximation by ridge functions and neural networks with one hidden layer , 1992 .

[24]  Renato Spigler,et al.  Convergence of a family of neural network operators of the Kantorovich type , 2014, J. Approx. Theory.

[25]  Paul C. Kainen,et al.  An Integral Upper Bound for Neural Network Approximation , 2009, Neural Computation.