LP Approximation Capabilities of Sum-of-Product and Sigma-pi-Sigma Neural Networks

This paper studies the L(p) approximation capabilities of sum-of-product (SOPNN) and sigma-pi-sigma (SPSNN) neural networks. It is proved that the set of functions that are generated by the SOPNN with its activation function in $L_{loc};p(\mathcal{R})$ is dense in $L;p(\mathcal{K})$ for any compact set $\mathcal{K}\subset \mathcal{R};N$, if and only if the activation function is not a polynomial almost everywhere. It is also shown that if the activation function of the SPSNN is in ${L_{loc};\infty(\mathcal{R})}$, then the functions generated by the SPSNN are dense in $L;p(\mathcal{K})$ if and only if the activation function is not a constant (a.e.).

[1]  Shi-Yi Shen,et al.  Lp approximation of Sigma-Pi neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[2]  Chien-Kuo Li.  A Sigma-Pi-Sigma Neural Network (SPSNN) , 2004, Neural Processing Letters.

[3]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[4]  Ward Cheney,et al.  A course in approximation theory , 1999 .

[5]  Chun-Shin Lin,et al.  A sum-of-product neural network (SOPNN) , 2000, Neurocomputing.

[6]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[7]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[8]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[9]  Pierre Courrieu,et al.  Function approximation on non-Euclidean spaces , 2005, Neural Networks.

[10]  Gilles Pagès,et al.  Approximations of Functions by a Multilayer Perceptron: a New Approach , 1997, Neural Networks.

[11]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[12]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[13]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..