Neural networks for localized approximation

We prove that feedforward artificial neural networks with a single hidden layer and an ideal sigmoidal response function cannot provide localized approximation in a Euclidean space of dimension higher than one. We also show that networks with two hidden layers can be designed to provide localized approximation. Since wavelet bases are most effective for local approximation, we give a discussion of the implementation of spline wavelets using multilayered networks where the response function is a sigmoidal function of order at least two.

[1]  C. Micchelli,et al.  Some remarks on ridge functions , 1987 .

[2]  I. J. Schoenberg.  Cardinal Spline Interpolation , 1987 .

[3]  B. Irie,et al.  Capabilities of three-layered perceptrons , 1988, IEEE 1988 International Conference on Neural Networks.

[4]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[5]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[6]  S. M. Carroll,et al.  Construction of neural nets using the radon transform , 1989, International 1989 Joint Conference on Neural Networks.

[7]  H. White,et al.  Universal approximation using feedforward networks with non-sigmoid hidden layer activation functions , 1989, International 1989 Joint Conference on Neural Networks.

[8]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[9]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[10]  Yoshifusa Ito,et al.  Representation of functions by superpositions of a step or sigmoid function and their applications to neural network theory , 1991, Neural Networks.

[11]  Yoshifusa Ito,et al.  Approximation of functions on a compact set by finite sums of a sigmoid function without scaling , 1991, Neural Networks.

[12]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[13]  C. Chui,et al.  On compactly supported spline wavelets and a duality principle , 1992 .

[14]  C. Chui,et al.  Approximation by ridge functions and neural networks with one hidden layer , 1992 .

[15]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[16]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[17]  Hrushikesh Narhar Mhaskar,et al.  Approximation properties of a multilayered feedforward artificial neural network , 1993, Adv. Comput. Math..

[18]  Xin Li,et al.  Realization of Neural Networks with One Hidden Layer , 1993 .