Neural Networks and Fuzzy Basis Functions for Functional Approximation

Universal approximation capabilities of neural networks and fuzzy basis functions are given in this chapter using the Stone-Weierstrass theorem, Kolmogorov’s theorem and functional analysis methods. This study focuses on few commonly-used neural networks such as multilayered feedforward neural networks (MFNNs) with sigmoidal activation functions, trigonometric networks, higher-order neural networks, Gaussian radial basis function networks, and fuzzy basis function networks. The results show that an arbitrary continuous function on a compact set may be approximated to any degree of accuracy by such a neural network or fuzzy system. However, the accuracy of the approximations is strongly related to the design of the learning phases of the network parameters. The theory presented in this chapter provides a theoretical basis for applications to the fields of identification, control and pattern recognition.

[1]  Madan M. Gupta,et al.  Introduction to Fuzzy Arithmetic , 1991 .

[2]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[3]  Joydeep Ghosh,et al.  Approximation of multivariate functions using ridge polynomial networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  D.R. Hush,et al.  Progress in supervised neural networks , 1993, IEEE Signal Processing Magazine.

[6]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[7]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[8]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[9]  Joydeep Ghosh,et al.  The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[10]  L X Wang,et al.  Fuzzy basis functions, universal approximation, and orthogonal least-squares learning , 1992, IEEE Trans. Neural Networks.

[11]  H. White,et al.  There exists a neural network that does not make avoidable mistakes , 1988, IEEE 1988 International Conference on Neural Networks.

[12]  G. Lorentz.  Approximation of Functions , 1966 .

[13]  Sukhan Lee,et al.  A Gaussian potential function network with hierarchically self-organizing learning , 1991, Neural Networks.

[14]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[15]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[16]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[17]  David A. Sprecher,et al.  A universal mapping for kolmogorov's superposition theorem , 1993, Neural Networks.

[18]  Neil E. Cotter,et al.  The Stone-Weierstrass theorem and its application to neural networks , 1990, IEEE Trans. Neural Networks.

[19]  R. Hecht-Nielsen,et al.  Theory of the Back Propagation Neural Network , 1989 .

[20]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[21]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[22]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[23]  Edward K. Blum,et al.  Approximation theory and feedforward networks , 1991, Neural Networks.

[24]  Y. Ito,et al.  Extension of approximation capability of three layered neural networks to derivatives , 1993, IEEE International Conference on Neural Networks.

[25]  Vera Kurková,et al.  Kolmogorov's theorem and multilayer neural networks , 1992, Neural Networks.

[26]  Yoshifusa Ito,et al.  Approximation of functions on a compact set by finite sums of a sigmoid function without scaling , 1991, Neural Networks.

[27]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[28]  S. M. Carroll,et al.  Construction of neural nets using the radon transform , 1989, International 1989 Joint Conference on Neural Networks.

[29]  D. Sprecher.  On the structure of continuous functions of several variables , 1965 .

[30]  Daniel M. Kammen,et al.  Correlations in high dimensional or asymmetric data sets: Hebbian neuronal processing , 1991, Neural Networks.

[31]  Pierre Cardaliaguet,et al.  Approximation of a function and its derivative with a neural network , 1992, Neural Networks.

[32]  Halbert White,et al.  On learning the derivatives of an unknown mapping with multilayer feedforward networks , 1992, Neural Networks.

[33]  Stephen Coombes,et al.  Learning higher order correlations , 1993, Neural Networks.