Neural subnet design by direct polynomial mapping

In a recent paper by M. Chen and M. Maury (1990), it was shown that multilayer perceptron neural networks can be used to form products of any number of inputs, thereby constructively proving universal approximation. This result is extended, and a method for the analysis and synthesis of single-input, single-output neural subnetworks is described. Given training samples of a function to be approximated, a feedforward neural network is designed which implements a polynomial approximation of the function with arbitrary accuracy. For comparison, example subnets are designed by classical backpropagation training and by mapping. The examples illustrate that the mapped subnets avoid local minima which backpropagation-trained subnets get trapped in and that the mapping approach is much faster.

[1]  Michael T. Manry,et al.  Iterative improvement of a nearest neighbor classifier , 1991, Neural Networks.

[2]  Michael T. Manry,et al.  Basis vector analyses of back-propagation neural networks , 1991, [1991] Proceedings of the 34th Midwest Symposium on Circuits and Systems.

[3]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[4]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[5]  Michael T. Manry,et al.  Backpropagation representation theorem using power series , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[6]  Michael T. Manry,et al.  Iterative improvement of a Gaussian classifier , 1990, Neural Networks.

[7]  Michael T. Manry,et al.  The design of multi-layer perceptrons using building blocks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[8]  I. W. Sandberg.  Approximation theorems for discrete-time systems , 1991 .

[9]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[10]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[11]  Alex Waibel,et al.  Consonant recognition by modular construction of large phonemic time-delay neural networks , 1989, International Conference on Acoustics, Speech, and Signal Processing,.