Specification of Training Sets and the Number of Hidden Neurons for Multilayer Perceptrons

This work concerns the selection of input-output pairs for improved training of multilayer perceptrons, in the context of approximation of univariate real functions. A criterion for the choice of the number of neurons in the hidden layer is also provided. The main idea is based on the fact that Chebyshev polynomials can provide approximations to bounded functions up to a prescribed tolerance, and, in turn, a polynomial of a certain order can be fitted with a three-layer perceptron with a prescribed number of hidden neurons. The results are applied to a sensor identification example.

[1]  Tsu-Tian Lee,et al.  The Chebyshev-polynomials-based unified model neural networks for function approximation , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[2]  A. Barron.  Approximation and Estimation Bounds for Artificial Neural Networks , 1991, COLT '91.

[3]  Thomas Elsken,et al.  Even on Finite Test Sets Smaller Nets may Perform Better , 1997, Neural Networks.

[4]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[5]  Hong Chen,et al.  Approximation capability in C(R¯n) by multilayer feedforward networks and related problems , 1995, IEEE Trans. Neural Networks.

[6]  Krishna Chintalapudi,et al.  A novel scheme to determine the architecture of a multilayer perceptron , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[7]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[8]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[9]  J. Miller.  Numerical Analysis , 1966, Nature.

[10]  O. Schuck,et al.  Measurement systems - Application and design [Book Reviews] , 1968, IEEE Transactions on Automatic Control.

[11]  Marios M. Polycarpou,et al.  An analytical framework for local feedforward networks , 1998, IEEE Trans. Neural Networks.

[12]  Vasileios Basios,et al.  A method for approximating one-dimensional functions , 1997 .

[13]  K. W. Lee,et al.  Optimal sizing of feedforward neural networks: Case studies , 1995, Proceedings 1995 Second New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems.

[14]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[15]  Andrei Andreev,et al.  Approximation and interpolation theory , 1994 .

[16]  M. J. Maron,et al.  Numerical Analysis: A Practical Approach , 1982 .

[17]  Arnold Neumaier,et al.  Introduction to Numerical Analysis , 2001 .

[18]  Shin Suzuki,et al.  Constructive function-approximation by three-layer artificial neural networks , 1998, Neural Networks.

[19]  Stephan Rudolph,et al.  On topology, size and generalization of non-linear feed-forward neural networks , 1997, Neurocomputing.