Size of Multilayer Networks for Exact Learning: Analytic Approach

This article presents a new result about the size of a multilayer neural network computing real outputs for exact learning of a finite set of real samples. The architecture of the network is feedforward, with one hidden layer and several outputs. Starting from a fixed training set, we consider the network as a function of its weights. We derive, for a wide family of transfer functions, a lower and an upper bound on the number of hidden units for exact learning, given the size of the dataset and the dimensions of the input and output spaces.

[1]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[2]  Eduardo D. Sontag,et al.  Feedforward Nets for Interpolation and Classification , 1992, J. Comput. Syst. Sci..

[3]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[4]  Wolfgang Maass.  Bounds for the Computational Power and Learning Complexity of Analog Neural Nets , 1997, SIAM J. Comput..

[5]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[6]  Marek Karpinski,et al.  Polynomial bounds for VC dimension of sigmoidal neural networks , 1995, STOC '95.

[7]  Eduardo D. Sontag,et al.  Neural Networks with Quadratic VC Dimension , 1995, J. Comput. Syst. Sci..

[8]  Michel Cosnard,et al.  Bounds on the Number of Units for Computing Arbitrary Dichotomies by Multilayer Perceptrons , 1994, J. Complex..

[9]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[10]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[11]  Edward K. Blum,et al.  Approximation theory and feedforward networks , 1991, Neural Networks.

[12]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[13]  Eduardo D. Sontag,et al.  Shattering All Sets of k Points in General Position Requires (k 1)/2 Parameters , 1997, Neural Computation.

[14]  Virginia L. Stonick,et al.  Topology and Geometry of Single Hidden Layer Network, Least Squares Weight Solutions , 1995, Neural Computation.