Determination of the number of redundant hidden units in a three-layered feedforward neural network

Determination of the number of redundant hidden units in a three-layered feedforward neural network trained on a learning data set is described. For this purpose, a linear equation, OW=t, which describes the three-layered feedforward neural network mapping for the training data set is introduced. It is shown that, if rank of the matrix, O, is not full-rank, we can remove "the number of hidden units minus the rank of O plus one" hidden units from the network without any increase of the error of the network for the training data. It is also shown that by using singular value decomposition this approach can be applicable to a full-rank matrix O with little increase of error. Computer experiments show the effectiveness of the approach.

[1]  S. Tamura,et al.  Capabilities of a three layer feedforward neural network , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[2]  Panos J. Antsaklis,et al.  A simple method to derive bounds on the size and to train multilayer neural networks , 1991, IEEE Trans. Neural Networks.

[3]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .