论文信息 - Convergence Rate of Minimization Learning for Neural Networks

Convergence Rate of Minimization Learning for Neural Networks

In this paper, we present the convergence rate of the error in a neural network which was learnt by a constructive method. The constructive mechanism is used to learn the neural network by adding hidden units to this neural network. The main idea of this work is to find the eigenvalues of the transformation matrix concerning the error before and after adding hidden units in the neural network. By using the eigenvalues, we show the relation between the convergence rate in neural networks without and with thresholds in the output layer.

Teruya Minamoto | Marghny H. Mohamed | Koichi Niijima

[1] Jean-Pierre Martens,et al. On the initialization and optimization of multilayer perceptrons , 1994, IEEE Trans. Neural Networks.

[2] Michael C. Mozer,et al. Using Relevance to Reduce Network Size Automatically , 1989 .

[3] Mark S. Nixon,et al. Generating-shrinking algorithm for learning arbitrary classification , 1994, Neural Networks.

[4] Jenq-Neng Hwang,et al. Regression modeling in back-propagation and projection pursuit learning , 1994, IEEE Trans. Neural Networks.

[5] Malcolm I. Heywood,et al. A framework for improved training of Sigma-Pi networks , 1995, IEEE Trans. Neural Networks.

[6] Bernd Fritzke,et al. Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[7] Albert Y. Zomaya,et al. Toward generating neural network structures for function approximation , 1994, Neural Networks.

[8] Babak Hassibi,et al. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[9] C. Lee Giles,et al. Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution , 1995, IEEE Trans. Neural Networks.