论文信息 - Analysis of Pruning in Backpropagation Networks for Artificial and Real Worls Mapping Problems

Analysis of Pruning in Backpropagation Networks for Artificial and Real Worls Mapping Problems

In this study, the properties of hidden nodes in backpropagatibn networks with respect to their contribution to the solution of the problem after initial training are examined. Using a pruning method, redundant nodes are removed and weights are redistributed in the pruned network. After reducing the network size, additional retraining may not be needed and the pruned network's generalization performance improves. The results show that the removal of one specific category of redundant nodes leads to improvement in terms of pruned network size, retraining speed, and improved generalization performance. An additional implication is that hidden nodes in a trained network are not evenly fault tolerant and that by changing the order of removal of different categories of redundant nodes we may find different solutions to the problem.

Hean-Lee Poh | Teo Jasic

[1] Lorien Y. Pratt,et al. Comparing Biases for Minimal Network Construction with Back-Propagation , 1988, NIPS.

[2] James L. McClelland,et al. Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[3] Yann LeCun,et al. Optimal Brain Damage , 1989, NIPS.

[4] Yoh-Han Pao,et al. Adaptive pattern recognition and neural networks , 1989 .

[5] Ken-ichi Funahashi,et al. On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[6] Korris Fu-Lai Chung,et al. A Node Pruning Algorithm for Backpropagation Networks , 1992, Int. J. Neural Syst..

[7] David Haussler,et al. What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[8] Michael C. Mozer,et al. Using Relevance to Reduce Network Size Automatically , 1989 .

[9] Kurt Hornik,et al. Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[10] Michael C. Mozer,et al. Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[11] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .

[12] Jocelyn Sietsma,et al. Creating artificial neural networks that generalize , 1991, Neural Networks.