Pruning strategies for the MTiling constructive learning algorithm

We present a framework for incorporating pruning strategies in the MTiling constructive neural network learning algorithm. Pruning involves elimination of redundant elements (connection weights or neurons) from a network and is of considerable practical interest. We describe three elementary sensitivity based strategies for pruning neurons. Experimental results demonstrate a moderate to significant reduction in the network size without compromising the network's generalization performance.

[1]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[2]  M. Golea,et al.  A Convergence Theorem for Sequential Learning in Two-Layer Perceptrons , 1990 .

[3]  Neil Burgess,et al.  A Constructive Algorithm that Converges for Real-Valued Input Patterns , 1994, Int. J. Neural Syst..

[4]  Marcus R. Frean,et al.  A "Thermal" Perceptron Learning Rule , 1992, Neural Computation.

[5]  Vasant Honavar,et al.  Generatlve Learning Structures and Processes for Connectionist Networks , 1993 .

[6]  Stephen I. Gallant,et al.  Neural network learning and expert systems , 1993 .

[7]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Vasant Honavar,et al.  Generative learning structures and processes for generalized connectionist networks , 1993, Inf. Sci..

[10]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[11]  Vasant G Honavar,et al.  MTiling A Constructive Neural Network Learning Algorithm for Multi Category Pattern Classi cation , 1996 .

[12]  Jihoon Yang,et al.  Constructive Neural Network Learning Algorithms , 1996, AAAI/IAAI, Vol. 2.

[13]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[14]  Rajesh Parekh,et al.  Constructive Neural Network Learning Algorithms for Multi-Category Pattern Classification , 1995 .