A new feedforward neural network pruning algorithm: SSM-iterative pruning (SSMIP)

We present a new pruning algorithm based on a combination of the Statistical Stepwise Method (SSM) (Cottrell et al., 1995) and the Iterative Pruning (IP) algorithms (Castellano and Fanelli, 1997). The proposed algorithm (SSMIP) is used to prune feedforward neural networks (NN) and combines the advantages of the algorithms. The new proposed pruning algorithm works as follows: i) The Iterative Pruning Algorithm of Castellano et al. is applied in order to cancel no significant units; ii) The Statistical Stepwise Method of Cottrell et al. is then used to remove unnecessary links between units. These steps are repeated layer by layer in the forward direction with holding the previous optimised structure in each layer. Simulation results are given on standard benchmarks that highlight the effectiveness of the proposed algorithm. A comparative study is given with respect to Castelanno and Cottrell algorithms used separately.

[1]  Faouzi Kossentini,et al.  Feedforward neural network design with tridiagonal symmetry constraints , 2000, IEEE Trans. Signal Process..

[2]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[3]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[4]  Mohamed Najim,et al.  A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm , 2001, IEEE Trans. Neural Networks.

[5]  Mohamed Najim,et al.  A new feedforward neural network hidden layer neuron pruning algorithm , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[6]  Giovanna Castellano,et al.  An iterative pruning algorithm for feedforward neural networks , 1997, IEEE Trans. Neural Networks.

[7]  Marie Cottrell,et al.  Neural modeling for time series: A statistical stepwise method for weight elimination , 1995, IEEE Trans. Neural Networks.

[8]  Gang Liu,et al.  A*Prune: an algorithm for finding K shortest paths subject to multiple constraints , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).