Neural network dimension selection for dynamical system identification

Choosing an appropriate size of a network is an important issue for any neural network applications. The common practice is to start with an ldquoover-sizedrdquo network, then gradually reduces its size to find the optimal solution. In this paper, a new hybrid neural network pruning algorithm for multi-layer feedforward neural networks is investigated. Computer simulation results on system identification and pattern classification problems show this algorithm can significantly reduce the network dimension while still maintaining satisfactory identification and classification accuracy.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  C. Lee Giles,et al.  Overfitting and neural networks: conjugate gradient and backpropagation , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[3]  F. Fnaiech,et al.  A modified version of a formal pruning algorithm based on local relative variance analysis , 2004, First International Symposium on Control, Communications and Signal Processing, 2004..

[4]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[5]  Mohamed Cheriet,et al.  A new feedforward neural network pruning algorithm: SSM-iterative pruning (SSMIP) , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[6]  R. Setiono,et al.  Effective neural network pruning using cross-validation , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[7]  Okyay Kaynak,et al.  An algorithm for fast convergence in training neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[8]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[9]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[10]  Mohamed Najim,et al.  A new feedforward neural network hidden layer neuron pruning algorithm , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[11]  K. C. Ho,et al.  A formal selection and pruning algorithm for feedforward artificial neural network optimization , 1999, IEEE Trans. Neural Networks.

[12]  Andries Petrus Engelbrecht,et al.  A new pruning heuristic based on variance analysis of sensitivity information , 2001, IEEE Trans. Neural Networks.

[13]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[14]  Mohamed Najim,et al.  A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm , 2001, IEEE Trans. Neural Networks.

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .