论文信息 - A Genetic-Algorithm-Based Weight Discretization Paradigm for Neural Networks

A Genetic-Algorithm-Based Weight Discretization Paradigm for Neural Networks

The traditional neural networks with continuous weights easy to implement in software might often be very burdensome in the embedded hardware systems and therefore more costly. Hardware-friendly neural networks are essential to ensure the functionality and effectiveness of the embedded implementation. To achieve this aim, A GA-based algorithm for training neural networks with discrete weights and quantized on-linear activation function is presented in this paper. The performance of this procedure is evaluated by comparing it with multi-threshold method and continuous discrete learning method based on computing the gradient of the error function, and the simulation results show this new learning algorithm outperforms the other two greatly in convergence and generalization.

Bin Zhou | Yi Yan | Jian Bao

[1] Philip C. Woodland. Weight limiting, weight quantisation and generalisation in multi-layer perceptrons , 1989 .

[2] Martin T. Hagan,et al. Neural network design , 1995 .

[3] Elizabeth C. Behrman,et al. Optical hardware backpropagation neural network , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[4] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .

[5] Evor L. Hines,et al. Integer-weight neural nets , 1994 .

[6] Sorin Draghici,et al. Some new results on the capabilities of integer weights neural networks in classification problems , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[7] Zhou Bin,et al. A New Learning Algorithm for Neural Networks with Integer Weights and Quantized Non-linear Activation Functions , 2008, IFIP AI.

[8] H. John Caulfield,et al. Weight discretization paradigm for optical neural networks , 1990, Other Conferences.

[9] Emile Fiesler,et al. Connectionist Quantization Functions , 1996 .

[10] Tzi-Dar Chiueh,et al. Learning algorithms for neural networks with ternary weights , 1988, Neural Networks.