The feasibility of restricting the weight values in multilayer perceptrons to powers of two or sums of powers of two is studied. Multipliers could be thus replaced by shifters and adders on digital hardware, saving both time and chip area, under the assumption that the neuron activation function is computed through a lookup table (LUT) and that a LUT can be shared among many neurons. A learning procedure based on back-propagation for obtaining a neural network with such discrete weights is presented. This learning procedure requires full real arithmetic and therefore must be performed offline. It starts from a multilayer perceptron with continuous weights learned using back-propagation. Then a weight normalization is made to ensure that the whole shifting dynamics is used and to maximize the match between continuous and discrete weights of neurons sharing the same LUT. Finally, a discrete version of BP algorithm with automatic learning rate control is applied up to convergence. Some test runs on a simple pattern recognition problem show the feasibility of the approach
[1]
Bede Liu,et al.
Design of cascade form FIR filters with discrete valued coefficients
,
1988,
IEEE Trans. Acoust. Speech Signal Process..
[2]
Richard P. Lippmann,et al.
An introduction to computing with neural nets
,
1987
.
[3]
Geoffrey E. Hinton,et al.
Learning internal representations by error propagation
,
1986
.
[4]
S. Y. Kung,et al.
Parallel architectures for artificial neural nets
,
1988,
IEEE 1988 International Conference on Neural Networks.
[5]
Y. Lim,et al.
FIR filter design over a discrete powers-of-two coefficient space
,
1983
.
[6]
Yoshiaki Tadokoro,et al.
A simple design of FIR filters with powers-of-two coefficients
,
1988
.