Experimental Determination of Precision Requirements for Back-propagation Training of Artiicial Neural Networks

The impact of reduced weight and output precision on the back-propagation training algorithm Wer74, RHW86] is experimentally determined for a feed-forward multi-layer perceptron. In contrast with previous such studies, the network is large with over 20,000 weights, and is trained with a large, real-world data set of over 130,000 patterns to perform a diicult task, that of phoneme classiication for a continuous speech recognition system. The results indicate that 16b weight values are suucient to achieve training and classiication results comparable to 32b oating point, provided that weight and bias values are scaled separately, and that rounding rather than truncation is employed to reduce the precision of intermediary values. Output precision can be reduced to 8 bits without signiicant eeects on performance.