Symmetric table addition methods for neural network approximations

Symmetric table addition methods (STAMs) approximate functions by performing parallel table lookups, followed by multioperand addition. STAMs require significantly less memory than direct table lookups and are faster than piecewise linear approximations. This paper investigates the application of STAMs to the sigmoid function and its derivative, which are commonly used in artificial neural networks. Compared to direct table lookups, STAMs require between 23 and 41 times less memory for sigmoid and between 24 and 46 times less memory for sigmoid's derivative, when the input operand size is 16 bits and the output precision is 12 bits.

[1]  P. Michael Farmwald.  High bandwidth evaluation of elementary functions , 1981, 1981 IEEE 5th Symposium on Computer Arithmetic (ARITH).

[2]  Arnaud Tisserand,et al.  Some improvements on multipartite table methods , 2001, Proceedings 15th IEEE Symposium on Computer Arithmetic. ARITH-15 2001.

[3]  Jean-Michel Muller,et al.  Elementary Functions: Algorithms and Implementation , 1997 .

[4]  Stamatis Vassiliadis,et al.  Elementary function generators for neural-network emulators , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  Stamatis Vassiliadis,et al.  Sigmoid Generators for Neural Computing Using Piecewise Approximations , 1996, IEEE Trans. Computers.

[6]  Debjit Das Sarma,et al.  Measuring the accuracy of ROM reciprocal tables , 1993, Proceedings of IEEE 11th Symposium on Computer Arithmetic.

[7]  Michael J. Schulte,et al.  The Symmetric Table Addition Method for Accurate Function Approximation , 1999, J. VLSI Signal Process..

[8]  Michael J. Schulte,et al.  Approximating Elementary Functions with Symmetric Bipartite Tables , 1999, IEEE Trans. Computers.

[9]  S. Vassiliadis,et al.  High performance with low implementation cost sigmoid generators , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[10]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[11]  Earl E. Swartzlander,et al.  Hardware Designs for Exactly Rounded Elemantary Functions , 1994, IEEE Trans. Computers.

[12]  D. J. Myers,et al.  Efficient implementation of piecewise linear activation function for digital VLSI neural networks , 1989 .

[13]  M. Wedlake,et al.  A CORDIC implementation of a digital artificial neuron , 1997, 1997 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, PACRIM. 10 Years Networking the Pacific Rim, 1987-1997.

[14]  M. Vellasco,et al.  VLSI architectures for neural networks , 1989, IEEE Micro.