Efficient digital implementation of the sigmoid function for reprogrammable logic

Special attention must be paid to an efficient approximation of the sigmoid function in implementing FPGA-based reprogrammable hardware-based artificial neural networks. Four previously published piecewise linear and one piecewise second-order approximation of the sigmoid function are compared with SIG-sigmoid, a purely combinational approximation. The approximations are compared in terms of speed, required area resources and accuracy measured by average and maximum error. It is concluded that the best performance is achieved by SIG-sigmoid.

[1]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[2]  Willard Van Orman Quine,et al.  A Way to Simplify Truth Functions , 1955 .

[3]  Peter J. Ashenden,et al.  The Designer's Guide to VHDL , 1995 .

[4]  Jagath C. Rajapakse,et al.  Neural networks in FPGAs , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[5]  C. Alippi,et al.  Simple approximation of sigmoidal functions: realistic design of digital neural networks capable of learning , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[6]  Jacek M. Zurada,et al.  Introduction to artificial neural systems , 1992 .

[7]  K. M. Curtis,et al.  Piecewise linear approximation applied to nonlinear function of a neural network , 1997 .

[8]  J. M. Tarela,et al.  Digital design of sigmoid approximator for artificial neural networks , 2002 .

[9]  S. R. Jones,et al.  IMPLEMENTING NONLINEAR ACTIVATION FUNCTIONS IN NEURAL NETWORK EMULATORS , 1991 .

[10]  Anton Kummert,et al.  FPGA implementation of a neural network for a real-time hand tracking system , 2002, Proceedings First IEEE International Workshop on Electronic Design, Test and Applications '2002.

[11]  D. J. Myers,et al.  Efficient implementation of piecewise linear activation function for digital VLSI neural networks , 1989 .

[12]  Michael John Sebastian Smith,et al.  Application-specific integrated circuits , 1997 .

[13]  Behrooz Parhami,et al.  Computer arithmetic - algorithms and hardware designs , 1999 .

[14]  Stamatis Vassiliadis,et al.  Sigmoid Generators for Neural Computing Using Piecewise Approximations , 1996, IEEE Trans. Computers.

[15]  Hon Keung Kwan.  Simple sigmoid-like activation function suitable for digital hardware implementation , 1992 .

[16]  Emile Fiesler,et al.  Neural Network Adaptations to Hardware Implementations , 1997 .

[17]  E. McCluskey.  Minimization of Boolean functions , 1956 .

[18]  César Torres-Huitzil,et al.  Compact Spiking Neural Network Implementation in FPGA , 2000, FPL.

[19]  Uwe Meyer-Baese.  Digital Signal Processing with Field Programmable Gate Arrays , 2001 .

[20]  Michel Dagenais,et al.  McBOOLE: A New Procedure for Exact Logic Minimization , 1986, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[21]  Kamel Besbes,et al.  Digital hardware implementation of sigmoid function and its derivative for artificial neural networks , 2001, ICM 2001 Proceedings. The 13th International Conference on Microelectronics..

[22]  Daniel D. Gajski.  Principles of Digital Design , 1996 .

[23]  Jenq-Neng Hwang,et al.  Finite Precision Error Analysis of Neural Network Hardware Implementations , 1993, IEEE Trans. Computers.