Precise digital implementations of hyperbolic tanh and sigmoid function

Sigmoid and Hyperbolic Tangent are widely used as activation functions in artificial neural networks. Exponential term and division are basic building blocks of these functions. This paper proposes precise and efficient hardware implementations for sigmoid and hyperbolic tangent functions using exponential function approximation. Performance of both functions has been verified which shows that the proposed implementations have up to 99.97% similarity with the ideal transfer functions while the circuits take maximum 2% of logic resources when implemented on a Vertex IV FPGA.

[1]  Pramod Kumar Meher.  An optimized lookup-table for the evaluation of sigmoid function for artificial neural networks , 2010, VLSI-SoC.

[2]  K. M. Curtis,et al.  Piecewise linear approximation applied to nonlinear function of a neural network , 1997 .

[3]  Israel Koren.  Computer arithmetic algorithms , 1993 .

[4]  Mitra Mirhassani,et al.  Efficient VLSI Implementation of Neural Networks With Hyperbolic Tangent Activation Function , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[5]  D. J. Myers,et al.  Efficient implementation of piecewise linear activation function for digital VLSI neural networks , 1989 .

[6]  Arash Ahmadi,et al.  Digital Multiplierless Implementation of Biological Adaptive-Exponential Neuron Model , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  Huapeng Wu,et al.  High Speed VLSI Implementation of the Hyperbolic Tangent Sigmoid Function , 2008, 2008 Third International Conference on Convergence and Hybrid Information Technology.

[8]  Majid Ahmadi,et al.  Efficient hardware implementation of the hyperbolic tangent sigmoid function , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[9]  J. M. Tarela,et al.  Approximation of sigmoid function and the derivative for hardware implementation of artificial neurons , 2004 .

[10]  Maicon A. Sartin,et al.  Approximation of hyperbolic tangent activation function using hybrid methods , 2013, 2013 8th International Workshop on Reconfigurable and Communication-Centric Systems-on-Chip (ReCoSoC).