Hardware implementation of hyperbolic tangent and sigmoid activation functions

This paper presents the high accuracy hardware implementation of the hyperbolic tangent and sigmoid activation functions for artificial neural networks. A kind of a direct implementation of the functions in a few different versions is proposed and investigated both by software and hardware modeling. A single precision floating point arithmetic is applied. Apart from conventional design style with hardware description language coding, high level synthesis design techniques with the Matlab HDL coder and Xilinx Vivado HLS have also been investigated.

[1]  Zbigniew Hajduk,et al.  High accuracy FPGA activation function implementation for neural networks , 2017, Neurocomputing.

[2]  Farida Cheriet,et al.  A Configurable FPGA Implementation of the Tanh Function Using DCT Interpolation , 2017, 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM).

[3]  Nilay Khare,et al.  Hardware implementation of neural network with Sigmoidal activation functions using CORDIC , 2015, Microprocess. Microsystems.

[4]  Bartosz Trybus,et al.  Architecture of FPGA Embedded Multiprocessor Programmable Controller , 2015, IEEE Transactions on Industrial Electronics.

[5]  Zbigniew Hajduk,et al.  An FPGA embedded microcontroller , 2014, Microprocess. Microsystems.

[6]  Javier Echanobe,et al.  Controlled accuracy approximation of sigmoid function for efficient FPGA-based implementation of artificial neurons , 2013 .

[7]  Jacek Kluska,et al.  Hardware Implementation of P1-TS Fuzzy Rule-Based Systems on FPGA , 2013, ICAISC.

[8]  F. M. Dias,et al.  Low-resource hardware implementation of the hyperbolic tangent for artificial neural networks , 2013, Neural Computing and Applications.

[9]  L. Fanucci,et al.  Low-error digital hardware implementation of artificial neuron activation functions and their derivative , 2011, Microprocess. Microsystems.

[10]  Teresa Orlowska-Kowalska,et al.  FPGA Implementation of the Multilayer Neural Network for the Speed Estimation of the Two-Mass Drive System , 2011, IEEE Transactions on Industrial Informatics.

[11]  Gustavo Sutter,et al.  High-Level Languages and Floating-Point Arithmetic for FPGA-Based CFD Simulations , 2011, IEEE Design & Test of Computers.

[12]  Abhisek Ukil,et al.  Development and Implementation of Parameterized FPGA-Based General Purpose Neural Networks for Online Applications , 2011, IEEE Transactions on Industrial Informatics.

[13]  Fan Zhou,et al.  Field-programmable gate array implementation of a probabilistic neural network for motor cortical decoding in rats , 2010, Journal of Neuroscience Methods.

[14]  Luca Fanucci,et al.  Low-error approximation of artificial neuron sigmoid function and its derivative , 2009 .

[15]  Mariusz Bajger,et al.  Low-error, High-speed Approximation of the Sigmoid Function for Large FPGA Implementations , 2008, J. Signal Process. Syst..

[16]  Fernando Morgado Dias,et al.  A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function , 2007, Neurocomputing.

[17]  Matti Tommiska,et al.  Efficient digital implementation of the sigmoid function for reprogrammable logic , 2003 .

[18]  Eric Crabill,et al.  Xilinx Unveils Vivado Design Suite for the Next Decade of ‘All Programmable’ Devices , 2012 .

[19]  M. Vajta.  Some remarks on Padé-approximations , 2000 .