FPGA Realization of Activation Function for Artificial Neural Networks

Implementation of artificial neural network (ANN) in hardware is needed to fully utilize the inherent parallelism. Presented work focuses on the configuration of field-programmable gate array (FPGA) to realize the activation function utilized in ANN. The computation of a nonlinear activation function (AF) is one of the factors that constraint the area or the computation time. The most popular AF is the log-sigmoid function, which has different possibilities of realizing in digital hardware. Equation approximation, lookup table (LUT) based approach and piecewise linear approximation (PWL) are a few to mention. A two-fold approach to optimize the resource requirement is presented here. Primarily, fixed-point computation (FXP) that needs minimal hardware, as against floating-point computation (FLP) is followed. Secondly, the PWL approximation of AF with more precision is proved to consume lesser Si area when compared to LUT based AF. Experimental results are presented for computation.

[1]  Jihan Zhu,et al.  FPGA Implementations of Neural Networks - A Survey of a Decade of Progress , 2003, FPL.

[2]  Atsushi Nakazawa,et al.  Hardware Implementation of a Pulse Density Neural Network Using Simultaneous Perturbation Learning Rule , 1999 .

[3]  Shawki Areibi,et al.  The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: A Study , 2007, IEEE Transactions on Neural Networks.

[4]  Shawki Areibi,et al.  Arithmetic formats for implementing artificial neural networks on FPGAs , 2006, Canadian Journal of Electrical and Computer Engineering.

[5]  Scott McMillan,et al.  A re-evaluation of the practicality of floating-point operations on FPGAs , 1998, Proceedings. IEEE Symposium on FPGAs for Custom Computing Machines (Cat. No.98TB100251).

[6]  Yutaka Maeda,et al.  Simultaneous perturbation learning rule for recurrent neural networks and its FPGA implementation , 2005, IEEE Transactions on Neural Networks.

[7]  Hiroomi Hikawa,et al.  A digital hardware pulse-mode neuron with piecewise linear activation function , 2003, IEEE Trans. Neural Networks.

[8]  J. L. Holt,et al.  Back propagation simulations using limited precision calculations , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[9]  Sing Kiong Nguang,et al.  Neural Network Implementation Using Bit Streams , 2007, IEEE Transactions on Neural Networks.

[10]  Davide Anguita,et al.  A digital architecture for support vector machines: theory, algorithm, and FPGA implementation , 2003, IEEE Trans. Neural Networks.