Hardware implementation of neural network with Sigmoidal activation functions using CORDIC

Activation function is the most important function in neural network processing. In this article, the field-programmable gate array (FPGA)-based hardware implementation of a multilayer feed-forward neural network, with a log sigmoid activation function and a tangent sigmoid (hyperbolic tangent) activation function has been presented, with more accuracy than any other previous implementation of a neural network with the same activation function. Accuracy is enhanced through the implementation of both the sigmoidal functions using COordinate Rotation DIgital Computer (CORDIC) algorithm. The CORDIC algorithm is a simple and effective method for calculation of the trigonometric and hyperbolic functions. Simulations and experiments have been performed on the ISim simulation engine of the Xilinx Framework, using the Very High Speed Integrated Circuit Hardware Description Language (VHDL) as the programming language. The results show accuracy for a 32-bit and 64-bit input/output, compromising with speed.

[1]  Keechul Jung,et al.  GPU implementation of neural networks , 2004, Pattern Recognit..

[2]  Xi Chen,et al.  Efficient Sigmoid Function for Neural Networks Based FPGA Design , 2006, ICIC.

[3]  Soo-Young Lee,et al.  Effects of multiplier output offsets on on-chip learning for analog neuro-chips , 2004, Neural Processing Letters.

[4]  N. Masmoudi,et al.  Electronic implementation of a neural observer in FPGA technology application to the control of electric vehicle , 2004, Proceedings. The 16th International Conference on Microelectronics, 2004. ICM 2004..

[5]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[6]  A. Fernandez,et al.  Design of a pipelined hardware architecture for real-time neural network computations , 2002, The 2002 45th Midwest Symposium on Circuits and Systems, 2002. MWSCAS-2002..

[7]  Francisco Sandoval Hernández,et al.  FPGA implementation of a systems identification module based upon Hopfield networks , 2007, Neurocomputing.

[8]  Tülay Yildirim,et al.  FPGA implementation of a General Regression Neural Network: An embedded pattern classification system , 2010, Digit. Signal Process..

[9]  Ray Andraka,et al.  A survey of CORDIC algorithms for FPGA based computers , 1998, FPGA '98.

[10]  Hiroaki Kobayashi,et al.  Evaluating Computational Performance of Backpropagation Learning on Graphics Hardware , 2009, Electron. Notes Theor. Comput. Sci..

[11]  Ronald S. Gyurcsik,et al.  Toward a general-purpose analog VLSI neural network with on-chip learning , 1997, IEEE Trans. Neural Networks.

[12]  Cihan Karakuzu,et al.  Neural identification of dynamic systems on FPGA with improved PSO learning , 2012, Appl. Soft Comput..

[13]  Federico Pedersini,et al.  FPGA implementation of a stochastic neural network for monotonic pseudo-Boolean optimization , 2008, Neural Networks.

[14]  J. E. Meggitt,et al.  Pseudo Division and Pseudo Multiplication Processes , 1962, IBM J. Res. Dev..

[16]  Fernando Morgado Dias,et al.  A new solution to the hyperbolic tangent implementation in hardware: polynomial modeling of the fractional exponential part , 2013, Neural Computing and Applications.

[17]  İsmail Koyuncu,et al.  Design and Implementation of Neural Networks Neurons with RadBas, LogSig, and TanSig Activation Functions on FPGA , 2012 .

[18]  Shawki Areibi,et al.  The Impact of Arithmetic Representation on Implementing MLP-BP on FPGAs: A Study , 2007, IEEE Transactions on Neural Networks.

[19]  Darío Baptista,et al.  Low-resource hardware implementation of the hyperbolic tangent for artificial neural networks , 2013, Neural Computing and Applications.

[21]  Shawki Areibi,et al.  A scalable pipelined architecture for real-time computation of MLP-BP neural networks , 2012, Microprocess. Microsystems.

[22]  Fernando Morgado Dias,et al.  A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function , 2007, Neurocomputing.

[23]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[24]  Armando Astarloa,et al.  Hardware architecture for a general regression neural network coprocessor , 2007, Neurocomputing.

[25]  B.K. Bose,et al.  Field Programmable Gate Array (FPGA) Based Neural Network Implementation of Stator Flux Oriented Vector Control of Induction Motor Drive , 2006, 2006 IEEE International Conference on Industrial Technology.

[26]  Tomas Nordström,et al.  Using FPLs to Implement a Reconfigurable Highly Parallel Computer , 1992, FPL.

[27]  Deni Torres Román,et al.  Near real time enhancement of geospatial imagery via systolic implementation of neural network-adapted convex regularization techniques , 2011, Pattern Recognit. Lett..

[28]  Nadia Nedjah,et al.  Analog Hardware Implementations of Artificial Neural Networks , 2011, J. Circuits Syst. Comput..