Efficient Design of Spiking Neural Network With STDP Learning Based on Fast CORDIC

In emerging Spiking Neural Network (SNN) based neuromorphic hardware design, energy efficiency and on-line learning are attractive advantages mainly contributed by bio-inspired local learning with nonlinear dynamics and at the cost of associated hardware complexity. This paper presents a novel SNN design employing fast COordinate Rotation DIgital Computer (CORDIC) algorithm to achieve fast spike timing–dependent plasticity (STDP) learning with high hardware efficiency. In this study, a system design and evaluation method of CORDIC-based SNN is proposed for finding optimal CORDIC type and precision, from theoretical CORDIC-level error to application-level learning performance. From the proposed design and evaluation method, a reconfigurable SNN design based on fast-convergence CORDIC is designed to achieve high classification accuracy on MNIST, fast on-line learning and good energy efficiency. By utilizing SNN’s fault tolerance and time-division-multiplexing (TDM) strategy, the reconfigurable SNN design employs 8-bit fast-convergence CORDIC and TDM-based hardware accelerator for high efficiency. FPGA implementation results confirm that the proposed fast-convergence CORDIC SNN design outperforms the state-of-the-art CORDIC method by 38.5%−45.3% in terms of learning speed and energy efficiency, with the STDP learning of 30.2 ns/SOP, energy efficiency of 176.6 pJ/SOP, processing speed of 6.1 ms/image, and on-line learning convergence of 21.4 s (time to reach the final accuracy, on average), on MNIST benchmark.

[1]  Jun Zhou,et al.  A 1-V to 0.29-V sub-100-pJ/operation ultra-low power fast-convergence CORDIC processor in 0.18-μm CMOS , 2018, Microelectron. J..

[2]  Byoung Hun Lee,et al.  Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron , 2015, IEEE Transactions on Industrial Electronics.

[3]  Earl E. Swartzlander,et al.  Adaptive CORDIC: Using Parallel Angle Recoding to Accelerate Rotations , 2010, IEEE Transactions on Computers.

[4]  Yu Hen Hu,et al.  An Angle Recoding Method for CORDIC Algorithm Implementation , 1993, IEEE Trans. Computers.

[5]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[6]  Tony Q. S. Quek,et al.  Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks , 2017, AAAI.

[7]  Arash Ahmadi,et al.  Digital Multiplierless Implementation of Biological Adaptive-Exponential Neuron Model , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Pierre Tirilly,et al.  Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[9]  Alexander Sboev,et al.  Solving a classification task by spiking neurons with STDP and temporal coding , 2017, BICA.

[10]  Shih-Chii Liu,et al.  Minitaur, an Event-Driven FPGA-Based Spiking Network Accelerator , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[11]  Darpan T. Sanghavi,et al.  BindsNET: A Machine Learning-Oriented Spiking Neural Networks Library in Python , 2018, Front. Neuroinform..

[12]  Majid Ahmadi,et al.  CORDIC-SNN: On-FPGA STDP Learning With Izhikevich Neurons , 2019, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Gopalakrishnan Srinivasan,et al.  Training Deep Spiking Convolutional Neural Networks With STDP-Based Unsupervised Pre-training Followed by Supervised Fine-Tuning , 2018, Front. Neurosci..

[14]  Rashid Rashidzadeh,et al.  A CORDIC Based Digital Hardware For Adaptive Exponential Integrate and Fire Neuron , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  René Schüffny,et al.  A Biological-Realtime Neuromorphic System in 28 nm CMOS Using Low-Leakage Switched Capacitor Circuits , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[16]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[17]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[18]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[19]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[20]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[21]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[22]  Gregory K. Chen,et al.  A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS , 2019, IEEE Journal of Solid-State Circuits.

[23]  Andrew S. Cassidy,et al.  Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization , 2013, Neural Networks.

[24]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[25]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[26]  Ruthvik Vaila,et al.  Feature Extraction using Spiking Convolutional Neural Networks , 2019, ICONS.

[27]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[28]  André van Schaik,et al.  A mixed-signal implementation of a polychronous spiking neural network with delay adaptation , 2014, Front. Neurosci..

[29]  Sang Yoon Park,et al.  CORDIC Designs for Fixed Angle of Rotation , 2013, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[30]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[31]  Mingguo Zhao,et al.  Towards artificial general intelligence with hybrid Tianjic chip architecture , 2019, Nature.

[32]  Andrew S. Cassidy,et al.  A combinational digital logic approach to STDP , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[33]  Abbas Nowzari-Dalini,et al.  Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks , 2019, Pattern Recognit..

[34]  K. Sridharan,et al.  50 Years of CORDIC: Algorithms, Architectures, and Applications , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[35]  Qian Wang,et al.  Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA , 2017, Neurocomputing.

[36]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[37]  Anthony S. Maida,et al.  Training Spiking ConvNets by STDP and Gradient Descent , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[38]  Behrooz Parhami,et al.  Computer arithmetic - algorithms and hardware designs , 1999 .