CORDIC-SNN: On-FPGA STDP Learning With Izhikevich Neurons

This paper proposes a neuromorphic platform for on-FPGA online spike timing dependant plasticity (STDP) learning, based on the COordinate Rotation DIgital Computer (CORDIC) algorithms. The implemented platform comprises two main components. First, the Izhikevich neuron model is modified for implementation using the CORDIC algorithm, simulated to ensure the model accuracy, described as hardware, and implemented on FPGA. Second, the STDP learning algorithm is adapted and optimized using the CORDIC method, synthesized for hardware, and implemented to perform on-FPGA online learning on a network of CORDIC Izhikevich neurons to demonstrate competitive Hebbian learning. The implementation results are compared with the original model and state-of-the-art to verify accuracy, effectiveness, and higher speed of the system. These comparisons confirm that the proposed neuromorphic system offers better performance and higher accuracy while being straightforward to implement and suitable to scale.

[1]  Mingoo Seok,et al.  Energy-Efficient Neuromorphic Classifiers , 2016, Neural Computation.

[2]  Tara Julia Hamilton,et al.  Efficient FPGA Implementations of Pair and Triplet-Based STDP for Neuromorphic Architectures , 2019, IEEE Transactions on Circuits and Systems I: Regular Papers.

[3]  Arash Ahmadi,et al.  Digital Multiplierless Implementation of Biological Adaptive-Exponential Neuron Model , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  Saraju P. Mohanty,et al.  Low-Power High-Level Synthesis for Nanoscale CMOS Circuits , 2008 .

[5]  Majid Ahmadi,et al.  Digital Realization of PSTDP and TSTDP Learning , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[6]  Giacomo Indiveri,et al.  On-chip unsupervised learning in winner-take-all networks of spiking neurons , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[7]  Hiroyuki Torikai,et al.  A Novel Asynchronous Cellular Automaton Multicompartment Neuron Model , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[8]  André van Schaik,et al.  Neuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[9]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[10]  Witold Pedrycz,et al.  Contrastive divergence for memristor-based restricted Boltzmann machine , 2015, Engineering applications of artificial intelligence.

[11]  Bin Deng,et al.  Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks , 2019, IEEE Transactions on Cybernetics.

[12]  Stephan Henker,et al.  A 32 GBit/s communication SoC for a waferscale neuromorphic system , 2012, Integr..

[13]  D. Abbott,et al.  Tunable Low Energy, Compact and High Performance Neuromorphic Circuit for Spike-Based Synaptic Plasticity , 2014, PloS one.

[14]  Mostafa Rahimi Azghadi,et al.  Unsupervised Character Recognition with a Simplified FPGA Neuromorphic System , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[15]  Spike Timing-Dependent Plasticity (STDP) , 2017 .

[16]  Byoung Hun Lee,et al.  Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron , 2015, IEEE Transactions on Industrial Electronics.

[17]  Ammar Belatreche,et al.  Challenges for large-scale implementations of spiking neural networks on FPGAs , 2007, Neurocomputing.

[18]  Mostafa Rahimi Azghadi,et al.  A neuromorphic VLSI design for spike timing and rate based synaptic plasticity , 2013, Neural Networks.

[19]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[20]  Hamid Soleimani,et al.  An Efficient and Reconfigurable Synchronous Neuron Model , 2018, IEEE Transactions on Circuits and Systems II: Express Briefs.

[21]  Stefano Brivio,et al.  Synaptic potentiation and depression in Al , 2015 .

[22]  F. Grassia,et al.  Silicon neuron: digital hardware implementation of the quartic model , 2014, Artificial Life and Robotics.

[23]  Rashid Rashidzadeh,et al.  A CORDIC Based Digital Hardware For Adaptive Exponential Integrate and Fire Neuron , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[24]  C. Morris,et al.  Voltage oscillations in the barnacle giant muscle fiber. , 1981, Biophysical journal.

[25]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[26]  G. Buzsáki,et al.  NeuroGrid: recording action potentials from the surface of the brain , 2014, Nature Neuroscience.

[27]  Daniel Gutierrez-Galan,et al.  Multilayer Spiking Neural Network for Audio Samples Classification Using SpiNNaker , 2016, ICANN.

[28]  Kea-Tiong Tang,et al.  A high learning capability probabilistic spiking neural network chip , 2018, 2018 International Symposium on VLSI Design, Automation and Test (VLSI-DAT).

[29]  Yannick Bornat,et al.  FPGA-based architecture for real-time synaptic plasticity computation , 2008, 2008 15th IEEE International Conference on Electronics, Circuits and Systems.

[30]  Kwabena Boahen,et al.  A Neuromorph's Prospectus , 2017, Computing in Science & Engineering.

[31]  Derek Abbott,et al.  Digital Multiplierless Realization of Two Coupled Biological Morris-Lecar Neuron Model , 2015, IEEE Transactions on Circuits and Systems I: Regular Papers.

[32]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[33]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[34]  Alain Destexhe,et al.  Tunable Neuromimetic Integrated System for Emulating Cortical Neuron Models , 2011, Front. Neurosci..

[35]  Gert Cauwenberghs,et al.  Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain , 2018, Front. Neurosci..

[36]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[37]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[38]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[39]  Steve B. Furber,et al.  Building a Spiking Neural Network Model of the Basal Ganglia on SpiNNaker , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[40]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[41]  Hassan Mostafa,et al.  A Low Power CORDIC-Based Hardware Implementation of Izhikevich Neuron Model , 2018, 2018 16th IEEE International New Circuits and Systems Conference (NEWCAS).

[42]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[43]  J. Hindmarsh,et al.  The assembly of ionic currents in a thalamic neuron I. The three-dimensional model , 1989, Proceedings of the Royal Society of London. B. Biological Sciences.

[44]  Mostafa Rahimi Azghadi,et al.  Spike-Based Synaptic Plasticity in Silicon: Design, Implementation, Application, and Challenges , 2014, Proceedings of the IEEE.

[45]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[46]  Mostafa Rahimi Azghadi,et al.  Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures , 2015, ACM J. Emerg. Technol. Comput. Syst..

[47]  H. Wilson.  Simplified dynamics of human and mammalian neocortical neurons. , 1999, Journal of theoretical biology.

[48]  Hiroyuki Torikai,et al.  A Novel Hardware-Efficient Asynchronous Cellular Automaton Model of Spike-Timing-Dependent Synaptic Plasticity , 2016, IEEE Transactions on Circuits and Systems II: Express Briefs.

[49]  Arash Ahmadi,et al.  Biologically Inspired Spiking Neurons: Piecewise Linear Models and Digital Implementation , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[50]  Seul Jung,et al.  Implementation of the RBF neural chip with the on-line learning back-propagation algorithm , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[51]  Arash Ahmadi,et al.  A digital implementation of 2D Hindmarsh–Rose neuron , 2017 .

[52]  Tetsuya Hishiki,et al.  A Generalized Rotate-and-Fire Digital Spiking Neuron Model and Its On-FPGA Learning , 2011, IEEE Transactions on Circuits and Systems II: Express Briefs.

[53]  Sergio Davies,et al.  Learning in spiking neural networks , 2013 .

[54]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[55]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[56]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[57]  André van Schaik,et al.  A mixed-signal implementation of a polychronous spiking neural network with delay adaptation , 2014, Front. Neurosci..

[58]  René Schüffny,et al.  A Biological-Realtime Neuromorphic System in 28 nm CMOS Using Low-Leakage Switched Capacitor Circuits , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[59]  Ana B. Porto-Pazos,et al.  Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications , 2016, International journal of molecular sciences.

[60]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[61]  Evangelos Stromatias,et al.  Supervised learning in Spiking Neural Networks with limited precision: SNN/LP , 2014, 2015 International Joint Conference on Neural Networks (IJCNN).