Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores

Marching along the DARPA SyNAPSE roadmap, IBM unveils a trilogy of innovations towards the TrueNorth cognitive computing system inspired by the brain's function and efficiency. Judiciously balancing the dual objectives of functional capability and implementation/operational cost, we develop a simple, digital, reconfigurable, versatile spiking neuron model that supports one-to-one equivalence between hardware and simulation and is implementable using only 1272 ASIC gates. Starting with the classic leaky integrate-and-fire neuron, we add: (a) configurable and reproducible stochasticity to the input, the state, and the output; (b) four leak modes that bias the internal state dynamics; (c) deterministic and stochastic thresholds; and (d) six reset modes for rich finite-state behavior. The model supports a wide variety of computational functions and neural codes. We capture 50+ neuron behaviors in a library for hierarchical composition of complex computations and behaviors. Although designed with cognitive algorithms and applications in mind, serendipitously, the neuron model can qualitatively replicate the 20 biologically-relevant behaviors of a dynamical neuron model.

[1]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[2]  D. Modha,et al.  Network architecture of the long-distance pathways in the macaque brain , 2010, Proceedings of the National Academy of Sciences.

[3]  Anthony M. Zador,et al.  Novel Integrate-and-re-like Model of Repetitive Firing in Cortical Neurons , 1998 .

[4]  C. Morris,et al.  Voltage oscillations in the barnacle giant muscle fiber. , 1981, Biophysical journal.

[5]  S. Sherman,et al.  Fourier analysis of sinusoidally driven thalamocortical relay neurons and a minimal integrate-and-fire-or-burst model. , 2000, Journal of neurophysiology.

[6]  Wolfgang Maass,et al.  Belief Propagation in Networks of Spiking Neurons , 2009, Neural Computation.

[7]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[8]  Wulfram Gerstner,et al.  How Good Are Neuron Models? , 2009, Science.

[9]  Ueli Rutishauser,et al.  State-Dependent Computation Using Coupled Recurrent Networks , 2008, Neural Computation.

[10]  Andrew S. Cassidy,et al.  Building block of a programmable neuromorphic substrate: A digital neurosynaptic core , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[11]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[12]  R. FitzHugh Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[13]  T. Sejnowski,et al.  Neurocomputational models of working memory , 2000, Nature Neuroscience.

[14]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[15]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[16]  H. Wilson Simplified dynamics of human and mammalian neocortical neurons. , 1999, Journal of theoretical biology.

[17]  L. F Abbott,et al.  Lapicque’s introduction of the integrate-and-fire model neuron (1907) , 1999, Brain Research Bulletin.

[18]  D. Hansel,et al.  How Spike Generation Mechanisms Determine the Neuronal Response to Fluctuating Inputs , 2003, The Journal of Neuroscience.

[19]  Wulfram Gerstner,et al.  Generalized integrate-and-fire models of neuronal activity approximate spike trains of a detailed model to a high degree of accuracy. , 2004, Journal of neurophysiology.

[20]  Dharmendra S. Modha,et al.  A Digital Neurosynaptic Core Using Event-Driven QDI Circuits , 2012, 2012 IEEE 18th International Symposium on Asynchronous Circuits and Systems.

[21]  Myron Flickner,et al.  Compass: A scalable simulator for an architecture for cognitive computing , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[22]  J. Hindmarsh,et al.  The assembly of ionic currents in a thalamic neuron I. The three-dimensional model , 1989, Proceedings of the Royal Society of London. B. Biological Sciences.

[23]  Andrew S. Cassidy,et al.  Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[24]  Andrew S. Cassidy,et al.  Cognitive computing programming paradigm: A Corelet Language for composing networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[25]  G. Ermentrout,et al.  Parabolic bursting in an excitable system coupled with a slow oscillation , 1986 .

[26]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[27]  R. Stein A THEORETICAL ANALYSIS OF NEURONAL VARIABILITY. , 1965, Biophysical journal.

[28]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[29]  Colin Norman 2011 International Science & Engineering Visualization Challenge. , 2012, Science.

[30]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[31]  Ernst Niebur,et al.  A Generalized Linear Integrate-and-Fire Neural Model Produces Diverse Spiking Behaviors , 2009, Neural Computation.

[32]  Colin Norman,et al.  2012 Visualization Challenge , 2013 .

[33]  Dharmendra S. Modha,et al.  Cognitive Computing , 2011, Informatik-Spektrum.

[34]  Eugene M. Izhikevich,et al.  Resonate-and-fire neurons , 2001, Neural Networks.