An Accelerated LIF Neuronal Network Array for a Large-Scale Mixed-Signal Neuromorphic Architecture

We present an array of leaky integrate-and-fire (LIF) neuron circuits designed for the second-generation BrainScaleS mixed-signal 65-nm CMOS neuromorphic hardware. The neuronal array is embedded in the analog network core of a scaled-down prototype high input count analog neural network with digital learning system chip. Designed as continuous-time circuits, the neurons are highly tunable and reconfigurable elements with accelerated dynamics. Each neuron integrates input current from a multitude of incoming synapses and evokes a digital spike event output. The circuit offers a wide tuning range for synaptic and membrane time constants, as well as for refractory periods to cover a number of computational models. We elucidate our design methodology, underlying circuit design, calibration, and measurement results from individual sub-circuits across multiple dies. The circuit dynamics matches with the behavior of the LIF mathematical model. We further demonstrate a winner-take-all network on the prototype chip as a typical element of cortical processing.

[1]  Chiara Bartolozzi,et al.  Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems , 2014, Proceedings of the IEEE.

[2]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[3]  G. Deco,et al.  Ongoing Cortical Activity at Rest: Criticality, Multistability, and Ghost Attractors , 2012, The Journal of Neuroscience.

[4]  Vishal Saxena,et al.  Indirect compensation techniques for three-stage fully-differential op-amps , 2010, 2010 53rd IEEE International Midwest Symposium on Circuits and Systems.

[5]  Johannes Schemmel,et al.  A highly tunable 65-nm CMOS LIF neuron for a large scale neuromorphic system , 2016, ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference.

[6]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[7]  Christofer Toumazou,et al.  Nanopower Subthreshold MCML in Submicrometer CMOS Technology , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Johannes Schemmel,et al.  A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores , 2018, IEEE Transactions on Biomedical Circuits and Systems.

[9]  Johannes Schemmel,et al.  Characterization and Compensation of Network-Level Anomalies in Mixed-Signal Neuromorphic Modeling Platforms , 2014, PloS one.

[10]  Alain Destexhe,et al.  Neuronal Computations with Stochastic Network States , 2006, Science.

[11]  Alain Destexhe,et al.  Self-sustained Asynchronous Irregular States and Up–down States in Thalamic, Cortical and Thalamocortical Networks of Nonlinear Integrate-and-fire Neurons , 2022 .

[12]  Karlheinz Meier.  Special report : Can we copy the brain? - The brain as computer , 2017, IEEE Spectrum.

[13]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[14]  Simon Friedmann,et al.  A new approach to learning in neuromorphic hardware , 2013 .

[15]  V. Saxena,et al.  Compensation of CMOS op-amps using split-length transistors , 2008, 2008 51st Midwest Symposium on Circuits and Systems.

[16]  Christofer Toumazou,et al.  Bulk-drain connected load for subthreshold MOS current-mode logic , 2007 .

[17]  Yingxue Wang,et al.  A Two-Dimensional Configurable Active Silicon Dendritic Neuron Array , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[19]  Wulfram Gerstner,et al.  Firing patterns in the adaptive exponential integrate-and-fire model , 2008, Biological Cybernetics.

[20]  John von Neumann,et al.  First draft of a report on the EDVAC , 1993, IEEE Annals of the History of Computing.

[21]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[22]  A. Tajalli,et al.  Subthreshold Source-Coupled Logic Circuits for Ultra-Low-Power Applications , 2008, IEEE Journal of Solid-State Circuits.

[23]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[24]  Karlheinz Meier,et al.  A mixed-signal universal neuromorphic computing system , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[25]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[26]  E. D’Angelo.  The human brain project. , 2012, Functional neurology.

[27]  Johannes Schemmel,et al.  Stochastic inference with spiking neurons in the high-conductance state , 2016, Physical review. E.

[28]  Johannes Schemmel,et al.  From LIF to AdEx neuron models: Accelerated analog 65 nm CMOS implementation , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[29]  Thierry Bal,et al.  Calculating event-triggered average synaptic conductances from the membrane potential. , 2006, Journal of neurophysiology.

[30]  Richard Hans Robert Hahnloser,et al.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit , 2000, Nature.

[31]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[32]  Wulfram Gerstner,et al.  Integrate-and-Fire models with adaptation are good enough , 2005, NIPS.

[33]  Arindam Basu,et al.  A Learning-Enabled Neuron Array IC Based Upon Transistor Channel Models of Biological Phenomena , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[34]  John Lazzaro,et al.  Winner-Take-All Networks of O(N) Complexity , 1988, NIPS.

[35]  Giacomo Indiveri,et al.  Memory and Information Processing in Neuromorphic Systems , 2015, Proceedings of the IEEE.

[36]  Manish Kumar.  Large-scale neuromorphic computing systems , 2016 .

[37]  Sebastian Millner,et al.  Development of a multi-compartment neuron model emulation , 2012 .

[38]  Johannes Schemmel,et al.  Six Networks on a Universal Neuromorphic Computing Substrate , 2012, Front. Neurosci..

[39]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[40]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[41]  Johannes Schemmel,et al.  Demonstrating Hybrid Learning in a Flexible Neuromorphic Hardware System , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[42]  John von Neumann,et al.  The Computer and the Brain , 1960 .

[43]  Matthias Hock,et al.  Modern semiconductor technologies for neuromorphic hardware , 2014 .

[44]  Ad Aertsen,et al.  Functional consequences of correlated excitatory and inhibitory conductances in cortical networks , 2010, Journal of Computational Neuroscience.

[45]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[46]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[47]  A. Destexhe,et al.  The high-conductance state of neocortical neurons in vivo , 2003, Nature Reviews Neuroscience.

[48]  J. Jacob Wikner,et al.  1.2-V Analog Interface for a 300-MSps HD Video Digitizer in Core 65-nm CMOS , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[49]  Kwabena Boahen,et al.  Silicon-Neuron Design: A Dynamical Systems Approach , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[50]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[51]  Syed Ahmed Aamir,et al.  Mixed-Signal Circuit Implementation of Spiking Neuron Models , 2018 .

[52]  Johannes Schemmel,et al.  An accelerated analog neuromorphic hardware system emulating NMDA- and calcium-based non-linear dendrites , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[53]  Steve B. Furber,et al.  Scalable energy-efficient, low-latency implementations of trained spiking Deep Belief Networks on SpiNNaker , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[54]  A. Destexhe,et al.  Impact of spontaneous synaptic activity on the resting properties of cat neocortical pyramidal neurons In vivo. , 1998, Journal of neurophysiology.

[55]  Gustavo Deco,et al.  Network Bursting Dynamics in Excitatory Cortical Neuron Cultures Results from the Combination of Different Adaptive Mechanism , 2013, PloS one.

[56]  Johannes Schemmel,et al.  An analog dynamic memory array for neuromorphic hardware , 2013, 2013 European Conference on Circuit Theory and Design (ECCTD).

[57]  Johannes Schemmel,et al.  The high-conductance state enables neural sampling in networks of LIF neurons , 2015, BMC Neuroscience.

[58]  Pier Stanislao Paolucci,et al.  Power, Energy and Speed of Embedded and Server Multi-Cores applied to Distributed Simulation of Spiking Neural Networks: ARM in NVIDIA Tegra vs Intel Xeon quad-cores , 2015, ArXiv.

[59]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[60]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[61]  J. Jacob Wikner,et al.  Frequency compensation of high-speed, low-voltage CMOS multistage amplifiers , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[62]  Randy H. Katz,et al.  Design of PLL-based clock generation circuits , 1987 .