A Mixed-Signal Structured AdEx Neuron for Accelerated Neuromorphic Cores

Here, we describe a multicompartment neuron circuit based on the adaptive-exponential I&F (AdEx) model, developed for the second-generation BrainScaleS hardware. Based on an existing modular leaky integrate-and-fire (LIF) architecture designed in 65-nm CMOS, the circuit features exponential spike generation, neuronal adaptation, intercompartmental connections as well as a conductance-based reset. The design reproduces a diverse set of firing patterns observed in cortical pyramidal neurons. Further, it enables the emulation of sodium and calcium spikes, as well as N-methyl-D-aspartate plateau potentials known from apical and thin dendrites. We characterize the AdEx circuit extensions and exemplify how the interplay between passive and nonlinear active signal processing enhances the computational capabilities of single (but structured) on-chip neurons.

[1]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[2]  R. Douglas,et al.  A silicon neuron , 1991, Nature.

[3]  D. Johnston,et al.  A Synaptically Controlled, Associative Signal for Hebbian Plasticity in Hippocampal Neurons , 1997, Science.

[4]  David Bol,et al.  A compact phenomenological digital neuron implementing the 20 Izhikevich behaviors , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[5]  Johannes Schemmel,et al.  A highly tunable 65-nm CMOS LIF neuron for a large scale neuromorphic system , 2016, ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference.

[6]  W. Senn,et al.  Neocortical pyramidal cells respond as integrate-and-fire neurons to in vivo-like input currents. , 2003, Journal of neurophysiology.

[7]  Wen-Liang L Zhou,et al.  The decade of the dendritic NMDA spike , 2010, Journal of neuroscience research.

[8]  Andrew S. Cassidy,et al.  Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[9]  M. Larkum.  A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex , 2013, Trends in Neurosciences.

[10]  H. Barlow,et al.  Selective Sensitivity to Direction of Movement in Ganglion Cells of the Rabbit Retina , 1963, Science.

[11]  R. Stein.  Some models of neuronal variability. , 1967, Biophysical journal.

[12]  Guosong Liu,et al.  Local structural balance and functional interaction of excitatory and inhibitory synapses in hippocampal dendrites , 2004, Nature Neuroscience.

[13]  Idan Segev.  Single neurone models: oversimple, complex and reduced , 1992, Trends in Neurosciences.

[14]  Jennifer Hasler,et al.  Neuron Array With Plastic Synapses and Programmable Dendrites , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[15]  Piotr Dudek,et al.  Compact silicon neuron circuit with spiking and bursting behaviour , 2008, Neural Networks.

[16]  Nicolas Brunel,et al.  Neuronal Dynamics , 2009, Encyclopedia of Complexity and Systems Science.

[17]  M. Konishi.  How the Owl Tracks Its Prey Experiments with trained barn owls reveal how their acute sense of hearing enables them to catch prey in the dark , 1973 .

[18]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[19]  Johannes Schemmel,et al.  An Accelerated LIF Neuronal Network Array for a Large-Scale Mixed-Signal Neuromorphic Architecture , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[20]  Wulfram Gerstner,et al.  Integrate-and-Fire models with adaptation are good enough , 2005, NIPS.

[21]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[22]  Wilfrid Rall,et al.  Theoretical significance of dendritic trees for neuronal input-output relations , 1964 .

[23]  Misha Mahowald,et al.  A Spike Based Learning Neuron in Analog VLSI , 1996, NIPS.

[24]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[25]  Kwabena Boahen,et al.  Dynamic computation in a recurrent network of heterogeneous silicon neurons , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[26]  Johannes Schemmel,et al.  A VLSI Implementation of the Adaptive Exponential Integrate-and-Fire Neuron Model , 2010, NIPS.

[27]  Ralph Etienne-Cummings,et al.  Silicon Modeling of the Mihalaş–Niebur Neuron , 2011, IEEE Transactions on Neural Networks.

[28]  Dendritic computation of direction selectivity by retinal ganglion cells. , 2000, Science.

[29]  A. Tajalli,et al.  Subthreshold Source-Coupled Logic Circuits for Ultra-Low-Power Applications , 2008, IEEE Journal of Solid-State Circuits.

[30]  Johannes Schemmel,et al.  An analog dynamic memory array for neuromorphic hardware , 2013, 2013 European Conference on Circuit Theory and Design (ECCTD).

[31]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[32]  Robert A. Legenstein,et al.  Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[33]  Syed Ahmed Aamir,et al.  Mixed-Signal Circuit Implementation of Spiking Neuron Models , 2018 .

[34]  Johannes Schemmel,et al.  An accelerated analog neuromorphic hardware system emulating NMDA- and calcium-based non-linear dendrites , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[35]  E. Vittoz,et al.  An analytical MOS transistor model valid in all regions of operation and dedicated to low-voltage and low-current applications , 1995 .

[36]  B. Sakmann,et al.  Active propagation of somatic action potentials into neocortical pyramidal cell dendrites , 1994, Nature.

[37]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[38]  M. Konishi,et al.  A circuit for detection of interaural time differences in the brain stem of the barn owl , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[39]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[40]  Yingxue Wang,et al.  A Two-Dimensional Configurable Active Silicon Dendritic Neuron Array , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[41]  Johannes Schemmel,et al.  From LIF to AdEx neuron models: Accelerated analog 65 nm CMOS implementation , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[42]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[43]  N. Spruston,et al.  Diversity and dynamics of dendritic signaling. , 2000, Science.

[44]  René Schüffny,et al.  A Biological-Realtime Neuromorphic System in 28 nm CMOS Using Low-Leakage Switched Capacitor Circuits , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[45]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[46]  B. Richmond,et al.  Intrinsic dynamics in neuronal networks. I. Theory. , 2000, Journal of neurophysiology.

[47]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[48]  Christofer Toumazou,et al.  Bulk-drain connected load for subthreshold MOS current-mode logic , 2007 .

[49]  M. London,et al.  Dendritic computation. , 2005, Annual review of neuroscience.

[50]  A. Polsky,et al.  Synaptic Integration in Tuft Dendrites of Layer 5 Pyramidal Neurons: A New Unifying Principle , 2009, Science.

[51]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[52]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[53]  Paul Müller.  Modeling and verification for a scalable neuromorphic substrate , 2017 .

[54]  Yannick Bornat,et al.  A Library of Analog Operators Based on the Hodgkin-Huxley Formalism for the Design of Tunable, Real-Time, Silicon Neurons , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[55]  A. Cassidy,et al.  Dynamical digital silicon neurons , 2008, 2008 IEEE Biomedical Circuits and Systems Conference.

[56]  Christof Koch,et al.  The role of single neurons in information processing , 2000, Nature Neuroscience.

[57]  Johannes Schemmel,et al.  Towards biologically realistic multi-compartment neuron model emulation in analog VLSI , 2012, ESANN.

[58]  Craig T. Jin,et al.  A log-domain implementation of the Izhikevich neuron model , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[59]  J. Schiller,et al.  NMDA spikes in basal dendrites of cortical pyramidal neurons , 2000, Nature.

[60]  Paul E. Hasler,et al.  A bio-physically inspired silicon neuron , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[61]  Matthias Hock,et al.  Modern semiconductor technologies for neuromorphic hardware , 2014 .

[62]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[63]  Wulfram Gerstner,et al.  Firing patterns in the adaptive exponential integrate-and-fire model , 2008, Biological Cybernetics.

[64]  Jonathan Touboul,et al.  Dynamics and bifurcations of the adaptive exponential integrate-and-fire model , 2008, Biological Cybernetics.

[65]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[66]  Rodney J. Douglas,et al.  Forward- and backpropagation in a silicon dendrite , 2001, IEEE Trans. Neural Networks.

[67]  Giacomo Indiveri,et al.  A current-mode conductance-based silicon neuron for address-event neuromorphic systems , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[68]  Bernabe Linares-Barranco,et al.  A CMOS Implementation of Fitzhugh-Nagumo Neuron Model , 1990, ESSCIRC '90: Sixteenth European Solid-State Circuits Conference.

[69]  N. Spruston,et al.  Action potential initiation and backpropagation in neurons of the mammalian CNS , 1997, Trends in Neurosciences.

[70]  Johannes Schemmel,et al.  Six Networks on a Universal Neuromorphic Computing Substrate , 2012, Front. Neurosci..