A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems

In this article, we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware mapping process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware–software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware mapping software. The functionality and flexibility of the latter is proven with a variety of experimental results.

[1]  Martin Rehn,et al.  Attractor dynamics in a modular network model of neocortex , 2006, Network.

[2]  Johannes Schemmel,et al.  Simulator-like exploration of cortical network architectures with a mixed-signal VLSI system , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[3]  A. Aertsen,et al.  Conditions for Propagating Synchronous Spiking and Asynchronous Firing Rates in a Cortical Network Model , 2008, The Journal of Neuroscience.

[4]  Stephan Henker,et al.  Optimized queue based communication in VLSI using a weakly ordered binary heap , 2010, Proceedings of the 17th International Conference Mixed Design of Integrated Circuits and Systems - MIXDES 2010.

[5]  Johannes Schemmel,et al.  Realizing biological spiking network models in a configurable wafer-scale hardware system , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[6]  Bernabé Linares-Barranco,et al.  A Spatial Contrast Retina With On-Chip Calibration for Neuromorphic Spike-Based AER Vision Systems , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[8]  René Schüffny,et al.  A Software Framework for Mapping Neural Networks to a Wafer-scale Neuromorphic Hardware System , 2016, ANNIIP.

[9]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[10]  M. Steriade.  Neuronal Substrates of Sleep and Epilepsy , 2003 .

[11]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[12]  Matthieu Gilson,et al.  Spike-timing-dependent plasticity for neurons with recurrent connections , 2007, Biological Cybernetics.

[13]  Michael L. Hines,et al.  The NEURON Book , 2006 .

[14]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[15]  J. Schemmel,et al.  Wafer-scale VLSI implementations of pulse coupled neural networks , 2007 .

[16]  Ad Aertsen,et al.  Stable propagation of synchronous spiking in cortical neural networks , 1999, Nature.

[17]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[18]  Alain Destexhe,et al.  Activated cortical states: Experiments, analyses and models , 2007, Journal of Physiology-Paris.

[19]  A QoS network architecture to interconnect large-scale VLSI neural networks , 2009, 2009 International Joint Conference on Neural Networks.

[20]  Arvind Kumar,et al.  Emergence of population synchrony in a layered network of the cat visual cortex , 2007, Neurocomputing.

[21]  Johannes Schemmel,et al.  Modeling Synaptic Plasticity within Networks of Highly Accelerated I&F Neurons , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[22]  Alain Destexhe,et al.  Self-sustained Asynchronous Irregular States and Up–down States in Thalamic, Cortical and Thalamocortical Networks of Nonlinear Integrate-and-fire Neurons , 2022 .

[23]  Ad Aertsen,et al.  Gating of Signal Propagation in Spiking Neural Networks by Balanced and Correlated Excitation and Inhibition , 2010, The Journal of Neuroscience.

[24]  Godred Fairhurst,et al.  Advice to link designers on link Automatic Repeat reQuest (ARQ) , 2002, RFC.

[25]  D. Buxhoeveden,et al.  The Minicolumn and Evolution of the Brain , 2002, Brain, Behavior and Evolution.

[26]  Yannick Bornat,et al.  Weights Convergence and Spikes Correlation in an Adaptive Neural Network Implemented on VLSI , 2008, BIOSIGNALS.

[27]  K. Meier,et al.  A new VLSI model of neural microcircuits including spike time dependent plasticity , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[28]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[29]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[30]  Johannes Schemmel,et al.  Compensating Inhomogeneities of Neuromorphic VLSI Devices Via Short-Term Synaptic Plasticity , 2010, Front. Comput. Neurosci..

[31]  Kwabena Boahen,et al.  Dynamic computation in a recurrent network of heterogeneous silicon neurons , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[32]  Johannes Schemmel,et al.  Wafer-scale integration of analog neural networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[33]  Daniel Brüderle,et al.  Neuroscientific modeling with a mixed signal VLSI hardware system , 2009 .

[34]  Eugenio Culurciello,et al.  Fall detection using an address-event temporal contrast vision sensor , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[35]  Yannick Bornat,et al.  Neuromimetic ICs with analog cores: an alternative for simulating spiking neural networks , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[36]  D. Labiner,et al.  Neuronal Substrates of Sleep and Epilepsy , 2004 .

[37]  R. Douglas,et al.  A Quantitative Map of the Circuit of Cat Primary Visual Cortex , 2004, The Journal of Neuroscience.

[38]  Thomas Netter,et al.  A robotic aircraft that follows terrain using a neuromorphic eye , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Shih-Chii Liu,et al.  A Hardware/Software Framework for Real-Time Spiking Systems , 2005, ICANN.

[40]  Henry Markram,et al.  Minimal Hodgkin–Huxley type models for different classes of cortical and thalamic neurons , 2008, Biological Cybernetics.

[41]  Johannes Schemmel,et al.  Training convolutional networks of threshold neurons suited for low-power hardware implementation , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[42]  Tobi Delbrück,et al.  A silicon early visual system as a model animal , 2004, Vision Research.

[43]  Ad Aertsen,et al.  Functional consequences of correlated excitatory and inhibitory conductances in cortical networks , 2010, Journal of Computational Neuroscience.

[44]  Giacomo Indiveri,et al.  Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition , 2009, Cognitive Computation.

[45]  René Schüffny,et al.  A graph theoretical approach for a multistep mapping software for the FACETS project , 2008 .

[46]  Philipp Häfliger,et al.  High-Speed Serial AER on FPGA , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[47]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[48]  Tor Sverre Lande,et al.  An analog floating-gate memory in a standard digital technology , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[49]  Anders Lansner,et al.  Bistable, Irregular Firing and Population Oscillations in a Modular Attractor Memory Network , 2010, PLoS Comput. Biol..

[50]  René Schüffny,et al.  GMPath - A Path Language for Navigation, Information Query and Modification of Data Graphs , 2010, ANNIIP.

[51]  A. Aertsen,et al.  Spiking activity propagation in neuronal networks: reconciling different perspectives on neural coding , 2010, Nature Reviews Neuroscience.

[52]  A. Garenne,et al.  A Real-Time Closed-Loop Setup for Hybrid Neural Networks , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[53]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[54]  Vittorio Dante,et al.  PCI-AER hardware and software for interfacing to address-event based neuromorphic systems , 2005 .

[55]  Fernando Díaz del Río,et al.  Real time multiple objects tracking based on a bio-inspired processing cascade architecture , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[56]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[57]  V. Mountcastle.  The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[58]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[59]  Misha Mahowald,et al.  A silicon model of early visual processing , 1993, Neural Networks.

[60]  Hou Zu-bing,et al.  Gigabit ethernet technology and applications , 2002 .

[61]  Wulfram Gerstner,et al.  Firing patterns in the adaptive exponential integrate-and-fire model , 2008, Biological Cybernetics.

[62]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[63]  Klaus Schuch,et al.  PCSIM: A Parallel Simulation Environment for Neural Circuits Fully Integrated with Python , 2008, Frontiers Neuroinformatics.

[64]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[65]  W. Regehr,et al.  Short-term synaptic plasticity. , 2002, Annual review of physiology.

[66]  Johannes Schemmel,et al.  Implementing Synaptic Plasticity in a VLSI Spiking Neural Network Model , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[67]  R. Shapley,et al.  An egalitarian network model for the emergence of simple and complex cells in visual cortex , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[68]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[69]  Hans Petter Langtangen,et al.  Python scripting for computational science , 2004 .

[70]  Stefan Rotter,et al.  Higher-Order Statistics of Input Ensembles and the Response of Simple Model Neurons , 2003, Neural Computation.

[71]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[72]  Johannes Schemmel,et al.  High-conductance states on a neuromorphic hardware system , 2009, 2009 International Joint Conference on Neural Networks.

[73]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[74]  Ray Horak,et al.  Telecommunications and Data Communications Handbook , 2007 .

[75]  Johannes Schemmel,et al.  A VLSI Implementation of the Adaptive Exponential Integrate-and-Fire Neuron Model , 2010, NIPS.

[76]  A. Aertsen,et al.  Gating of signal propagation in spiking neural networks by balanced and correlated excitation and inhibition , 2010 .

[77]  Ralph Etienne-Cummings,et al.  Toward biomorphic control using custom aVLSI CPG chips , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[78]  Andrew P Davison,et al.  Learning Cross-Modal Spatial Transformations through Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[79]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[80]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[81]  B. Connors,et al.  Intrinsic firing patterns of diverse neocortical neurons , 1990, Trends in Neurosciences.

[82]  Michael L. Hines,et al.  Neuroinformatics Original Research Article Neuron and Python , 2022 .

[83]  Giacomo Indiveri,et al.  Neuromorphic VLSI Models of Selective Attention: From Single Chip Vision Sensors to Multi-chip Systems , 2008, Sensors.

[84]  Stephan Henker,et al.  Highly integrated packet-based AER communication infrastructure with 3Gevent/S throughput , 2010, 2010 17th IEEE International Conference on Electronics, Circuits and Systems.

[85]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[86]  Moritz Helias,et al.  Neuroinformatics Original Research Article Pynest: a Convenient Interface to the Nest Simulator , 2022 .

[87]  Tobi Delbrück,et al.  AER Building Blocks for Multi-Layer Multi-Chip Neuromorphic Vision Systems , 2005, NIPS.

[88]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[89]  D. Contreras,et al.  Mechanisms underlying the synchronizing action of corticothalamic feedback through inhibition of thalamic relay cells. , 1998, Journal of neurophysiology.

[90]  Philipp Häfliger.  Adaptive WTA With an Analog VLSI Neuromorphic Learning Chip , 2007, IEEE Transactions on Neural Networks.

[91]  P. J. Sjöström,et al.  Dendritic excitability and synaptic plasticity. , 2008, Physiological reviews.

[92]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[93]  Johannes Schemmel,et al.  Neuroinformatics Original Research Article Establishing a Novel Modeling Tool: a Python-based Interface for a Neuromorphic Hardware System , 2022 .

[94]  Extending a Hardware Neural Network Beyond Chip Boundaries , 2009 .

[95]  Moshe Abeles,et al.  On Embedding Synfire Chains in a Balanced Network , 2003, Neural Computation.

[96]  Wolfgang Maass,et al.  Reward-Modulated Hebbian Learning of Decision Making , 2010, Neural Computation.

[97]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.