A Review of Spiking Neuromorphic Hardware Communication Systems

Multiple neuromorphic systems use spiking neural networks (SNNs) to perform computation in a way that is inspired by concepts learned about the human brain. SNNs are artificial networks made up of neurons that fire a pulse, or spike, once the accumulated value of the inputs to the neuron exceeds a threshold. One of the most challenging parts of designing neuromorphic hardware is handling the vast degree of connectivity that neurons have with each other in the form of synaptic connections. This paper analyzes the neuromorphic systems Neurogrid, Braindrop, SpiNNaker, BrainScaleS, TrueNorth, Loihi, Darwin, and Dynap-SEL; and discusses the design of large scale spiking communication networks used in such systems. In particular, this paper looks at how each of these systems solved the challenges of forming packets with spiking information and how these packets are routed within the system. The routing of packets is analyzed at two scales: How the packets should be routed when traveling a short distance, and how the packets should be routed over longer global connections. Additional topics, such as the use of asynchronous circuits, robustness in communication, connection with a host machine, and network synchronization are also covered.

[1]  Catherine D. Schuman,et al.  Neuromorphic Computing : A Post-Moore ’ s Law Complementary Architecture , 2016 .

[2]  Kwabena Boahen,et al.  Braindrop: A Mixed-Signal Neuromorphic Architecture With a Dynamical Systems-Based Programming Model , 2019, Proceedings of the IEEE.

[3]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[4]  Enrico Macii,et al.  Flexible On-Line Reconfiguration of Multi-Core Neuromorphic Platforms , 2019, IEEE Transactions on Emerging Topics in Computing.

[5]  Steve Furber,et al.  Power-efficient simulation of detailed cortical microcircuits on SpiNNaker , 2012, Journal of Neuroscience Methods.

[6]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[7]  Alain J. Martin.  Programming in VLSI: from communicating processes to delay-insensitive circuits , 1991 .

[8]  Kwabena Boahen,et al.  A Neuromorph's Prospectus , 2017, Computing in Science & Engineering.

[9]  Karlheinz Meier,et al.  Introducing the Human Brain Project , 2011, FET.

[10]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[11]  Edith Beigné,et al.  Spiking Neural Networks Hardware Implementations and Challenges , 2019, ACM J. Emerg. Technol. Comput. Syst..

[12]  Gang Pan,et al.  Darwin: a neuromorphic hardware co-processor based on Spiking Neural Networks , 2015, Science China Information Sciences.

[13]  Narayan Srinivasa,et al.  Low-Power Neuromorphic Hardware for Signal Processing Applications: A review of architectural and system-level design approaches , 2019, IEEE Signal Processing Magazine.

[14]  Angel Jiménez-Fernandez,et al.  AER-based robotic closed-loop control system , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[15]  Philipp Häfliger,et al.  High-Speed Serial AER on FPGA , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[16]  Saeed Afshar,et al.  Event-based Sensing for Space Situational Awareness , 2019, The Journal of the Astronautical Sciences.

[17]  Jonathan Heathcote,et al.  Building and operating large-scale SpiNNaker machines , 2016 .

[18]  Johannes Schemmel,et al.  Wafer-scale integration of analog neural networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[19]  Frederico A. C. Azevedo,et al.  Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled‐up primate brain , 2009, The Journal of comparative neurology.

[20]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[21]  Kwabena Boahen,et al.  A Serial H-Tree Router for Two-Dimensional Arrays , 2018, 2018 24th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC).

[22]  Enrico Macii,et al.  Optimizing Network Traffic for Spiking Neural Network Simulations on Densely Interconnected Many-Core Neuromorphic Platforms , 2018, IEEE Transactions on Emerging Topics in Computing.

[23]  Steven M. Burns,et al.  The design of an asynchronous microprocessor , 1989, CARN.

[24]  Giacomo Indiveri,et al.  A serial communication infrastructure for multi-chip address event systems , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[25]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[26]  Andrew D. Brown,et al.  Biologically-Inspired Massively-Parallel Architectures - Computing Beyond a Million Processors , 2009, ACSD.

[27]  Andrew S. Cassidy,et al.  Cognitive computing programming paradigm: A Corelet Language for composing networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[28]  Catherine D. Schuman,et al.  Dynamic Adaptive Neural Network Array , 2014, UCNC.

[29]  Andrew S. Cassidy,et al.  TrueNorth Ecosystem for Brain-Inspired Computing: Scalable Systems, Software, and Applications , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[30]  Gert Cauwenberghs,et al.  Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain , 2018, Front. Neurosci..

[31]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[32]  Rodrigo Alvarez-Icaza,et al.  A Multicast Tree Router for Multichip Neuromorphic Systems , 2014, IEEE Transactions on Circuits and Systems I: Regular Papers.

[33]  John W. Backus,et al.  Can programming be liberated from the von Neumann style?: a functional style and its algebra of programs , 1978, CACM.

[34]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[35]  Jim D. Garside,et al.  A Programmable Adaptive Router for a GALS Parallel System , 2009, 2009 15th IEEE Symposium on Asynchronous Circuits and Systems.

[36]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[37]  Stephan Henker,et al.  A 32 GBit/s communication SoC for a waferscale neuromorphic system , 2012, Integr..

[38]  Catherine D. Schuman,et al.  A Survey of Neuromorphic Computing and Neural Networks in Hardware , 2017, ArXiv.

[39]  Stephan Hartmann,et al.  VLSI Implementation of a 2.8 Gevent/s Packet-Based AER Interface with Routing and Event Sorting Functionality , 2011, Front. Neurosci..

[40]  Alain J. Martin,et al.  Asynchronous Techniques for System-on-Chip Design , 2006, Proceedings of the IEEE.

[41]  Andrew D. Brown,et al.  On-chip and inter-chip networks for modeling large-scale neural systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[42]  Giacomo Indiveri,et al.  A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs) , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[43]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[44]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.