Low Cost Interconnected Architecture for the Hardware Spiking Neural Networks

A novel low cost interconnected architecture (LCIA) is proposed in this paper, which is an efficient solution for the neuron interconnections for the hardware spiking neural networks (SNNs). It is based on an all-to-all connection that takes each paired input and output nodes of multi-layer SNNs as the source and destination of connections. The aim is to maintain an efficient routing performance under low hardware overhead. A Networks-on-Chip (NoC) router is proposed as the fundamental component of the LCIA, where an effective scheduler is designed to address the traffic challenge due to irregular spikes. The router can find requests rapidly, make the arbitration decision promptly, and provide equal services to different network traffic requests. Experimental results show that the LCIA can manage the intercommunication of the multi-layer neural networks efficiently and have a low hardware overhead which can maintain the scalability of hardware SNNs.

[1]  Joe McGeehan,et al.  Biologically compatible neural networks with reconfigurable hardware , 2015, Microprocess. Microsystems.

[2]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[3]  Liam McDaid,et al.  A Reconfigurable and Biologically Inspired Paradigm for Computation Using Network-On-Chip and Spiking Neural Networks , 2009, Int. J. Reconfigurable Comput..

[4]  Stephen B. Furber,et al.  Virtual synaptic interconnect using an asynchronous network-on-chip , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[5]  Camel Tanougast,et al.  CuNoC: A dynamic scalable communication structure for dynamically reconfigurable FPGAs , 2009, Microprocess. Microsystems.

[6]  Philip H. W. Leong,et al.  FPGA implementation of biologically-inspired auto-associative memory , 2012 .

[7]  Ravi Shankar,et al.  Survey of Network on Chip (NoC) Architectures & Contributions , 2009 .

[8]  Jim Harkin,et al.  An Efficient, Low-Cost Routing Architecture for Spiking Neural Network Hardware Implementations , 2018, Neural Processing Letters.

[9]  Jim Harkin,et al.  Efficient neuron architecture for FPGA-based spiking neural networks , 2016, 2016 27th Irish Signals and Systems Conference (ISSC).

[10]  Murray Shanahan,et al.  Accelerated simulation of spiking neural networks using GPUs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[11]  Dong Yang,et al.  Influence of dietary supplementation with Bacillus licheniformis and Saccharomyces cerevisiae as alternatives to monensin on growth performance, antioxidant, immunity, ruminal fermentation and microbial diversity of fattening lambs , 2018, Scientific Reports.

[12]  Liam McDaid,et al.  Hardware spiking neural network prototyping and application , 2011, Genetic Programming and Evolvable Machines.

[13]  Laurent Perrinet.  Sparse spike coding : applications of neuroscience to the processing of natural images , 2008, SPIE Photonics Europe.

[14]  Liam McDaid,et al.  An Efficient, High-Throughput Adaptive NoC Router for Large Scale Spiking Neural Network Hardware Implementations , 2010, ICES.

[15]  Luca Benini,et al.  Networks on Chips : A New SoC Paradigm , 2022 .

[16]  Masoud Daneshtalab,et al.  CuPAN - High Throughput On-chip Interconnection for Neural Networks , 2015, ICONIP.

[17]  Steve B. Furber,et al.  Modeling Spiking Neural Networks on SpiNNaker , 2010, Computing in Science & Engineering.

[18]  Arindam Basu,et al.  Neural Dynamics in Reconfigurable Silicon , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[19]  D. A. Baxter,et al.  Simulator for neural networks and action potentials. , 2007, Methods in molecular biology.

[20]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[21]  Robert H. Lee,et al.  An FPGA-based approach to high-speed simulation of conductance-based neuron models , 2007, Neuroinformatics.

[22]  Liam McDaid,et al.  Scalable Hierarchical Network-on-Chip Architecture for Spiking Neural Network Hardware Implementations , 2013, IEEE Transactions on Parallel and Distributed Systems.

[23]  KasabovNikola,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008 .

[24]  Yu-Ping Chen,et al.  A small-granularity solution on fault-tolerant in 2D-Mesh Network-on-Chip , 2010, 2010 10th IEEE International Conference on Solid-State and Integrated Circuit Technology.

[25]  Jim D. Garside,et al.  SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation , 2013, IEEE Journal of Solid-State Circuits.

[26]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[27]  Reinhard Männer,et al.  The ENABLE Machine - A Systolic Second Level Trigger Processor for Track Finding , 1992 .

[28]  Peng Li,et al.  Simulation of large neuronal networks with biophysically accurate models on graphics processors , 2011, The 2011 International Joint Conference on Neural Networks.

[29]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[30]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[31]  William J. Dally,et al.  Principles and Practices of Interconnection Networks , 2004 .

[32]  André van Schaik,et al.  Neuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[33]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[34]  Mei Yang,et al.  Algorithm-Hardware Codesign of Fast Parallel Round-Robin Arbiters , 2007, IEEE Transactions on Parallel and Distributed Systems.

[35]  Hugues Berry,et al.  Robustness of STDP to spike timing jitter , 2018, Scientific Reports.

[36]  Laurence R. Rilett,et al.  Spectral Basis Neural Networks for Real-Time Travel Time Forecasting , 1999 .

[37]  Steve B. Furber,et al.  Breaking the millisecond barrier on SpiNNaker: implementing asynchronous event-based plastic models with microsecond resolution , 2015, Front. Neurosci..

[38]  Hyoukjun Kwon,et al.  Rethinking NoCs for spatial neural network accelerators , 2017, 2017 Eleventh IEEE/ACM International Symposium on Networks-on-Chip (NOCS).

[39]  B Mc Ginley,et al.  Exploring the evolution of NoC-based Spiking Neural Networks on FPGAs , 2009, 2009 International Conference on Field-Programmable Technology.

[40]  Johannes Schemmel,et al.  Wafer-scale integration of analog neural networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[41]  Robert A. Legenstein,et al.  Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[42]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[43]  Gerard J. M. Smit,et al.  Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network , 2013, Neural Processing Letters.

[44]  Andres Upegui,et al.  An FPGA platform for on-line topology exploration of spiking neural networks , 2005, Microprocess. Microsystems.

[45]  Liam McDaid,et al.  Advancing interconnect density for spiking neural network hardware implementations using traffic-aware adaptive network-on-chip routers , 2012, Neural Networks.

[46]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[47]  Liz Campbell,et al.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model , 2018 .

[48]  Eustace Painkras,et al.  A chip multiprocessor for a large-scale neural simulator , 2013 .

[49]  Catherine D. Schuman,et al.  A Survey of Neuromorphic Computing and Neural Networks in Hardware , 2017, ArXiv.

[50]  Alois Knoll,et al.  Towards a neuromorphic implementation of hierarchical temporal memory on SpiNNaker , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[51]  Onur Mutlu,et al.  A case for bufferless routing in on-chip networks , 2009, ISCA '09.

[52]  Jim Harkin,et al.  Fault-Tolerant Networks-on-Chip Routing With Coarse and Fine-Grained Look-Ahead , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[53]  Subutai Ahmad,et al.  Porting HTM Models to the Heidelberg Neuromorphic Computing Platform , 2015, ArXiv.

[54]  Nikil D. Dutt,et al.  Efficient simulation of large-scale Spiking Neural Networks using CUDA graphics processors , 2009, 2009 International Joint Conference on Neural Networks.

[55]  Ramón González-Camarena,et al.  Acoustic thoracic image of crackle sounds using linear and nonlinear processing techniques , 2010, Medical & Biological Engineering & Computing.

[56]  Jim Harkin,et al.  Low cost fault-tolerant routing algorithm for Networks-on-Chip , 2015, Microprocess. Microsystems.

[57]  Steve B. Furber,et al.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model , 2018, Front. Neurosci..

[58]  W. Dally,et al.  Route packets, not wires: on-chip interconnection networks , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[59]  S. P. Simon,et al.  A New Spike Based Neural Network for Short-Term Electrical Load Forecasting , 2012, 2012 Fourth International Conference on Computational Intelligence and Communication Networks.

[60]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[61]  Qingxiang Wu,et al.  Processing visual stimuli using hierarchical spiking neural networks , 2008, Neurocomputing.

[62]  Hyoukjun Kwon,et al.  MAESTRO: An Open-source Infrastructure for Modeling Dataflows within Deep Learning Accelerators , 2018, ArXiv.

[63]  Alexandre Yakovlev,et al.  Connection-centric network for spiking neural networks , 2009, 2009 3rd ACM/IEEE International Symposium on Networks-on-Chip.

[64]  Maxime Pelcat,et al.  Exploring the performance of partially reconfigurable point-to-point interconnects , 2017, 2017 12th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC).

[65]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[66]  Radu Marculescu,et al.  Application-specific buffer space allocation for networks-on-chip router design , 2004, IEEE/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004..

[67]  Mitsuhisa Sato,et al.  Extremely Scalable Spiking Neuronal Network Simulation Code: From Laptops to Exascale Computers , 2018, Front. Neuroinform..

[68]  Alain Greiner,et al.  A reconfigurable routing algorithm for a fault-tolerant 2D-Mesh Network-on-Chip , 2008, 2008 45th ACM/IEEE Design Automation Conference.