SENECA: building a fully digital neuromorphic processor, design trade-offs and challenges

Neuromorphic processors aim to emulate the biological principles of the brain to achieve high efficiency with low power consumption. However, the lack of flexibility in most neuromorphic architecture designs results in significant performance loss and inefficient memory usage when mapping various neural network algorithms. This paper proposes SENECA, a digital neuromorphic architecture that balances the trade-offs between flexibility and efficiency using a hierarchical-controlling system. A SENECA core contains two controllers, a flexible controller (RISC-V) and an optimized controller (Loop Buffer). This flexible computational pipeline allows for deploying efficient mapping for various neural networks, on-device learning, and pre-post processing algorithms. The hierarchical-controlling system introduced in SENECA makes it one of the most efficient neuromorphic processors, along with a higher level of programmability. This paper discusses the trade-offs in digital neuromorphic processor design, explains the SENECA architecture, and provides detailed experimental results when deploying various algorithms on the SENECA platform. The experimental results show that the proposed architecture improves energy and area efficiency and illustrates the effect of various trade-offs in algorithm design. A SENECA core consumes 0.47 mm2 when synthesized in the GF-22 nm technology node and consumes around 2.8 pJ per synaptic operation. SENECA architecture scales up by connecting many cores with a network-on-chip. The SENECA platform and the tools used in this project are freely available for academic research upon request.

[1]  M. Verhelst,et al.  DepFiN: A 12-nm Depth-First, High-Resolution CNN Processor for IO-Efficient Inference , 2023, IEEE Journal of Solid-State Circuits.

[2]  M. Verhelst,et al.  Towards Heterogeneous Multi-core Accelerators Exploiting Fine-grained Scheduling of Layer-Fused Deep Neural Networks , 2022, ArXiv.

[3]  Yexin Yan,et al.  E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware , 2022, Frontiers in Neuroscience.

[4]  A. Yousefzadeh,et al.  Delta Activation Layer exploits temporal sparsity for efficient embedded video processing , 2022, 2022 International Joint Conference on Neural Networks (IJCNN).

[5]  Federico Corradi,et al.  SENeCA: Scalable Energy-efficient Neuromorphic Computer Architecture , 2022, 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS).

[6]  D. Ielmini,et al.  A CMOS–memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity , 2022, Philosophical Transactions of the Royal Society A.

[7]  Krishna Teja Chitty-Venkata,et al.  Neural Architecture Search Survey: A Hardware Perspective , 2022, ACM Comput. Surv..

[8]  Sara Summerton,et al.  Online learning in SNNs with e-prop and Neuromorphic Hardware , 2022, NICE.

[9]  Lei Deng,et al.  Spiking Neural Network Integrated Circuits: A Review of Trends and Future Directions , 2022, 2022 IEEE Custom Integrated Circuits Conference (CICC).

[10]  G. Indiveri,et al.  ReckOn: A 28nm Sub-mm2 Task-Agnostic Spiking Recurrent Neural Network Processor Enabling On-Chip Learning over Second-Long Timescales , 2022, 2022 IEEE International Solid- State Circuits Conference (ISSCC).

[11]  G. Beyer,et al.  3D SoC integration, beyond 2.5D chiplets , 2021, 2021 IEEE International Electron Devices Meeting (IEDM).

[12]  M. Debbah,et al.  On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks , 2021, ICC 2022 - IEEE International Conference on Communications.

[13]  Neelesh Kumar,et al.  BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks , 2021, ArXiv.

[14]  Hamza Ouarnoughi,et al.  Hardware-Aware Neural Architecture Search: Survey and Taxonomy , 2021, IJCAI.

[15]  Louis Tao,et al.  The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware , 2021, ArXiv.

[16]  A. Dosovitskiy,et al.  MLP-Mixer: An all-MLP Architecture for Vision , 2021, NeurIPS.

[17]  S. Furber,et al.  The SpiNNaker 2 Processing Element Architecture for Hybrid Digital Neuromorphic Computing , 2021, ArXiv.

[18]  Sander M. Bohte,et al.  Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks , 2021, Nature Machine Intelligence.

[19]  Mohsin M. Jamali,et al.  Multi-Object Detection and Tracking, Based on DNN, for Autonomous Vehicles: A Review , 2021, IEEE Sensors Journal.

[20]  W. Maass,et al.  Online spatio-temporal learning in deep neural networks , 2020, ArXiv.

[21]  Benjamin Recht,et al.  Evaluating Machine Accuracy on ImageNet , 2020, ICML.

[22]  Adrian Alan Pol,et al.  Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors , 2020, Nature Machine Intelligence.

[23]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[24]  Maurizio Martina,et al.  An Efficient Spiking Neural Network for Recognizing Gestures with a DVS Camera on the Loihi Neuromorphic Processor , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[25]  Sebastian Billaudelle,et al.  Accelerated Analog Neuromorphic Computing , 2020, ArXiv.

[26]  Orlando Moreira,et al.  NeuronFlow: a neuromorphic processor architecture for Live AI applications , 2020, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[27]  Wei He,et al.  Tianjic: A Unified and Scalable Chip Bridging Spike-Based and Continuous Neural Computation , 2020, IEEE Journal of Solid-State Circuits.

[28]  Steve Furber,et al.  SpiNNaker 2: A 10 Million Core Processor System for Brain Simulation and Machine Learning , 2019, ArXiv.

[29]  Bernabe Linares-Barranco,et al.  Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity , 2019, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[30]  Tiberiu T. Cocias,et al.  A survey of deep learning techniques for autonomous driving , 2019, J. Field Robotics.

[31]  Wolfgang Maass,et al.  A solution to the learning dilemma for recurrent networks of spiking neurons , 2019, Nature Communications.

[32]  Pradeep Dubey,et al.  A Study of BFLOAT16 for Deep Learning Training , 2019, ArXiv.

[33]  Francky Catthoor,et al.  Exploration of Segmented Bus As Scalable Global Interconnect for Neuromorphic Computing , 2019, ACM Great Lakes Symposium on VLSI.

[34]  Davide Zambrano,et al.  Sparse Computation in Adaptive Spiking Neural Networks , 2019, Front. Neurosci..

[35]  Bernabé Linares-Barranco,et al.  Scene Context Classification with Event-Driven Spiking Deep Neural Networks , 2018, 2018 25th IEEE International Conference on Electronics, Circuits and Systems (ICECS).

[36]  Aytaç Altan,et al.  Real-Time Control based on NARX Neural Network of Hexarotor UAV with Load Transporting System for Path Tracking , 2018, 2018 6th International Conference on Control Engineering & Information Technology (CEIT).

[37]  David Bol,et al.  A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS , 2018, IEEE Transactions on Biomedical Circuits and Systems.

[38]  Jing Li,et al.  Adaptive Quantization of Neural Networks , 2018, ICLR.

[39]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[40]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Xin Wang,et al.  Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks , 2017, NIPS.

[42]  Marian Verhelst,et al.  Minimum energy quantized neural networks , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[43]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[44]  Luca Benini,et al.  Slow and steady wins the race? A comparison of ultra-low-power RISC-V cores for Internet-of-Things applications , 2017, 2017 27th International Symposium on Power and Timing Modeling, Optimization and Simulation (PATMOS).

[45]  Giacomo Indiveri,et al.  A Scalable Multicore Architecture With Heterogeneous Memory Structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs) , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[46]  Bernabé Linares-Barranco,et al.  On Multiple AER Handshaking Channels Over High-Speed Bit-Serial Bidirectional LVDS Links With Flow-Control and Clock-Correction on Commercial FPGAs for Scalable Neuromorphic Systems , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[47]  Tobi Delbrück,et al.  A Low Power, Fully Event-Based Gesture Recognition System , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[49]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[50]  Bernabé Linares-Barranco,et al.  Hardware implementation of convolutional STDP for on-line visual feature learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[51]  Jeffrey L. Krichmar,et al.  A self-driving robot using deep convolutional neural networks on neuromorphic hardware , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[52]  Luca Benini,et al.  Power, Area, and Performance Optimization of Standard Cell Memory Arrays Through Controlled Placement , 2016, TODE.

[53]  Miguel A L Nicolelis,et al.  Embedding a Panoramic Representation of Infrared Light in the Adult Rat Somatosensory Cortex through a Sensory Neuroprosthesis , 2016, The Journal of Neuroscience.

[54]  Bernabé Linares-Barranco,et al.  Fast Predictive Handshaking in Synchronous FPGAs for Fully Asynchronous Multisymbol Chip Links: Application to SpiNNaker 2-of-7 Links , 2016, IEEE Transactions on Circuits and Systems II: Express Briefs.

[55]  Stephen Richardson,et al.  Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era , 2016, IEEE Design & Test.

[56]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[57]  Bernabé Linares-Barranco,et al.  Fast Pipeline 128×128 pixel spiking convolution core for event-driven vision processing in FPGAs , 2015, 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP).

[58]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[59]  Steve B. Furber,et al.  Power analysis of large-scale, real-time neural networks on SpiNNaker , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[60]  Andrew S. Cassidy,et al.  Building block of a programmable neuromorphic substrate: A digital neurosynaptic core , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[61]  Philipp Häfliger,et al.  A time domain winner-take-all network of integrate-and-fire neurons , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[62]  Joseph E LeDoux Emotion, memory and the brain. , 1994, Scientific American.

[63]  J. Mink,et al.  Ratio of central nervous system to body metabolism in vertebrates: its constancy and functional basis. , 1981, The American journal of physiology.

[64]  Michael J. Flynn,et al.  Some Computer Organizations and Their Effectiveness , 1972, IEEE Transactions on Computers.

[65]  K. Michmizos,et al.  Decoding EEG With Spiking Neural Networks on Neuromorphic Hardware , 2022, Trans. Mach. Learn. Res..

[66]  Mingsong Lv,et al.  Efficient DNN Execution on Intermittently-Powered IoT Devices With Depth-First Inference , 2022, IEEE Access.

[67]  Martin Volker Butz,et al.  Learning What and Where - Unsupervised Disentangling Location and Identity Tracking , 2022, ArXiv.

[68]  Tony Stansfield 4 Improving the Efficiency of AI Applications Using In-Memory Computation , 2022 .

[69]  Xingzhong Xiong,et al.  A Survey of Intelligent Chip Design Research Based on Spiking Neural Networks , 2022, IEEE Access.

[70]  R. Nagisetty,et al.  2.5D and 3D Heterogeneous Integration: Emerging applications , 2021, IEEE Solid-State Circuits Magazine.

[71]  Bernabé Linares-Barranco,et al.  Event-driven implementation of deep spiking convolutional neural networks for supervised classification using the SpiNNaker neuromorphic platform , 2020, Neural Networks.

[72]  Dan Alistarh,et al.  Inducing and Exploiting Activation Sparsity for Fast Neural Network Inference , 2020 .

[73]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[74]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[75]  G. Kreiman,et al.  Measuring sparseness in the brain: comment on Bowers (2009). , 2010, Psychological review.