On-Chip Error-Triggered Learning of Multi-Layer Memristive Spiking Neural Networks

Recent breakthroughs in neuromorphic computing show that local forms of gradient descent learning are compatible with Spiking Neural Networks (SNNs) and synaptic plasticity. Although SNNs can be scalably implemented using neuromorphic VLSI, an architecture that can learn using gradient-descent in situ is still missing. In this paper, we propose a local, gradient-based, error-triggered learning algorithm with online ternary weight updates. The proposed algorithm enables online training of multi-layer SNNs with memristive neuromorphic hardware showing a small loss in the performance compared with the state-of-the-art. We also propose a hardware architecture based on memristive crossbar arrays to perform the required vector-matrix multiplications. The necessary peripheral circuitry including presynaptic, post-synaptic and write circuits required for online training, have been designed in the subthreshold regime for power saving with a standard 180 nm CMOS process.

[1]  Melika Payvand,et al.  Error-triggered Three-Factor Learning Dynamics for Crossbar Arrays , 2019, 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS).

[2]  Themis Prodromakis,et al.  Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning , 2016, Front. Neurosci..

[3]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[4]  Steve B. Furber,et al.  Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[5]  Theodore Antonakopoulos,et al.  Mixed-Precision Deep Learning Based on Computational Memory , 2020, Frontiers in Neuroscience.

[6]  Tobi Delbrück,et al.  A Low Power, Fully Event-Based Gesture Recognition System , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Garrick Orchard,et al.  Online Few-Shot Gesture Learning on a Neuromorphic Processor , 2020, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[8]  Simone Balatti,et al.  A 2-transistor/1-resistor artificial synapse capable of communication and stochastic learning in neuromorphic systems , 2015, Front. Neurosci..

[9]  Daniele Ielmini,et al.  Resistive switching memories based on metal oxides: mechanisms, reliability and scaling , 2016 .

[10]  Arild Nøkland,et al.  Direct Feedback Alignment Provides Learning in Deep Neural Networks , 2016, NIPS.

[11]  A S Spinelli,et al.  Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity , 2017, Scientific Reports.

[12]  Ahmed M. Eltawil,et al.  Independent Component Analysis Using RRAMs , 2019, IEEE Transactions on Nanotechnology.

[13]  Peng Lin,et al.  Reinforcement learning with analogue memristor arrays , 2019, Nature Electronics.

[14]  T. Delbruck.  'Bump' circuits for computing similarity and dissimilarity of analog voltages , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[15]  David Bol,et al.  A 0.086-mm2 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS , 2019, IEEE Trans. Biomed. Circuits Syst..

[16]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[17]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[18]  Bradley A. Minch.  A simple variable-width CMOS bump circuit , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[19]  Jim D. Garside,et al.  SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation , 2013, IEEE Journal of Solid-State Circuits.

[20]  Johannes Schemmel,et al.  Demonstrating Hybrid Learning in a Flexible Neuromorphic Hardware System , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[21]  E. Vianello,et al.  Bio-Inspired Stochastic Computing Using Binary CBRAM Synapses , 2013, IEEE Transactions on Electron Devices.

[22]  J. Yang,et al.  Memristive crossbar arrays for brain-inspired computing , 2019, Nature Materials.

[23]  Ronald J. Williams,et al.  Gradient-based learning algorithms for recurrent networks and their computational complexity , 1995 .

[24]  Shimeng Yu,et al.  Low-Energy Robust Neuromorphic Computation Using Synaptic Devices , 2012, IEEE Transactions on Electron Devices.

[25]  Jacques Kaiser,et al.  Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE) , 2018, Frontiers in Neuroscience.

[26]  Yann LeCun,et al.  Large Scale Online Learning , 2003, NIPS.

[27]  Alex Graves,et al.  Decoupled Neural Interfaces using Synthetic Gradients , 2016, ICML.

[28]  Adam Santoro,et al.  Backpropagation and the brain , 2020, Nature Reviews Neuroscience.

[29]  Giacomo Indiveri,et al.  Spike-Based Plasticity Circuits for Always-on On-Line Learning in Neuromorphic Systems , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[30]  Manuel Le Gallo,et al.  Memory devices and applications for in-memory computing , 2020, Nature Nanotechnology.

[31]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[32]  David Bol,et al.  A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28-nm CMOS , 2018, IEEE Transactions on Biomedical Circuits and Systems.

[33]  Colin J. Akerman,et al.  Random synaptic feedback weights support error backpropagation for deep learning , 2016, Nature Communications.

[34]  Giacomo Indiveri,et al.  Hybrid CMOS-RRAM Neurons with Intrinsic Plasticity , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[35]  Tayfun Gokmen,et al.  Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices , 2017, Front. Neurosci..

[36]  Giacomo Indiveri,et al.  A VLSI neuromorphic device for implementing spike-based neural networks , 2011, WIRN.

[37]  Evangelos Eleftheriou,et al.  Deep learning incorporating biologically inspired neural dynamics and in-memory computing , 2020, Nature Machine Intelligence.

[38]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[39]  Gert Cauwenberghs,et al.  Hierarchical Address Event Routing for Reconfigurable Large-Scale Neuromorphic Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[41]  Pierre Baldi,et al.  Learning in the machine: The symmetries of the deep learning channel , 2017, Neural Networks.

[42]  Giacomo Indiveri,et al.  A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation , 2018, Faraday discussions.

[43]  Thomas P. Parnell,et al.  Temporal correlation detection using computational phase-change memory , 2017, Nature Communications.

[44]  Tobias Delbrück,et al.  Frame-free dynamic digital vision , 2008 .

[45]  E. Neftci,et al.  Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective , 2019, Memristive Devices for Brain-Inspired Computing.

[46]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[47]  Claire Fenouillet-Beranger,et al.  Improvement of performances HfO 2 -based RRAM from elementary cell to 16 kb demonstrator by introduction of thin layer of Al 2 O 3 , 2016 .

[48]  Mohammed E. Fouda,et al.  Effect of Asymmetric Nonlinearity Dynamics in RRAMs on Spiking Neural Network Performance , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[49]  Surya Ganguli,et al.  SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks , 2017, Neural Computation.

[50]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[51]  Friedemann Zenke,et al.  Brain-Inspired Learning on Neuromorphic Substrates , 2020, Proceedings of the IEEE.

[52]  Chiara Bartolozzi,et al.  Synaptic Dynamics in Analog VLSI , 2007, Neural Computation.

[53]  Arild Nøkland,et al.  Training Neural Networks with Local Error Signals , 2019, ICML.

[54]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[55]  Jongkil Park,et al.  A 65k-neuron 73-Mevents/s 22-pJ/event asynchronous micro-pipelined integrate-and-fire array transceiver , 2014, 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings.

[56]  Giacomo Indiveri,et al.  A Systematic Method for Configuring VLSI Networks of Spiking Neurons , 2011, Neural Computation.

[57]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[58]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[59]  F. Merrikh Bayat,et al.  Spike-timing-dependent plasticity learning of coincidence detection with passively integrated memristive circuits , 2018, Nature Communications.

[60]  Yannik Stradmann,et al.  Training spiking multi-layer networks with surrogate gradients on an analog neuromorphic substrate , 2020, ArXiv.

[61]  Piotr Dudek,et al.  Gradient-descent-based learning in memristive crossbar arrays , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[62]  Gert Cauwenberghs,et al.  Neural and Synaptic Array Transceiver: A Brain-Inspired Computing Framework for Embedded Learning , 2017, Front. Neurosci..

[63]  Rodney J. Douglas,et al.  A pulse-coded communications infrastructure for neuromorphic systems , 1999 .

[64]  Giacomo Indiveri,et al.  Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms , 2019, APL Materials.

[65]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[66]  Mohammed E. Fouda,et al.  Mask Technique for Fast and Efficient Training of Binary Resistive Crossbar Arrays , 2019, IEEE Transactions on Nanotechnology.

[67]  Gert Cauwenberghs,et al.  Deep Supervised Learning Using Local Errors , 2017, Front. Neurosci..

[68]  S. Ambrogio,et al.  Emerging neuromorphic devices , 2019, Nanotechnology.

[69]  Somnath Paul,et al.  Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines , 2016, Front. Neurosci..

[70]  Dmitri B. Strukov,et al.  Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits , 2017, Nature Communications.

[71]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[72]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[73]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[74]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[75]  Giacomo Indiveri,et al.  A differential memristive synapse circuit for on-line learning in neuromorphic computing systems , 2017, ArXiv.

[76]  Hesham Mostafa,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[77]  Garrick Orchard,et al.  SLAYER: Spike Layer Error Reassignment in Time , 2018, NeurIPS.