A System-Level Simulator for RRAM-Based Neuromorphic Computing Chips

Advances in non-volatile resistive switching random access memory (RRAM) have made it a promising memory technology with potential applications in low-power and embedded in-memory computing devices owing to a number of advantages such as low-energy consumption, low area cost and good scaling. There have been proposals to employ RRAM in architecting chips for neuromorphic computing and artificial neural networks where matrix-vector multiplication can be computed in the analog domain in a single timestep. However, it is challenging to employ RRAM devices in neuromorphic chips owing to the non-ideal behavior of RRAM. In this article, we propose a cycle-accurate and scalable system-level simulator that can be used to study the effects of using RRAM devices in neuromorphic computing chips. The simulator models a spatial neuromorphic chip architecture containing many neural cores with RRAM crossbars connected via a Network-on-Chip (NoC). We focus on system-level simulation and demonstrate the effectiveness of our simulator in understanding how non-linear RRAM effects such as stuck-at-faults (SAFs), write variability, and random telegraph noise (RTN) can impact an application’s behavior. By using our simulator, we show that RTN and write variability can have adverse effects on an application. Nevertheless, we show that these effects can be mitigated through proper design choices and the implementation of a write-verify scheme.

[1]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[2]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[3]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[4]  Andrew B. Kahng,et al.  ORION 2.0: A fast and accurate NoC power and area model for early-stage design space exploration , 2009, 2009 Design, Automation & Test in Europe Conference & Exhibition.

[5]  M. Prezioso,et al.  A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit , 2017, Scientific reports.

[6]  Narayan Srinivasa,et al.  HRLSim: A High Performance Spiking Neural Network Simulator for GPGPU Clusters , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[8]  Yu Wang,et al.  MErging the Interface: Power, area and accuracy co-optimization for RRAM crossbar-based mixed-signal computing system , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[9]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[10]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[11]  John Paul Strachan,et al.  Dot-product engine as computing memory to accelerate machine learning algorithms , 2016, 2016 17th International Symposium on Quality Electronic Design (ISQED).

[12]  Romain Brette,et al.  The Brian Simulator , 2009, Front. Neurosci..

[13]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[14]  Wei Lu,et al.  A Low-Power Variation-Aware Adaptive Write Scheme for Access-Transistor-Free Memristive Memory , 2015, ACM J. Emerg. Technol. Comput. Syst..

[15]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[16]  He Liu,et al.  Optimized Mapping Spiking Neural Networks onto Network-on-Chip , 2016, ICA3PP.

[17]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[18]  J. Kim,et al.  Neuromorphic speech systems using advanced ReRAM-based synapse , 2013, 2013 IEEE International Electron Devices Meeting.

[19]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[20]  Rajeev Balasubramonian,et al.  INXS: Bridging the throughput and energy gap for spiking neural networks , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[21]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[22]  Alessandro Calderoni,et al.  Statistical Fluctuations in HfOx Resistive-Switching Memory: Part I - Set/Reset Variability , 2014, IEEE Transactions on Electron Devices.

[23]  Giacomo Indiveri,et al.  Integration of nanoscale memristor synapses in neuromorphic computing architectures , 2013, Nanotechnology.

[24]  Kwabena Boahen,et al.  Optic nerve signals in a neuromorphic chip I: Outer and inner retina models , 2004, IEEE Transactions on Biomedical Engineering.

[25]  Wenguang Chen,et al.  Bridge the Gap between Neural Networks and Neuromorphic Hardware with a Neural Network Compiler , 2017, ASPLOS.

[26]  Leon Reznik,et al.  GPU-based simulation of spiking neural networks with real-time performance & high accuracy , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[27]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[28]  Don Monroe,et al.  Neuromorphic computing gets ready for the (really) big time , 2014, CACM.

[29]  Jim D. Garside,et al.  SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation , 2013, IEEE Journal of Solid-State Circuits.

[30]  O. Kwon,et al.  Process Optimization of Integrated SiCr Thin-Film Resistor for High-Performance Analog Circuits , 2014, IEEE Transactions on Electron Devices.

[31]  Catherine Graves,et al.  Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication , 2016, 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[32]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[33]  Myron Flickner,et al.  Compass: A scalable simulator for an architecture for cognitive computing , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[34]  James C. Hoe,et al.  CONNECT: re-examining conventional wisdom for designing nocs in the context of FPGAs , 2012, FPGA '12.

[35]  Rong Luo,et al.  Spiking neural network with RRAM: Can we use it for real-world application? , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[36]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[37]  V. G. Karpov,et al.  Log-Normal Statistics in Filamentary RRAM Devices and Related Systems , 2017, IEEE Electron Device Letters.

[38]  Kaushik Roy,et al.  RESPARC: A reconfigurable and energy-efficient architecture with Memristive Crossbars for deep Spiking Neural Networks , 2017, 2017 54th ACM/EDAC/IEEE Design Automation Conference (DAC).

[39]  H.-S. Philip Wong,et al.  Face classification using electronic synapses , 2017, Nature Communications.

[40]  Murray Shanahan,et al.  Accelerated simulation of spiking neural networks using GPUs , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[41]  J.V. Arthur,et al.  Recurrently connected silicon neurons with active dendrites for one-shot learning , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[42]  Chaitali Chakrabarti,et al.  A System Level Energy Model and Energy-Quality Evaluation for Integrated Transceiver Front-Ends , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[43]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[44]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[45]  Yu Wang,et al.  MNSIM: Simulation Platform for Memristor-Based Neuromorphic Computing System , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[46]  Hiroshi Nakamura,et al.  7.2 4Mb STT-MRAM-based cache with memory-access-aware power optimization and write-verify-write / read-modify-write scheme , 2016, 2016 IEEE International Solid-State Circuits Conference (ISSCC).

[47]  S. Ambrogio,et al.  Statistical Fluctuations in HfOx Resistive-Switching Memory: Part II—Random Telegraph Noise , 2014, IEEE Transactions on Electron Devices.