Energy consumption analysis for various memristive networks under different learning strategies

Abstract Recently, various memristive systems emerge to emulate the efficient computing paradigm of the brain cortex; whereas, how to make them energy efficient still remains unclear, especially from an overall perspective. Here, a systematical and bottom-up energy consumption analysis is demonstrated, including the memristor device level and the network learning level. We propose an energy estimating methodology when modulating the memristive synapses, which is simulated in three typical neural networks with different synaptic structures and learning strategies for both offline and online learning. These results provide an in-depth insight to create energy efficient brain-inspired neuromorphic devices in the future.

[1]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[2]  Qing Wan,et al.  Artificial synapse network on inorganic proton conductor for neuromorphic systems. , 2014, Nature communications.

[3]  Dong Wang,et al.  Complex Learning in Bio-plausible Memristive Networks , 2015, Scientific Reports.

[4]  H.-S. Philip Wong,et al.  Energy efficient programming of nanoelectronic synaptic devices for large-scale implementation of associative and temporal sequence learning , 2011, 2011 International Electron Devices Meeting.

[5]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[6]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[7]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[8]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[9]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[10]  Kaushik Roy,et al.  Cognitive computing with spin-based neural networks , 2012, DAC Design Automation Conference 2012.

[11]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[12]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[13]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[14]  Eric Pop,et al.  Low-Power Switching of Phase-Change Materials with Carbon Nanotube Electrodes , 2011, Science.

[15]  R Stanley Williams,et al.  Sub-100 fJ and sub-nanosecond thermally driven threshold switching in niobium oxide crosspoint nanodevices , 2012, Nanotechnology.

[16]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[17]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[18]  Shimeng Yu,et al.  Low-Energy Robust Neuromorphic Computation Using Synaptic Devices , 2012, IEEE Transactions on Electron Devices.

[19]  Narayan Srinivasa,et al.  A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. , 2012, Nano letters.

[20]  Fabien Alibart,et al.  A Memristive Nanoparticle/Organic Hybrid Synapstor for Neuroinspired Computing , 2011, ArXiv.

[21]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[22]  Jim D. Garside,et al.  SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation , 2013, IEEE Journal of Solid-State Circuits.

[23]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[24]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[25]  Leon O. Chua,et al.  Memristor Bridge Synapse-Based Neural Network and Its Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Yoon-Ha Jeong,et al.  ReRAM-based synaptic device for neuromorphic computing , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[27]  Kwabena Boahen,et al.  Learning in Silicon: Timing is Everything , 2005, NIPS.

[28]  T. Hasegawa,et al.  Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. , 2011, Nature materials.

[29]  Paul E. Hasler,et al.  Floating Gate Synapses With Spike-Time-Dependent Plasticity , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[30]  Mohammed Ismail,et al.  Analog VLSI Implementation of Neural Systems , 2011, The Kluwer International Series in Engineering and Computer Science.

[31]  Gregory S. Snider,et al.  Spike-timing-dependent learning in memristive nanodevices , 2008, 2008 IEEE International Symposium on Nanoscale Architectures.

[32]  J. Yang,et al.  Memristive switching mechanism for metal/oxide/metal nanodevices. , 2008, Nature nanotechnology.

[33]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[34]  Geoffrey W. Burr,et al.  Nanoscale electronic synapses using phase change devices , 2013, JETC.