Resistive memory device requirements for a neural algorithm accelerator

Resistive memories enable dramatic energy reductions for neural algorithms. We propose a general purpose neural architecture that can accelerate many different algorithms and determine the device properties that will be needed to run backpropagation on the neural architecture. To maintain high accuracy, the read noise standard deviation should be less than 5% of the weight range. The write noise standard deviation should be less than 0.4% of the weight range and up to 300% of a characteristic update (for the datasets tested). Asymmetric nonlinearities in the change in conductance vs pulse cause weight decay and significantly reduce the accuracy, while moderate symmetric nonlinearities do not have an effect. In order to allow for parallel reads and writes the write current should be less than 100 nA as well.

[1]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[2]  Shimeng Yu,et al.  Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[3]  Jonathan A. Cox,et al.  A Signal Processing Approach for Cyber Data Classification with Deep Neural Networks , 2015, Complex Adaptive Systems.

[4]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[5]  Uri C. Weiser,et al.  TEAM: ThrEshold Adaptive Memristor Model , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[6]  J. Yang,et al.  Switching dynamics in titanium dioxide memristive devices , 2009 .

[7]  Shimeng Yu,et al.  On the Switching Parameter Variation of Metal-Oxide RRAM—Part I: Physical Modeling and Simulation Methodology , 2012, IEEE Transactions on Electron Devices.

[8]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[9]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[10]  L. Chua Memristor-The missing circuit element , 1971 .

[11]  Shinhyun Choi,et al.  Tuning resistive switching characteristics of tantalum oxide memristors through Si doping. , 2014, ACS nano.

[12]  Stefano Ambrogio,et al.  Noise-Induced Resistance Broadening in Resistive Switching Memory—Part II: Array Statistics , 2015, IEEE Transactions on Electron Devices.

[13]  Sarma Vrudhula,et al.  Incremental resistance programming of programmable metallization cells for use as electronic synapses , 2014 .

[14]  Narayan Srinivasa,et al.  A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. , 2012, Nano letters.

[15]  M. Marinella,et al.  Isothermal Switching and Detailed Filament Evolution in Memristive Systems , 2014, Advanced materials.

[16]  Kyeong-Sik Min,et al.  New Memristor-Based Crossbar Array Architecture with 50-% Area Reduction and 48-% Power Saving for Matrix-Vector Multiplication of Analog Neuromorphic Computing , 2014 .

[17]  Ojas Parekh,et al.  Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding , 2016, Front. Neurosci..

[18]  Yukihiro Kaneko,et al.  Back-Propagation Operation for Analog Neural Network Hardware with Synapse Components Having Hysteresis Characteristics , 2014, PloS one.

[19]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[20]  Shimeng Yu,et al.  Understanding the conduction and switching mechanism of metal oxide RRAM through low frequency noise and AC conductance measurement and analysis , 2011, 2011 International Electron Devices Meeting.

[21]  Shimeng Yu,et al.  Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[22]  D. Gilmer,et al.  Random telegraph noise (RTN) in scaled RRAM devices , 2013, 2013 IEEE International Reliability Physics Symposium (IRPS).

[23]  Chia-En Huang,et al.  Electron trapping effect on the switching behavior of contact RRAM devices through random telegraph noise analysis , 2010, 2010 International Electron Devices Meeting.

[24]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[25]  Chris Yakopcic,et al.  Exploring the design space of specialized multicore neural processors , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[26]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[27]  Siddharth Gaba,et al.  Ultralow Sub-1-nA Operating Current Resistive Memory With Intrinsic Non-Linear Characteristics , 2014, IEEE Electron Device Letters.

[28]  Yuchao Yang,et al.  Building Neuromorphic Circuits with Memristive Devices , 2013, IEEE Circuits and Systems Magazine.

[29]  L. Goux,et al.  Microscopic origin of random telegraph noise fluctuations in aggressively scaled RRAM and its impact on read disturb variability , 2013, 2013 IEEE International Reliability Physics Symposium (IRPS).

[30]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[31]  G. Ghibaudo,et al.  Current Conduction Model for Oxide-Based Resistive Random Access Memory Verified by Low-Frequency Noise Analysis , 2013, IEEE Transactions on Electron Devices.

[32]  Chris Yakopcic,et al.  Generalized Memristive Device SPICE Model and its Application in Circuit Design , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[33]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[34]  Yiran Chen,et al.  Memristor Crossbar-Based Neuromorphic Computing System: A Case Study , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Yong Zhang,et al.  A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing , 2015, ACM J. Emerg. Technol. Comput. Syst..

[36]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[37]  Tarek M. Taha,et al.  Enabling back propagation training of memristor crossbar neuromorphic processors , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[38]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[39]  Jongkil Park,et al.  Live demonstration: Hierarchical Address-Event Routing architecture for reconfigurable large scale neuromorphic systems , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[40]  Albert Chin,et al.  High performance ultra-low energy RRAM with good retention and endurance , 2010, 2010 International Electron Devices Meeting.

[41]  Stefano Ambrogio,et al.  Noise-Induced Resistance Broadening in Resistive Switching Memory—Part I: Intrinsic Cell Behavior , 2015, IEEE Transactions on Electron Devices.

[42]  G. Palm,et al.  Density of neurons and synapses in the cerebral cortex of the mouse , 1989, The Journal of comparative neurology.