Experimental Demonstration of Firing Rate Neural Networks Based on Metal-Oxide Memristive Crossbars

Limitations of currently dominating von Neumann architectures have pushed the research toward brain-inspired solutions like neural networks to reach new levels of computing efficiency. While these highly parallelized architectures have achieved outstanding performances at software level, their hardware implementation is still a challenging problem due to the large amount of memory and arithmetic units required to store synaptic weights and implement synaptic transmission functions. One possible solution to tackle this challenging problem is to build neural networks with analog circuits and implement the functionality of synapses with the same nonvolatile memory unit that stores synaptic weights (a so-called in-memory computing). Scalability, analog behavior (multilevel programmability), speed, and power consumption are fundamental performance requirements for such memory arrays, given the huge number of synapses in state-of-the-art neural networks. Among different choices, Resistive Random Access Memories (RRAMs) are considered as one of the main candidates due to their excellent scalability and the possibility of their integration with fully mature CMOS technology. Here in this chapter, we will show the basic principles of such devices, alongside their performance as single devices and also when integrated in crossbars, all induced from experiments. We will also present the results of the first experimental realizations of single- and multi-layer neural networks with synapses implemented through crossbar arrays with detailed explanation on their in situ training. Finally, we conclude the chapter by presenting our recent progress on the 3D integration of resistive switching devices on top of CMOS-based circuitries and also by discussing the limitations of current implementations and future challenges.

[1]  Konstantin K. Likharev,et al.  Hybrid CMOS/Nanoelectronic Circuits: Opportunities and Challenges , 2008 .

[2]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[3]  M. Prezioso,et al.  A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit , 2017, Scientific reports.

[4]  Mehdi Asnaashari,et al.  3D ReRAM with Field Assisted Super-Linear Threshold (FASTTM) Selector technology for super-dense, low power, low latency data storage systems , 2015, The 20th Asia and South Pacific Design Automation Conference.

[5]  Amirali Ghofrani,et al.  Architecting energy efficient crossbar-based memristive random-access memories , 2015, Proceedings of the 2015 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH´15).

[6]  H-S Philip Wong,et al.  Multi-level control of conductive nano-filament evolution in HfO2 ReRAM by pulse-train operations. , 2014, Nanoscale.

[7]  H. Wong,et al.  Forming-free nitrogen-doped AlOX RRAM with sub-μA programming current , 2011, 2011 Symposium on VLSI Technology - Digest of Technical Papers.

[8]  Luke Theogarajan,et al.  A configurable CMOS memory platform for 3D-integrated memristors , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[9]  Farnood Merrikh-Bayat,et al.  Advancing Memristive Analog Neuromorphic Networks: Increasing Complexity, and Coping with Imperfect Hardware Components , 2016, ArXiv.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  M. Prezioso,et al.  Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2−x/Pt Memristors , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Hyunsang Hwang,et al.  Vertically Stacked ReRAM Composed of a Bidirectional Selector and CB-RAM for Cross-Point Array Applications , 2013, IEEE Electron Device Letters.

[14]  Farnood Merrikh-Bayat,et al.  Memristor-based perceptron classifier: Increasing complexity and coping with imperfect hardware , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[15]  Byung Joon Choi,et al.  Engineering nonlinearity into memristors for passive crossbar applications , 2012 .

[16]  Masahide Matsumoto,et al.  A 130.7-$\hbox{mm}^{2}$ 2-Layer 32-Gb ReRAM Memory Device in 24-nm Technology , 2014, IEEE Journal of Solid-State Circuits.

[17]  J. Yang,et al.  Switching dynamics in titanium dioxide memristive devices , 2009 .

[18]  Eric B. Baum,et al.  On the capabilities of multilayer perceptrons , 1988, J. Complex..

[19]  Ligang Gao,et al.  High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm , 2011, Nanotechnology.

[20]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[21]  Konstantin K. Likharev,et al.  CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks , 2011 .

[22]  Khaled N. Salama,et al.  Memristor-based memory: The sneak paths problem and solutions , 2013, Microelectron. J..

[23]  Heng-Yuan Lee,et al.  A 4Mb embedded SLC resistive-RAM macro with 7.2ns read-write random-access time and 160ns MLC-access capability , 2011, 2011 IEEE International Solid-State Circuits Conference.

[24]  Wolfram Schiffmann,et al.  Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons , 1994 .

[25]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[26]  Carver A. Mead,et al.  A single-transistor silicon synapse , 1996 .

[27]  Ru Huang,et al.  A New Dynamic Selector Based on the Bipolar RRAM for the Crossbar Array Application , 2012, IEEE Transactions on Electron Devices.

[28]  Sihwan Kim,et al.  A Programmable and Configurable Mixed-Mode FPAA SoC , 2016, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[29]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..

[30]  Farnood Merrikh-Bayat,et al.  Efficient training algorithms for neural networks based on memristive crossbar circuits , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[31]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[32]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[33]  Warren Robinett,et al.  Memristor-CMOS hybrid integrated circuits for reconfigurable logic. , 2009, Nano letters.

[34]  Qiangfei Xia,et al.  3D integration of planar crossbar memristive devices with CMOS substrate , 2014, Nanotechnology.

[35]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.