Towards the Development of Analog Neuromorphic Chip Prototype with 2.4M Integrated Memristors

We have designed and fabricated a neuromorphic accelerator chip which features 180-nm CMOS circuitry and 2.4-million 0.04-μm2-footprint Al2O3/TiO2—x memristors. Memristors were passively integrated on top of the CMOS wafer into 48×48 crossbar circuits, with a total of 1032 crossbars on a chip. Each memristive crossbar is accessed via on-chip CMOS interface circuits which are controlled by a custom FPGA board. The whole system is designed to implement a variety of deep neural networks by performing vector-by-matrix multiplication, a core operation of any neural network, with a memristive crossbar circuit in analog domain. In this this paper we provide details of the on-chip CMOS circuits and the FPGA board, both of which have already been successfully tested. We also briefly discuss integration results and the on-going work.

[1]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[2]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[5]  D. Strukov,et al.  Resistive switching phenomena in thin films: Materials, devices, and applications , 2012 .

[6]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[7]  Song Han,et al.  EIE: Efficient Inference Engine on Compressed Deep Neural Network , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[8]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[9]  Farnood Merrikh-Bayat,et al.  Memristor-based perceptron classifier: Increasing complexity and coping with imperfect hardware , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[10]  Vivienne Sze,et al.  14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks , 2016, ISSCC.

[11]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[12]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[13]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[14]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..

[15]  Dmitri B. Strukov,et al.  Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits , 2017, Nature Communications.

[16]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[17]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[18]  Yukihiro Kaneko,et al.  Ferroelectric Artificial Synapses for Recognition of a Multishaded Image , 2014, IEEE Transactions on Electron Devices.

[19]  Jiaming Zhang,et al.  Analogue signal and image processing with large memristor crossbars , 2017, Nature Electronics.

[20]  Farnood Merrikh-Bayat,et al.  3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications , 2017, IEEE Transactions on Electron Devices.

[21]  Farnood Merrikh-Bayat,et al.  High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Jia Wang,et al.  DaDianNao: A Machine-Learning Supercomputer , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.

[23]  Wei D. Lu,et al.  Sparse coding with memristor networks. , 2017, Nature nanotechnology.

[24]  V. Cros,et al.  Spin-torque building blocks. , 2014, Nature Materials.

[25]  Farnood Merrikh-Bayat,et al.  Efficient training algorithms for neural networks based on memristive crossbar circuits , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[26]  Rahul Sarpeshkar,et al.  Analog Versus Digital: Extrapolating from Electronics to Neurobiology , 1998, Neural Computation.

[27]  Konstantin K. Likharev,et al.  CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks , 2011 .

[28]  S. Chakrabartty,et al.  Sub-Microwatt Analog VLSI Trainable Pattern Classifier , 2007, IEEE Journal of Solid-State Circuits.

[29]  Byoung Hun Lee,et al.  Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron , 2015, IEEE Transactions on Industrial Electronics.

[30]  T. Hasegawa,et al.  Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. , 2011, Nature materials.