High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays

Potential advantages of analog- and mixed-signal nanoelectronic circuits, based on floating-gate devices with adjustable conductance, for neuromorphic computing had been realized long time ago. However, practical realizations of this approach suffered from using rudimentary floating-gate cells of relatively large area. Here, we report a prototype <inline-formula> <tex-math notation="LaTeX">$28\times28$ </tex-math></inline-formula> binary-input, ten-output, three-layer neuromorphic network based on arrays of highly optimized embedded nonvolatile floating-gate cells, redesigned from a commercial 180-nm nor flash memory. All active blocks of the circuit, including 101 780 floating-gate cells, have a total area below 1 mm<sup>2</sup>. The network has shown a 94.7% classification fidelity on the common Modified National Institute of Standards and Technology benchmark, close to the 96.2% obtained in simulation. The classification of one pattern takes a sub-1-<inline-formula> <tex-math notation="LaTeX">$\mu \text{s}$ </tex-math></inline-formula> time and a sub-20-nJ energy—both numbers much better than in the best reported digital implementations of the same task. Estimates show that a straightforward optimization of the hardware and its transfer to the already available 55-nm technology may increase this advantage to more than <inline-formula> <tex-math notation="LaTeX">$10^{2}\times $ </tex-math></inline-formula> in speed and <inline-formula> <tex-math notation="LaTeX">$10^{4}\times $ </tex-math></inline-formula> in energy efficiency.

[1]  Konstantin K. Likharev,et al.  CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks , 2011 .

[2]  Carver A. Mead,et al.  A single-transistor silicon synapse , 1996 .

[3]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[4]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[5]  L. Ceze,et al.  Nanoelectronic neurocomputing: Status and prospects , 2016, 2016 74th Annual Device Research Conference (DRC).

[6]  M. Kozicki,et al.  Electrochemical metallization cells—blending nanoionics into nanoelectronics? , 2012 .

[7]  R. Jordan,et al.  NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[8]  Farnood Merrikh-Bayat,et al.  3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications , 2017, IEEE Transactions on Electron Devices.

[9]  E. Sackinger.  Measurement of Finite-Precision Effects in Handwriting- and Speech-Recognition Algorithms , 1997 .

[10]  S. Raoux.  Phase Change Materials , 2009 .

[11]  D. Ielmini,et al.  Phase Change Materials , 2009 .

[12]  Sihwan Kim,et al.  A Programmable and Configurable Mixed-Mode FPAA SoC , 2016, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[13]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[14]  B. Murmann,et al.  Passive charge redistribution digital-to-analogue multiplier , 2015 .

[15]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[16]  Paul E. Hasler,et al.  A Highly Dense, Low Power, Programmable Analog Vector-Matrix Multiplier: The FPAA Implementation , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[17]  Steven R. Young,et al.  A 1 TOPS/W Analog Deep Machine-Learning Engine With Floating-Gate Storage in 0.13 µm CMOS , 2014, IEEE Journal of Solid-State Circuits.

[18]  Dharmendra S. Modha,et al.  Backpropagation for Energy-Efficient Neuromorphic Computing , 2015, NIPS.

[19]  H. Kim,et al.  RRAM-based synapse for neuromorphic system with pattern recognition function , 2012, 2012 International Electron Devices Meeting.

[20]  Yukihiro Kaneko,et al.  Supervised Learning Using Spike-Timing-Dependent Plasticity of Memristive Synapses , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[21]  D. Strukov,et al.  Resistive switching phenomena in thin films: Materials, devices, and applications , 2012 .

[22]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..

[23]  Xinjie Guo,et al.  Redesigning commercial floating-gate memory for analog computing applications , 2014, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  S. Simon Wong,et al.  24.2 A 2.5GHz 7.7TOPS/W switched-capacitor matrix multiplier with co-designed local memory in 40nm , 2016, 2016 IEEE International Solid-State Circuits Conference (ISSCC).

[26]  F. Merrikh Bayat,et al.  Model-based high-precision tuning of NOR flash memory cells for analog computing applications , 2016, 2016 74th Annual Device Research Conference (DRC).

[27]  Thomas Mikolajick,et al.  Metal oxide memories based on thermochemical and valence change mechanisms , 2012 .

[28]  Vincent Garcia,et al.  Ferroelectric and multiferroic tunnel junctions , 2012 .

[29]  Farnood Merrikh-Bayat,et al.  Temperature-insensitive analog vector-by-matrix multiplier based on 55 nm NOR flash memory cells , 2016, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[30]  Farnood Merrikh-Bayat,et al.  Advancing Memristive Analog Neuromorphic Networks: Increasing Complexity, and Coping with Imperfect Hardware Components , 2016, ArXiv.

[31]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[32]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .