Using Floating-Gate Memory to Train Ideal Accuracy Neural Networks

Floating-gate silicon-oxygen-nitrogen-oxygen-silicon (SONOS) transistors can be used to train neural networks to ideal accuracies that match those of floating-point digital weights on the MNIST handwritten digit data set when using multiple devices to represent a weight or within 1% of ideal accuracy when using a single device. This is enabled by operating devices in the subthreshold regime, where they exhibit symmetric write nonlinearities. A neural training accelerator core based on SONOS with a single device per weight would increase energy efficiency by <inline-formula> <tex-math notation="LaTeX">$120\times $ </tex-math></inline-formula>, operate <inline-formula> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> faster, and require <inline-formula> <tex-math notation="LaTeX">$5\times $ </tex-math></inline-formula> lower area than an optimized SRAM-based ASIC.

[1]  Jennifer Hasler,et al.  Vector-Matrix Multiply and Winner-Take-All as an Analog Classifier , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[2]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[3]  Sapan Agarwal,et al.  Li‐Ion Synaptic Transistor for Low Power Analog Computing , 2017, Advanced materials.

[4]  Ojas Parekh,et al.  Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding , 2016, Front. Neurosci..

[5]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[6]  Eric Beyne,et al.  Ultra-Fine Pitch 3D Integration Using Face-to-Face Hybrid Wafer Bonding Combined with a Via-Middle Through-Silicon-Via Process , 2016, 2016 IEEE 66th Electronic Components and Technology Conference (ECTC).

[7]  F. Merrikh Bayat,et al.  Fast, energy-efficient, robust, and reproducible mixed-signal neuromorphic classifier based on embedded NOR flash memory technology , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[8]  Dong-Hyun Kim,et al.  High-speed and logic-compatible split-gate embedded flash on 28-nm low-power HKMG logic process , 2017, 2017 Symposium on VLSI Technology.

[9]  Steven J. Plimpton,et al.  Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator , 2017, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[10]  Shimeng Yu,et al.  Three-Dimensional nand Flash for Vector–Matrix Multiplication , 2019, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[11]  M. Marinella,et al.  A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. , 2017, Nature materials.

[12]  Steven J. Plimpton,et al.  Achieving ideal accuracies in analog neuromorphic computing using periodic carry , 2017, 2017 Symposium on VLSI Technology.

[13]  Steven J. Plimpton,et al.  Resistive memory device requirements for a neural algorithm accelerator , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[14]  Tetsuo Endoh,et al.  Design impacts on NAND Flash memory core circuits with vertical MOSFETs , 2010, 2010 IEEE International Memory Workshop.

[15]  Yusuf Leblebici,et al.  Improved Deep Neural Network Hardware-Accelerators Based on Non-Volatile-Memory: The Local Gains Technique , 2017, 2017 IEEE International Conference on Rebooting Computing (ICRC).

[16]  Hideto Hidaka,et al.  A 28 nm Embedded Split-Gate MONOS (SG-MONOS) Flash Macro for Automotive Achieving 6.4 GB/s Read Throughput by 200 MHz No-Wait Read Operation and 2.0 MB/s Write Throughput at Tj of 170$^{\circ}$ C , 2016, IEEE Journal of Solid-State Circuits.

[17]  Jonathan A. Cox,et al.  A Signal Processing Approach for Cyber Data Classification with Deep Neural Networks , 2015, Complex Adaptive Systems.

[18]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.