Graphene memristive synapses for high precision neuromorphic computing

Memristive crossbar architectures are evolving as powerful in-memory computing engines for artificial neural networks. However, the limited number of non-volatile conductance states offered by state-of-the-art memristors is a concern for their hardware implementation since trained weights must be rounded to the nearest conductance states, introducing error which can significantly limit inference accuracy. Moreover, the incapability of precise weight updates can lead to convergence problems and slowdown of on-chip training. In this article, we circumvent these challenges by introducing graphene-based multi-level (>16) and non-volatile memristive synapses with arbitrarily programmable conductance states. We also show desirable retention and programming endurance. Finally, we demonstrate that graphene memristors enable weight assignment based on k-means clustering, which offers greater computing accuracy when compared with uniform weight quantization for vector matrix multiplication, an essential component for any artificial neural network.

[1]  Frederick T. Chen,et al.  Excellent resistive memory characteristics and switching mechanism using a Ti nanolayer at the Cu/TaOx interface , 2012, Nanoscale Research Letters.

[2]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[3]  Dmitri B. Strukov,et al.  Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits , 2017, Nature Communications.

[4]  Dapeng Yu,et al.  Hysteresis reversion in graphene field-effect transistors. , 2010, The Journal of chemical physics.

[5]  G. Lo,et al.  Analytic model of endurance degradation and its practical applications for operation scheme optimization in metal oxide based RRAM , 2013, 2013 IEEE International Electron Devices Meeting.

[6]  Ophir Vermesh,et al.  Hysteresis caused by water molecules in carbon nanotube field-effect transistors , 2003 .

[7]  James M Tour,et al.  Electronic two-terminal bistable graphitic memories. , 2008, Nature materials.

[8]  Ming Zhang,et al.  Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs , 2019, Neurocomputing.

[9]  E. Vianello,et al.  HfO2-Based OxRAM Devices as Synapses for Convolutional Neural Networks , 2015, IEEE Transactions on Electron Devices.

[10]  Yihong Wu,et al.  Hysteresis of electronic transport in graphene transistors. , 2010, ACS nano.

[11]  Hui Li,et al.  Interfacial structure and wetting properties of water droplets on graphene under a static electric field. , 2015, Physical chemistry chemical physics : PCCP.

[13]  Yang Liu,et al.  Two-Step Quantization for Low-bit Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Alberto Testolin,et al.  Deep learning systems as complex networks , 2018, Journal of Complex Networks.

[15]  Jehoshua Bruck,et al.  Graphene-based atomic-scale switches. , 2008, Nano letters.

[16]  E. Kandel,et al.  Is Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory , 2000, Nature Reviews Neuroscience.

[17]  Jack J. Dongarra,et al.  Exascale computing and big data , 2015, Commun. ACM.

[18]  A. Geim,et al.  Nonvolatile Switching in Graphene Field-Effect Devices , 2008, IEEE Electron Device Letters.

[19]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[20]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[21]  Yuan Taur,et al.  Device scaling limits of Si MOSFETs and their application dependencies , 2001, Proc. IEEE.

[22]  G. Lynch,et al.  Heterosynaptic depression: a postsynaptic correlate of long-term potentiation , 1977, Nature.

[23]  Martin Weis,et al.  Tuning of Threshold Voltage in Organic Field-Effect Transistor by Dipole Monolayer , 2009 .

[24]  D. Ielmini,et al.  Pulsed cycling operation and endurance failure of metal-oxide resistive (RRAM) , 2014, 2014 IEEE International Electron Devices Meeting.

[25]  Water Trapping at the Graphene/Al2O3Interface , 2013 .

[26]  Evidence of water-related discrete trap state formation in pentacene single-crystal field-effect transistors , 2005, cond-mat/0508607.

[27]  Lukás Sekanina,et al.  Automated Circuit Approximation Method Driven by Data Distribution , 2019, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[28]  Ali Khiat,et al.  Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses , 2016, Nature Communications.

[29]  Spyros Stathopoulos,et al.  Multibit memory operation of metal-oxide bi-layer memristors , 2017, Scientific Reports.

[30]  Catherine E. Graves,et al.  Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine , 2018, Advanced materials.

[31]  Philip S. Chodrow,et al.  Configuration Models of Random Hypergraphs and their Applications , 2019, J. Complex Networks.

[32]  Zhijian Liu,et al.  HAQ: Hardware-Aware Automated Quantization With Mixed Precision , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  B. Batlogg,et al.  Threshold voltage shift in organic field effect transistors by dipole monolayers on the gate insulator , 2004 .

[34]  Carbon-based resistive memory , 2008, 2008 IEEE International Electron Devices Meeting.

[35]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[36]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Avik W. Ghosh,et al.  Controlling transistor threshold voltages using molecular dipoles , 2008, 0807.3378.

[38]  V. Panchal,et al.  Water on graphene: review of recent progress , 2018, 1804.09518.

[39]  Luis Hernández-Martínez,et al.  Understanding the Resistive Switching Phenomena of Stacked Al/Al2O3/Al Thin Films from the Dynamics of Conductive Filaments , 2017, Complex..

[40]  Weiguang Xie,et al.  Graphene Based Non‐Volatile Memory Devices , 2014, Advanced materials.

[41]  J. Yang,et al.  High switching endurance in TaOx memristive devices , 2010 .

[42]  N. Koratkar,et al.  Tunable bandgap in graphene by the controlled adsorption of water molecules. , 2010, Small.

[43]  Tim O. Wehling,et al.  First-principles studies of water adsorption on graphene: The role of the substrate , 2008, 0809.2894.

[44]  Heiko Schwarz,et al.  DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks , 2019, IEEE Journal of Selected Topics in Signal Processing.

[45]  On the Mechanism of Hydrophilicity of Graphene. , 2016, Nano letters.

[46]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[47]  T. Ren,et al.  Graphene resistive random memory — the promising memory device in next generation* , 2017 .

[48]  Sung-Mo Kang,et al.  Memristors: Devices, Models, and Applications [Scanning the Issue] , 2012, Proc. IEEE.

[49]  R. Waser,et al.  Memristors: Devices, Models, and Applications , 2012 .

[50]  E. Vianello,et al.  On the Origin of Low-Resistance State Retention Failure in HfO2-Based RRAM and Impact of Doping/Alloying , 2015, IEEE Transactions on Electron Devices.

[51]  H. Withagen,et al.  Reducing the effect of quantization by weight scaling , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[52]  F. Peeters,et al.  Electric-field-induced structural changes in water confined between two graphene layers , 2016, 1601.06073.

[53]  Jung Min Lee,et al.  Synaptic Barristor Based on Phase‐Engineered 2D Heterostructures , 2018, Advanced materials.

[54]  Andrew Pannone,et al.  Gaussian synapses for probabilistic neural networks , 2019, Nature Communications.

[55]  S. Phinn,et al.  Australian vegetated coastal ecosystems as global hotspots for climate change mitigation , 2019, Nature Communications.

[56]  Shimeng Yu,et al.  Resistive Memory-Based Analog Synapse: The Pursuit for Linear and Symmetric Weight Update , 2018, IEEE Nanotechnology Magazine.

[57]  Congli He,et al.  Artificial synapse based on van der Waals heterostructures with tunable synaptic functions for neuromorphic computing. , 2020, ACS applied materials & interfaces.

[58]  Sen Song,et al.  Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges , 2019, Advanced materials.