Fully parallel write/read in resistive synaptic array for accelerating on-chip learning

A neuro-inspired computing paradigm beyond the von Neumann architecture is emerging and it generally takes advantage of massive parallelism and is aimed at complex tasks that involve intelligence and learning. The cross-point array architecture with synaptic devices has been proposed for on-chip implementation of the weighted sum and weight update in the learning algorithms. In this work, forming-free, silicon-process-compatible Ta/TaOx/TiO2/Ti synaptic devices are fabricated, in which >200 levels of conductance states could be continuously tuned by identical programming pulses. In order to demonstrate the advantages of parallelism of the cross-point array architecture, a novel fully parallel write scheme is designed and experimentally demonstrated in a small-scale crossbar array to accelerate the weight update in the training process, at a speed that is independent of the array size. Compared to the conventional row-by-row write scheme, it achieves >30× speed-up and >30× improvement in energy efficiency as projected in a large-scale array. If realistic synaptic device characteristics such as device variations are taken into an array-level simulation, the proposed array architecture is able to achieve ∼95% recognition accuracy of MNIST handwritten digits, which is close to the accuracy achieved by software using the ideal sparse coding algorithm.

[1]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[2]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[3]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[4]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[5]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[6]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[7]  H. Hwang,et al.  Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device , 2011, Nanotechnology.

[8]  Olivier Bichler,et al.  Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction , 2011, 2011 International Electron Devices Meeting.

[9]  T. Hasegawa,et al.  Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. , 2011, Nature materials.

[10]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[11]  Chi-Sang Poon,et al.  Neuromorphic Silicon Neurons and Large-Scale Neural Networks: Challenges and Opportunities , 2011, Front. Neurosci..

[12]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[13]  B. DeSalvo,et al.  CBRAM devices as binary synapses for low-power stochastic neuromorphic systems: Auditory (Cochlea) and visual (Retina) cognitive processing applications , 2012, 2012 International Electron Devices Meeting.

[14]  H. Kim,et al.  RRAM-based synapse for neuromorphic system with pattern recognition function , 2012, 2012 International Electron Devices Meeting.

[15]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[16]  Narayan Srinivasa,et al.  A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. , 2012, Nano letters.

[17]  Ligang Gao,et al.  High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm , 2011, Nanotechnology.

[18]  Zhidong Teng,et al.  Exponential synchronization for reaction-diffusion networks with mixed delays in terms of p-norm via intermittent driving , 2012, Neural Networks.

[19]  Yiran Chen,et al.  Memristor crossbar based hardware realization of BSB recall function , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[20]  Masakazu Aono,et al.  On-demand nanodevice with electrical and neuromorphic multifunction realized by local ion migration. , 2012, ACS nano.

[21]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[22]  Shimeng Yu,et al.  A Low Energy Oxide‐Based Electronic Synaptic Device for Neuromorphic Visual Systems with Tolerance to Device Variation , 2013, Advanced materials.

[23]  An Chen.  Comprehensive methodology for the design and assessment of crossbar memory array with nonlinear and asymmetric selector devices , 2013, 2013 IEEE International Electron Devices Meeting.

[24]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[25]  Doo Seok Jeong,et al.  Towards artificial neurons and synapses: a materials point of view , 2013 .

[26]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[27]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..

[28]  Chung Lam,et al.  Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array , 2014, Front. Neurosci..

[29]  Chung-Wei Hsu,et al.  Homogeneous barrier modulation of TaOx/TiO2 bilayers for ultra-high endurance three-dimensional storage-class memory , 2014, Nanotechnology.

[30]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[31]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[32]  Tuo-Hung Hou,et al.  3D synaptic architecture with ultralow sub-10 fJ energy per spike for neuromorphic computation , 2014, 2014 IEEE International Electron Devices Meeting.

[33]  Shimeng Yu,et al.  Parallel Programming of Resistive Cross-point Array for Synaptic Plasticity , 2014, BICA.

[34]  E. Vianello,et al.  Variability-tolerant Convolutional Neural Network for Pattern Recognition applications based on OxRAM synapses , 2014, 2014 IEEE International Electron Devices Meeting.

[35]  Shimeng Yu,et al.  Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[36]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[37]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[38]  Shimeng Yu,et al.  Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[39]  Ligang Gao,et al.  Programming protocol optimization for analog weight tuning in resistive memories , 2015, 2015 73rd Annual Device Research Conference (DRC).

[40]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[41]  Damien Querlioz,et al.  Bioinspired Programming of Memory Devices for Implementing an Inference Engine , 2015, Proceedings of the IEEE.

[42]  Jintao Yu,et al.  Memristive devices for computing: Beyond CMOS and beyond von Neumann , 2017, 2017 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC).