Energy Scaling Advantages of Resistive Memory Crossbar Based Computation and Its Application to Sparse Coding

The exponential increase in data over the last decade presents a significant challenge to analytics efforts that seek to process and interpret such data for various applications. Neural-inspired computing approaches are being developed in order to leverage the computational properties of the analog, low-power data processing observed in biological systems. Analog resistive memory crossbars can perform a parallel read or a vector-matrix multiplication as well as a parallel write or a rank-1 update with high computational efficiency. For an N × N crossbar, these two kernels can be O(N) more energy efficient than a conventional digital memory-based architecture. If the read operation is noise limited, the energy to read a column can be independent of the crossbar size (O(1)). These two kernels form the basis of many neuromorphic algorithms such as image, text, and speech recognition. For instance, these kernels can be applied to a neural sparse coding algorithm to give an O(N) reduction in energy for the entire algorithm when run with finite precision. Sparse coding is a rich problem with a host of applications including computer vision, object tracking, and more generally unsupervised learning.

[1]  Albert Chin,et al.  High performance ultra-low energy RRAM with good retention and endurance , 2010, 2010 International Electron Devices Meeting.

[2]  Sarma Vrudhula,et al.  Incremental resistance programming of programmable metallization cells for use as electronic synapses , 2014 .

[3]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[4]  Shimeng Yu,et al.  Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[5]  Christian Enz,et al.  CMOS low-power analog circuit design , 1996, Emerging Technologies: Designing Low Power Digital Systems.

[6]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[7]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[8]  Shimeng Yu,et al.  Towards high-speed, write-disturb tolerant 3D vertical RRAM arrays , 2014, 2014 Symposium on VLSI Technology (VLSI-Technology): Digest of Technical Papers.

[9]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[10]  Sanjeev Arora,et al.  Simple, Efficient, and Neural Algorithms for Sparse Coding , 2015, COLT.

[11]  Mrigank Sharad,et al.  Energy-Efficient Non-Boolean Computing With Spin Neurons and Resistive Memory , 2014, IEEE Transactions on Nanotechnology.

[12]  Narayan Srinivasa,et al.  A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. , 2012, Nano letters.

[13]  Shimeng Yu,et al.  Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[14]  Martin Fürer,et al.  Faster integer multiplication , 2007, STOC '07.

[15]  Yuchao Yang,et al.  Building Neuromorphic Circuits with Memristive Devices , 2013, IEEE Circuits and Systems Magazine.

[16]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[17]  An Chen,et al.  A Comprehensive Crossbar Array Model With Solutions for Line Resistance and Nonlinear Device Characteristics , 2013, IEEE Transactions on Electron Devices.

[18]  Hsin-Shu Chen,et al.  11.2 A 0.85fJ/conversion-step 10b 200kS/s subranging SAR ADC in 40nm CMOS , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[19]  W. Lu,et al.  High-density Crossbar Arrays Based on a Si Memristive System , 2008 .

[20]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[21]  Boris Murmann,et al.  Digitally Assisted Pipeline ADCs: Theory and Implementation , 2004 .

[22]  Yong Zhang,et al.  A Reconfigurable Digital Neuromorphic Processor with Memristive Synaptic Crossbar for Cognitive Computing , 2015, ACM J. Emerg. Technol. Comput. Syst..

[23]  David A. B. Miller,et al.  Device Requirements for Optical Interconnects to Silicon Chips , 2009, Proceedings of the IEEE.

[24]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[25]  Narayan Srinivasa,et al.  A scalable neural chip with synaptic electronics using CMOS integrated memristors , 2013, Nanotechnology.

[26]  Tarek M. Taha,et al.  Enabling back propagation training of memristor crossbar neuromorphic processors , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[27]  G. Palm,et al.  Density of neurons and synapses in the cerebral cortex of the mouse , 1989, The Journal of comparative neurology.

[28]  Tao Zhang,et al.  Overcoming the challenges of crossbar resistive memory architectures , 2015, 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA).

[29]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[30]  Rob R. de Ruyter van Steveninck,et al.  The metabolic cost of neural information , 1998, Nature Neuroscience.

[31]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[32]  Andrew S. Cassidy,et al.  Real-Time Scalable Cortical Computing at 46 Giga-Synaptic OPS/Watt with ~100× Speedup in Time-to-Solution and ~100,000× Reduction in Energy-to-Solution , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[33]  Maya Gokhale,et al.  Processing in Memory: The Terasys Massively Parallel PIM Array , 1995, Computer.

[34]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[35]  L. Chua.  Memristor-The missing circuit element , 1971 .

[36]  Hae-Seung Lee,et al.  A Column-Row-Parallel ASIC architecture for 3D wearable / portable medical ultrasonic imaging , 2014, 2014 Symposium on VLSI Circuits Digest of Technical Papers.

[37]  Paul M. Solomon,et al.  In Quest of the “Next Switch”: Prospects for Greatly Reduced Power Dissipation in a Successor to the Silicon Field-Effect Transistor , 2010, Proceedings of the IEEE.