Author Correction: Ferroelectric ternary content-addressable memory for one-shot learning

Deep neural networks are efficient at learning from large sets of labelled data, but struggle to adapt to previously unseen data. In pursuit of generalized artificial intelligence, one approach is to augment neural networks with an attentional memory so that they can draw on already learnt knowledge patterns and adapt to new but similar tasks. In current implementations of such memory augmented neural networks (MANNs), the content of a network’s memory is typically transferred from the memory to the compute unit (a central processing unit or graphics processing unit) to calculate similarity or distance norms. The processing unit hardware incurs substantial energy and latency penalties associated with transferring the data from the memory and updating the data at random memory addresses. Here, we show that ternary content-addressable memories (TCAMs) can be used as attentional memories, in which the distance between a query vector and each stored entry is computed within the memory itself, thus avoiding data transfer. Our compact and energy-efficient TCAM cell is based on two ferroelectric field-effect transistors. We evaluate the performance of our ferroelectric TCAM array prototype for one- and few-shot learning applications. When compared with a MANN where cosine distance calculations are performed on a graphics processing unit, the ferroelectric TCAM approach provides a 60-fold reduction in energy and 2,700-fold reduction in latency for a single memory search operation. A compact ternary content-addressable memory cell, which is based on two ferroelectric field-effect transistors, can provide memory augmented neural networks with improved energy and latency performance compared with traditional approaches based on graphics processing units.

[1]  Koji Nii,et al.  13.6 A 28nm 400MHz 4-parallel 1.6Gsearch/s 80Mb ternary CAM , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Bogdan Gabrys,et al.  Metalearning: a survey of trends and technologies , 2013, Artificial Intelligence Review.

[4]  Jing Li,et al.  1 Mb 0.41 µm² 2T-2R Cell Nonvolatile TCAM With Two-Bit Encoding and Clocked Self-Referenced Sensing , 2014, IEEE Journal of Solid-State Circuits.

[5]  Michael T. Niemier,et al.  Design and benchmarking of ferroelectric FET based TCAM , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[6]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[8]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[9]  Luan Tran,et al.  45nm low power CMOS logic compatible embedded STT MRAM utilizing a reverse-connection 1T/1MTJ cell , 2009, 2009 IEEE International Electron Devices Meeting (IEDM).

[10]  Farid N. Najm,et al.  A Soft-Error Tolerant Content-Addressable Memory (CAM) Using An Error-Correcting-Match Scheme , 2006, IEEE Custom Integrated Circuits Conference 2006.

[11]  Suman Datta,et al.  Write Disturb in Ferroelectric FETs and Its Implication for 1T-FeFET AND Memory Arrays , 2018, IEEE Electron Device Letters.

[12]  J. Kittl,et al.  SoC Logic Compatible Multi-Bit FeMFET Weight Cell for Neuromorphic Applications , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[13]  Bingsheng He,et al.  Efficient gather and scatter operations on graphics processors , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[14]  Shoji Ikeda,et al.  A 3.14 um2 4T-2MTJ-cell fully parallel TCAM based on nonvolatile logic-in-memory architecture , 2012, 2012 Symposium on VLSI Circuits (VLSIC).

[15]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[16]  Michael Niemier,et al.  An Ultra-Dense 2FeFET TCAM Design Based on a Multi-Domain FeFET Model , 2019, IEEE Transactions on Circuits and Systems II: Express Briefs.

[17]  Tetsuo Endoh,et al.  Fully parallel 6T-2MTJ nonvolatile TCAM with single-transistor-based self match-line discharge control , 2011, 2011 Symposium on VLSI Circuits - Digest of Technical Papers.

[18]  Kinam Kim,et al.  Highly manufacturable high density phase change memory of 64Mb and beyond , 2004, IEDM Technical Digest. IEEE International Electron Devices Meeting, 2004..

[19]  Mohsen Imani,et al.  Approximate Computing Using Multiple-Access Single-Charge Associative Memory , 2018, IEEE Transactions on Emerging Topics in Computing.

[20]  Meng-Fan Chang,et al.  7.4 A 256b-wordlength ReRAM-based TCAM with 1ns search-time and 14× improvement in wordlength-energyefficiency-density product using 2.5T1R cell , 2016, 2016 IEEE International Solid-State Circuits Conference (ISSCC).

[21]  Meng-Fan Chang,et al.  ReRAM-based 4T2R nonvolatile TCAM with 7x NVM-stress reduction, and 4x improvement in speed-wordlength-capacity for normally-off instant-on filter-based search engines used in big-data processing , 2014, 2014 Symposium on VLSI Circuits Digest of Technical Papers.

[22]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[23]  Stefan Slesazeck,et al.  Accumulative Polarization Reversal in Nanoscale Ferroelectric Transistors. , 2018, ACS applied materials & interfaces.

[24]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[25]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[26]  Wei Hwang,et al.  A 65 nm 0.165 fJ/Bit/Search 256 $\,\times\,$144 TCAM Macro Design for IPv6 Lookup Tables , 2011, IEEE Journal of Solid-State Circuits.

[27]  M. Trentzsch,et al.  A 28nm HKMG super low power embedded NVM technology based on ferroelectric FETs , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[28]  Swaroop Ghosh,et al.  Emerging Trends in Design and Applications of Memory-Based Computing and Content-Addressable Memories , 2015, Proceedings of the IEEE.

[29]  Aurko Roy,et al.  Learning to Remember Rare Events , 2017, ICLR.

[30]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[31]  Hoi-Jun Yoo,et al.  A 0.7-fJ/bit/search 2.2-ns search time hybrid-type TCAM architecture , 2004, IEEE Journal of Solid-State Circuits.

[32]  Tetsuo Endoh,et al.  Fine-Grained Power-Gating Scheme of a Metal–Oxide–Semiconductor and Magnetic-Tunnel-Junction-Hybrid Bit-Serial Ternary Content-Addressable Memory , 2010 .

[33]  Sunil P. Khatri,et al.  An area-efficient Ternary CAM design using floating gate transistors , 2014, 2014 IEEE 32nd International Conference on Computer Design (ICCD).

[34]  Seong-Ook Jung,et al.  A 10T-4MTJ Nonvolatile Ternary CAM Cell for Reliable Search Operation and a Compact Area , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[35]  Ashish Goel,et al.  Similarity search and locality sensitive hashing using ternary content addressable memories , 2010, SIGMOD Conference.

[36]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[37]  S. Slesazeck,et al.  Ferroelectricity in HfO2 enables nonvolatile data storage in 28 nm HKMG , 2012, 2012 Symposium on VLSI Technology (VLSIT).

[38]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[39]  Jan M. Rabaey,et al.  Exploring Hyperdimensional Associative Memory , 2017, 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[40]  M. Trentzsch,et al.  A FeFET based super-low-power ultra-fast embedded NVM technology for 22nm FDSOI and beyond , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[41]  S. Slesazeck,et al.  Switching Kinetics in Nanoscale Hafnium Oxide Based Ferroelectric Field-Effect Transistors. , 2017, ACS applied materials & interfaces.

[42]  Michael Niemier,et al.  Design of Hardware-Friendly Memory Enhanced Neural Networks , 2019, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[43]  Yiran Chen,et al.  Design of Spin-Torque Transfer Magnetoresistive RAM and CAM/TCAM with High Sensing and Search Speed , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[44]  Qiong Luo,et al.  Revisiting Multi-pass Scatter and Gather on GPUs , 2018, ICPP.

[45]  David Blaauw,et al.  11.2 A 1Mb embedded NOR flash memory with 39µW program power for mm-scale high-temperature sensor nodes , 2017, 2017 IEEE International Solid-State Circuits Conference (ISSCC).

[46]  S. Datta,et al.  Critical Role of Interlayer in Hf0.5Zr0.5O2 Ferroelectric FET Nonvolatile Memory Performance , 2018, IEEE Transactions on Electron Devices.

[47]  Suman Datta,et al.  A Circuit Compatible Accurate Compact Model for Ferroelectric-FETs , 2018, 2018 IEEE Symposium on VLSI Technology.

[48]  O. Richard,et al.  10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation , 2011, 2011 International Electron Devices Meeting.