PR-CIM: a Variation-Aware Binary-Neural-Network Framework for Process-Resilient Computation-in-memory

Binary neural networks (BNNs) that use 1-bit weights and activations have garnered interest as extreme quantization provides low power dissipation. By implementing BNNs as computing-in-memory (CIM), which computes multiplication and accumulations on memory arrays in an analog fashion, namely analog CIM, we can further improve the energy efficiency to process neural networks. However, analog CIMs suffer from the potential problem that process variation degrades the accuracy of BNNs. Our Monte-Carlo simulations show that in an SRAMbased analog CIM of VGG-9, the classification accuracy of CIFAR-10 is degraded even below 20% under process variations of 65nm CMOS. To overcome this problem, we present a variation-aware BNN framework. The proposed framework is developed for SRAM-based BNN CIMs since SRAM is most widely used as on-chip memory, however easily extensible to BNN CIMs based on other memories. Our extensive experimental results show that under process variation of 65nm CMOS, our framework significantly improves the CIFAR-10 accuracies of SRAM-based BNN CIMs, from 10% and 10.1% to 87.76% and 77.74% for VGG-9 and RESNET-18 respectively.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Jae-Joon Kim,et al.  Effect of Device Variation on Mapping Binary Neural Network to Memristor Crossbar Array , 2019, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[3]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[4]  Xiaochen Peng,et al.  Fully parallel RRAM synaptic array for implementing binary neural network with (+1, −1) weights and (+1, 0) neurons , 2018, 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC).

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Xiaochen Peng,et al.  XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[7]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, NIPS.

[8]  Damien Querlioz,et al.  Implementation of Ternary Weights With Resistive RAM Using a Single Sense Operation Per Synapse , 2021, IEEE Transactions on Circuits and Systems I: Regular Papers.

[9]  Meng-Fan Chang,et al.  Parallelizing SRAM Arrays with Customized Bit-Cell for Binary Neural Networks , 2018, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[10]  Hyungjun Kim,et al.  BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations , 2020, ICLR.

[11]  Tung Thanh Hoang,et al.  An In-Flash Binary Neural Network Accelerator with SLC NAND Flash Array , 2020, ISCAS.

[12]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Hyungjun Kim,et al.  Energy-efficient XNOR-free In-Memory BNN Accelerator with Input Distribution Regularization , 2020, 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD).

[14]  Anastacia B. Alvarez,et al.  Static Noise Margin of 6T SRAM Cell in 90-nm CMOS , 2011, 2011 UkSim 13th International Conference on Computer Modelling and Simulation.

[15]  Jae-Joon Kim,et al.  In-memory batch-normalization for resistive memory based binary neural network hardware , 2019, ASP-DAC.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jae-Joon Kim,et al.  Algorithm/Hardware Co-Design for In-Memory Neural Network Computing with Minimal Peripheral Circuit Overhead , 2020, 2020 57th ACM/IEEE Design Automation Conference (DAC).

[18]  Amro Awad,et al.  MRIMA: An MRAM-Based In-Memory Accelerator , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[19]  Mouloud Belbahri,et al.  How Does Batch Normalization Help Binary Training , 2019 .

[20]  Jae-sun Seo,et al.  XNOR-SRAM: In-Memory Computing SRAM Macro for Binary/Ternary Deep Neural Networks , 2018, 2018 IEEE Symposium on VLSI Technology.

[21]  Jae-Joon Kim,et al.  Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators , 2018, ArXiv.