Calibrating Process Variation at System Level with In-Situ Low-Precision Transfer Learning for Analog Neural Network Processors

Process Variation (PV) may cause accuracy loss of the analog neural network (ANN) processors, and make it hard to be scaled down, as well as feasibility degrading. This paper first analyses the impact of PV on the performance of ANN chips. Then proposes an in-situ transfer learning method at system level to reduce PV's influence with low-precision back-propagation. Simulation results show the proposed method could increase 50% tolerance of operating point drift and 70%∼00% tolerance of mismatch with less than 1% accuracy loss of benchmarks. It also reduces 66.7% memories and has about 50 × energy-efficiency improvement of multiplication in the learning stage, compared with the conventional full-precision (32bit float) training system.

[1]  David Blaauw,et al.  Analog in-memory subthreshold deep neural network accelerator , 2017, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[2]  Steven R. Young,et al.  A 1 TOPS/W Analog Deep Machine-Learning Engine With Floating-Gate Storage in 0.13 µm CMOS , 2014, IEEE Journal of Solid-State Circuits.

[3]  Tadashi Shibata,et al.  An On-Chip-Trainable Gaussian-Kernel Analog Support Vector Machine , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  Maurizio Valle,et al.  Analog VLSI Implementation of Artificial Neural Networks with Supervised On-Chip Learning , 2002 .

[5]  Saurabh Dighe,et al.  Within-Die Variation-Aware Dynamic-Voltage-Frequency-Scaling With Optimal Core Allocation and Thread Hopping for the 80-Core TeraFLOPS Processor , 2011, IEEE Journal of Solid-State Circuits.

[6]  Marcel J. M. Pelgrom,et al.  Matching properties of MOS transistors , 1989 .

[7]  Song Han,et al.  EIE: Efficient Inference Engine on Compressed Deep Neural Network , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[8]  Olivier Temam,et al.  Leveraging the error resilience of machine-learning applications for designing highly energy efficient accelerators , 2014, 2014 19th Asia and South Pacific Design Automation Conference (ASP-DAC).

[9]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[10]  Li Shang,et al.  Process variation characterization of chip-level multiprocessors , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[11]  Puneet Gupta,et al.  VaMV: Variability-aware Memory Virtualization , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[12]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[13]  Swarup Bhunia,et al.  Reliability-Driven ECC Allocation for Multiple Bit Error Resilience in Processor Cache , 2011, IEEE Transactions on Computers.

[14]  Lin Zhong,et al.  RedEye: Analog ConvNet Image Sensor Architecture for Continuous Mobile Vision , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[15]  Chun Jason Xue,et al.  SLC-enabled wear leveling for MLC PCM considering process variation , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[16]  L. Gatet,et al.  Analog Neural Network Implementation for a Real-Time Surface Classification Application , 2008, IEEE Sensors Journal.

[17]  Jun Yang,et al.  Enhancing phase change memory lifetime through fine-grained current regulation and voltage upscaling , 2011, IEEE/ACM International Symposium on Low Power Electronics and Design.

[18]  Engin Ipek,et al.  Memristive Boltzmann machine: A hardware accelerator for combinatorial optimization and deep learning , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[19]  Jia Wang,et al.  DaDianNao: A Machine-Learning Supercomputer , 2014, 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture.