Design of A Bit-Serial Artificial Neuron VLSI Architecture with Early Termination

In this paper, a VLSI design of a bit-serial artificial neuron circuit is proposed. Different from the ordinary bit-serial architectures which usually start from the least significant bit (LSB), the proposed design will start from processing the most significant bit (MSB). For the MSB-first approach, the more significant part of results will be generated earlier, and the intermediate results will be progressively refined by processing the less significant bits. An artificial neuron is equipped with an activation function at the output, and many common used activation functions such as sigmoid, a rectified linear unit (ReLU) etc will saturate to 0 for large negative inputs. Some will saturate to 1 for large positive inputs. Therefore, when the intermediate results are positive or negative enough, the remaining processing of less significant bits can be neglected. Our preliminary results shows that the approximation results due to the proposed early termination can still lead to the same classification accuracy as the full precision, but the processing cycles can be reduced by more than 25%. The proposed methodology can be applied to the design of hardware accelerators for those machine learning networks based on neurons such as neural network (NN) and convolution NN (CNN).

[1]  Keshab K. Parhi,et al.  VLSI digital signal processing systems , 1999 .

[2]  Marian Verhelst,et al.  An Energy-Efficient Precision-Scalable ConvNet Processor in 40-nm CMOS , 2017, IEEE Journal of Solid-State Circuits.

[3]  Zhongfeng Wang,et al.  An Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks , 2018, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[4]  Yu Wang,et al.  Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[5]  Jun-Seok Park,et al.  14.6 A 1.42TOPS/W deep convolutional neural network recognition processor for intelligent IoE systems , 2016, 2016 IEEE International Solid-State Circuits Conference (ISSCC).

[6]  Hadi Esmaeilzadeh,et al.  Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network , 2017, 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA).

[7]  Joel Emer,et al.  Eyeriss: an Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks Accessed Terms of Use , 2022 .

[8]  Patrick Judd,et al.  Stripes: Bit-serial deep neural network computing , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[9]  Hoi-Jun Yoo,et al.  UNPU: A 50.6TOPS/W unified deep neural network accelerator with 1b-to-16b fully-variable weight bit-precision , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[10]  Shaoli Liu,et al.  Cambricon-X: An accelerator for sparse neural networks , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[11]  Lee-Sup Kim,et al.  Energy-Efficient Design of Processing Element for Convolutional Neural Network , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.