Reduction of Neural Network Circuits by Constant and Nearly Constant Signal Propagation

This work focuses on optimizing circuits representing neural networks (NNs) in the form of and-inverter graphs (AIGs). The optimization is done by analyzing the training set of the neural network to find constant bit values at the primary inputs. The constant values are then propagated through the AIG, which results in removing unnecessary nodes. Furthermore, a trade-off between neural network accuracy and its reduction due to constant propagation is investigated by replacing with constants those inputs that are likely to be zero or one. The experimental results show a significant reduction in circuit size with negligible loss in accuracy.

[1]  Yu Wang,et al.  Going Deeper with Embedded FPGA Platform for Convolutional Neural Network , 2016, FPGA.

[2]  Chen Wang,et al.  Approximate Disjoint Bi-Decomposition and Its Application to Approximate Logic Synthesis , 2017, 2017 IEEE International Conference on Computer Design (ICCD).

[3]  Earl E. Swartzlander,et al.  Computer Arithmetic , 1980 .

[4]  Robert K. Brayton,et al.  DAG-aware AIG rewriting: a fresh look at combinational logic synthesis , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[5]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Jie Han,et al.  Approximate computing: An emerging paradigm for energy-efficient design , 2013, 2013 18th IEEE European Test Symposium (ETS).

[7]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[8]  Ronald Davis,et al.  Neural networks and deep learning , 2017 .

[9]  Armin Biere.  The AIGER And-Inverter Graph (AIG) Format Version 20070427 , 2007 .

[10]  Robert K. Brayton,et al.  Scalable don't-care-based logic optimization and resynthesis , 2009, FPGA '09.

[11]  Alan Mishchenko,et al.  Fast Algebraic Rewriting Based on And-Inverter Graphs , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[12]  Naoya Onizawa,et al.  VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[13]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yi Wu,et al.  An efficient method for multi-level approximate logic synthesis under error rate constraint , 2016, 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[15]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[16]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[17]  Kaushik Roy,et al.  Substitute-and-simplify: A unified design paradigm for approximate and quality configurable circuits , 2013, 2013 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[18]  Giovanni De Micheli,et al.  Deep Learning for Logic Optimization Algorithms , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).