A 41.3pJ/26.7pJ per neuron weight RBM processor for on-chip learning/inference applications

A restricted Boltzmann machine (RBM) processor (RBM-P) supporting on-chip learning and inference is proposed for machine learning applications in this paper. Featuring neural network (NN) model reduction for external memory bandwidth saving, low power neuron binarizer (LPNB) with dynamic clock gating and area-efficient NN-like activation function calculators, user-defined connection map (UDCM) for both computation time and bandwidth saving, and early stopping (ES) mechanism in learning process, the proposed system integrates 32 RBM cores with maximal 4k neurons per layer and 128 candidates per sample for machine learning applications. Implemented in 65nm CMOS technology, the proposed RBM-P chip costs 2.2M gates and 128kB SRAM with 8.8mm2 area. Operated at 1.2V and 210MHz, this chip respectively achieves 114.3x and 3.9x faster processing time than CPU and GPGPU. And the proposed RBM-P chip consumes 41.3pJ and 26.7pJ per neuron weight (NW) for learning and inference, respectively.

[1]  Chen-Yi Lee,et al.  A Hardware-Efficient Sigmoid Function With Adjustable Precision for a Neural Network System , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[2]  Yingchieh Ho,et al.  A 48.6-to-105.2 µW Machine Learning Assisted Cardiac Sensor SoC for Mobile Healthcare Applications , 2014, IEEE Journal of Solid-State Circuits.

[3]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[4]  Zhengya Zhang,et al.  A 640M pixel/s 3.65mW sparse event-driven neuromorphic object recognition processor with on-chip learning , 2015, 2015 Symposium on VLSI Circuits (VLSI Circuits).

[5]  Hoi-Jun Yoo,et al.  4.6 A1.93TOPS/W scalable deep learning/inference processor with tetra-parallel MIMD architecture for big-data applications , 2015, 2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers.

[6]  Berin Martini,et al.  A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.