A Digital Implementation of Extreme Learning Machines for Resource-Constrained Devices

The availability of compact digital circuitry for the support of neural networks is a key requirement for resource-constrained embedded systems. This brief tackles the implementation of single hidden-layer feedforward neural networks, based on hard-limit activation functions, on reconfigurable devices. The resulting design strategy relies on a novel learning procedure that inherits the approach adopted in the Extreme Learning Machine paradigm. The eventual training process balances accuracy and network complexity effectively, thus supporting a digital architecture that prioritizes area utilization over computational performance. Experimental tests confirm that the design approach leads to efficient digital implementations of the predictor on low-performance devices.

[1]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[2]  Eugenio Culurciello,et al.  Evaluation of neural network architectures for embedded systems , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[3]  Paolo Gastaldo,et al.  Learning with similarity functions: A novel design for the extreme learning machine , 2017 .

[4]  Hongming Zhou,et al.  Silicon spiking neurons for hardware implementation of extreme learning machines , 2013, Neurocomputing.

[5]  Leo Breiman,et al.  Technical note: Some properties of splitting criteria , 2004, Machine Learning.

[6]  Paolo Gastaldo,et al.  Efficient Digital Implementation of Extreme Learning Machines for Classification , 2012, IEEE Transactions on Circuits and Systems II: Express Briefs.

[7]  Diane J. Cook,et al.  Pervasive computing at scale: Transforming the state of the art , 2012, Pervasive Mob. Comput..

[8]  Li-Rong Zheng,et al.  A Low-Power Accelerator for Deep Neural Networks with Enlarged Near-Zero Sparsity , 2017, ArXiv.

[9]  Igor Carron,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016 .

[10]  Marek Wegrzyn,et al.  Hardware implementation of real-time Extreme Learning Machine in FPGA: Analysis of precision, resource occupation and performance , 2016, Comput. Electr. Eng..

[11]  Chee Kheong Siew,et al.  Can threshold networks be trained directly? , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.