FPGA acceleration on a multi-layer perceptron neural network for digit recognition

This paper proposes field-programmable gate array (FPGA) acceleration on a scalable multi-layer perceptron (MLP) neural network for classifying handwritten digits. First, an investigation to the network architectures is conducted to find the optimal FPGA design corresponding to different classification rates. As a case study, then a specific single-hidden-layer MLP network is implemented with an eight-stage pipelined structure on Xilinx Ultrascale FPGA. It mainly contains a timing controller designed by Verilog Hardware Description Language (HDL) and sigmoid neurons integrated by Xilinx IPs. Finally, experimental results show a greater than $$\times 10$$ speedup compared with prior implementations. The proposed FPGA architecture is expandable to other specifications on different accuracy (up to 95.82%) and hardware cost.

[1]  Ronald Davis,et al.  Neural networks and deep learning , 2017 .

[2]  Qi Zhang,et al.  FPGA Implementation of Quantized Convolutional Neural Networks , 2019, 2019 IEEE 19th International Conference on Communication Technology (ICCT).

[3]  Sarah L. Harris,et al.  A SS-CNN on an FPGA for Handwritten Digit Recognition , 2019, 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[4]  Ming-Hwa Sheu,et al.  Implementation of FPGA-based Accelerator for Deep Neural Networks , 2019, 2019 IEEE 22nd International Symposium on Design and Diagnostics of Electronic Circuits & Systems (DDECS).

[5]  Youngmin Kim,et al.  Implementation of Data-optimized FPGA-based Accelerator for Convolutional Neural Network , 2020, 2020 International Conference on Electronics, Information, and Communication (ICEIC).

[6]  Pavlo Molchanov,et al.  Importance Estimation for Neural Network Pruning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Archit Gajjar,et al.  Real-Time Automatic Music Transcription (AMT) with Zync FPGA , 2019, 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI).

[8]  Xiaohui Liu,et al.  Hardware Implementation of Energy Efficient Deep Learning Neural Network Based on Nanoscale Flash Computing Array , 2019, Advanced Materials Technologies.

[9]  Jeremy Kepner,et al.  Survey and Benchmarking of Machine Learning Accelerators , 2019, 2019 IEEE High Performance Extreme Computing Conference (HPEC).

[10]  Yu Wang,et al.  A Survey of FPGA-Based Neural Network Accelerator , 2017, 1712.08934.

[11]  Yasar Becerikli,et al.  FPGA-Based Optimized Convolutional Neural Network Framework for Handwritten Digit Recognition , 2019, 2019 1st International Informatics and Software Engineering Conference (UBMYK).

[12]  Jwan Najeeb Saeed,et al.  A Comparison of Three Classification Algorithms for Handwritten Digit Recognition , 2019, 2019 International Conference on Advanced Science and Engineering (ICOASE).

[13]  Haluk Rahmi Topcuoglu,et al.  Neural network based multi-objective evolutionary algorithm for dynamic workflow scheduling in cloud computing , 2020, Future Gener. Comput. Syst..

[14]  Gregory K. Chen,et al.  Evaluating and Enhancing Intel® Stratix® 10 FPGAs for Persistent Real-Time AI , 2019, FPGA.

[15]  Tobi Delbrück,et al.  Real-Time Speech Recognition for IoT Purpose using a Delta Recurrent Neural Network Accelerator , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[16]  Tang Yongming,et al.  Accelerator Implementation of Lenet-5 Convolution Neural Network Based on FPGA with HLS , 2019, 2019 3rd International Conference on Circuits, System and Simulation (ICCSS).

[17]  Ahmad Shawahna,et al.  FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review , 2019, IEEE Access.

[18]  Amit Choudhary,et al.  Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN) , 2020, Sensors.

[19]  Syama Sundar Rangapuram,et al.  Neural forecasting: Introduction and literature overview , 2020, ArXiv.

[20]  Chao Zhang,et al.  FPGA Implementation of CNN for Handwritten Digit Recognition , 2020, 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[21]  Sarah L. Harris,et al.  Handwritten digit recognition system on an FPGA , 2018, 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC).

[22]  Joonki Paik,et al.  Deep Learning–based Number Detection and Recognition for Gas Meter Reading , 2019 .

[23]  David Gschwend,et al.  ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network , 2020, ArXiv.

[24]  Tariq Mahmood,et al.  An efficient and improved scheme for handwritten digit recognition based on convolutional neural network , 2019, SN Applied Sciences.

[25]  Omer Can Akgun,et al.  An energy efficient time-mode digit classification neural network implementation , 2019, Philosophical Transactions of the Royal Society A.

[26]  Jason Cong,et al.  High-Level Synthesis for FPGAs: From Prototyping to Deployment , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[27]  Jun Guo,et al.  An Exceedingly Fast Model for Low Resolution Handwritten Digit String Recognition , 2019, 2019 IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT).

[28]  Qin Li,et al.  Implementing neural machine translation with bi-directional GRU and attention mechanism on FPGAs using HLS , 2019, ASP-DAC.