Design and Implementation of Real-time Semantic Segmentation Network Based on FPGA

With the rapid development of deep learning, the neural network of semantic segmentation has been developed towards the miniaturization of the network structure and the lightweight development of the network model. At the same time, FPGA-based neural network hardware accelerators have been proposed. The situation that the network module is too complex and computationally intensive to implement and apply on edge platforms is gradually being solved. However, the implementation of real-time processing network on the edge platform is still of great significance in many areas, such as robots, UAVs, driverless, etc. In this paper, a lightweight semantically segmented neural network Efficient neural network (E-Net) is designed and implemented on the image acquisition board with Zynq 7035 FPGA as processing unit, which achieves the frame rate of 32.9 FPS and meets the requirements of real-time processing.

[1]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[2]  Fude Cao,et al.  A Survey On Image Semantic Segmentation Methods With Convolutional Neural Network , 2020, 2020 International Conference on Communications, Information System and Computer Engineering (CISCE).

[3]  Alexander Karimov,et al.  Comparison of UNet, ENet, and BoxENet for Segmentation of Mast Cells in Scans of Histological Slices , 2019, 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON).

[4]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Lu Tian,et al.  Real-Time Object Detection and Semantic Segmentation Hardware System with Deep Learning Networks , 2018, 2018 International Conference on Field-Programmable Technology (FPT).

[8]  Haolin Liu,et al.  An Efficient Task Assignment Framework to Accelerate DPU-Based Convolutional Neural Network Inference on FPGAs , 2020, IEEE Access.