ResCoNN: Resource-Efficient FPGA-Accelerated CNN for Traffic Sign Classification

Precise classification and detection of (distorted and normal) traffic signs in real-time is one of the non-trivial requirements for safe autonomous driving. The state-of-the-art convolutional neural networks (CNNs) for traffic sign detection though accurate are resource-hungry due to their inherent structure with massive networks with millions of full-precision parameters making them infeasible for low-end FPGA platforms leading to higher implementation costs. The existing works employing low-precision bits and similar techniques though hardware friendly leads to degradation in accuracy. In this work, we propose a resource-efficient CNN (ResCoNN) architecture with a small number of weights (only 60,000 compared to a few million in state-of-the-art CNNs) and employ for traffic sign detection and classification. For increased efficiency, the network takes advantage of binary weights and integer activations, rather than employing complex computations like Batch Normalization and Exponential Linear Units. ResCoNN achieves a classification accuracy of > 96% on real-world images at a framerate of 36fps on a Zynq SoC (xc7z020clg484-1) with 90% reduced weights compared to state-of-the-art CNNs.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Axel Jantsch,et al.  Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW Implementation , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[3]  Peter H. N. de With,et al.  Color exploitation in hog-based traffic sign detection , 2010, 2010 IEEE International Conference on Image Processing.

[4]  Johannes Stallkamp,et al.  Detection of traffic signs in real-world images: The German traffic sign detection benchmark , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[5]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Greg Brown,et al.  A performance and energy comparison of FPGAs, GPUs, and multicores for sliding-window applications , 2012, FPGA '12.

[7]  Wang Cong,et al.  A traffic sign detection algorithm based on deep convolutional neural network , 2016, 2016 IEEE International Conference on Signal and Image Processing (ICSIP).

[8]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[9]  M. Gokmen,et al.  Traffic sign recognition using Scale Invariant Feature Transform and color classification , 2008, 2008 23rd International Symposium on Computer and Information Sciences.

[10]  Zhiyi Yu,et al.  An FPGA-Based Hardware Accelerator for Traffic Sign Detection , 2017, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Yoshua Bengio,et al.  BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1 , 2016, ArXiv.

[13]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  David Ribeiro,et al.  A Real-Time Pedestrian Detector using Deep Learning for Human-Aware Navigation , 2016, ArXiv.

[16]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[17]  Michael Hübner,et al.  FPGA based traffic sign detection for automotive camera systems , 2015, 2015 10th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC).

[18]  Jörg Henkel,et al.  Machine Learning for Power, Energy, and Thermal Management on Multicore Processors: A Survey , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[19]  Xiaolin Hu,et al.  Traffic sign detection by ROI extraction and histogram features-based recognition , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[24]  Axel Jantsch,et al.  Neural network based ECG anomaly detection on FPGA and trade-off analysis , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[25]  Jonathan Tompson,et al.  Efficient object localization using Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Khaldoon M. Mhaidat,et al.  Implementing image processing algorithms in FPGA hardware , 2013, 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[27]  Frans Coenen,et al.  Road surface traffic sign detection with hybrid region proposal and fast R-CNN , 2016, 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[28]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[29]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[31]  Xuegong Zhou,et al.  A high performance FPGA-based accelerator for large-scale convolutional neural networks , 2016, 2016 26th International Conference on Field Programmable Logic and Applications (FPL).

[32]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[33]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[34]  Xu Wang,et al.  Traffic Signs Detection Based on Faster R-CNN , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[35]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[36]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[37]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.