FPGA-based Acceleration of Lidar Point Cloud Processing and Detection on the Edge

Edge nodes such as Intelligent Transportation System Stations are becoming increasingly relevant in the context of automated driving as they provide connected vehicles with additional information to support their automated driving functions. However, the power budget for these edge nodes is limited and data has to be processed in real-time to be of use to automated driving functions. In this work, we present a system for processing raw lidar data in real-time on an FPGA, resulting in a significant reduction in power consumption compared to conventional hardware. Our approach leads to a 42.4% reduction in power consumption while maintaining the quality of the results. Processing two 128-layer surround-view lidar point clouds takes 522 ms per frame and an average power consumption of 39.3 W for the CPU and 34.5W for the FPGA. Our optimizations surpass the state-of-the-art by up to 193 times.

[1]  T. Gemmeke,et al.  Design of High-Throughput Mixed-Precision CNN Accelerators on FPGA , 2022, 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL).

[2]  K. Lis,et al.  Implementation of the PointPillars Network for 3D Object Detection in Reprogrammable Heterogeneous Devices Using FINN , 2021, Journal of Signal Processing Systems.

[3]  João L. Monteiro,et al.  Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy , 2021, Inf. Fusion.

[4]  Yajun Ha,et al.  Efficient FPGA Implementation of K-Nearest-Neighbor Search Algorithm for 3D LIDAR Localization and Mapping in Smart Vehicles , 2020, IEEE Transactions on Circuits and Systems II: Express Briefs.

[5]  Lutz Eckstein,et al.  Real-Time Point Cloud Fusion of Multi-LiDAR Infrastructure Sensor Setups with Unknown Spatial Location and Orientation , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[6]  Lutz Eckstein,et al.  High-Precision Digital Traffic Recording with Multi-LiDAR Infrastructure Sensor Setups* , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[7]  Shuji Oishi,et al.  Voxelized GICP for Fast and Accurate 3D Point Cloud Registration , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Jiong Yang,et al.  PointPillars: Fast Encoders for Object Detection From Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Eriko Nurvitadhi,et al.  Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks? , 2017, FPGA.

[10]  Hao Yu,et al.  A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks , 2017, ACM J. Emerg. Technol. Comput. Syst..

[11]  Philip Heng Wai Leong,et al.  FINN: A Framework for Fast, Scalable Binarized Neural Network Inference , 2016, FPGA.

[12]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[13]  Klaus C. J. Dietmayer,et al.  Simulation and calibration of infrastructure based laser scanner networks at intersections , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[14]  Nico Blodow,et al.  Towards 3D Point cloud based object maps for household environments , 2008, Robotics Auton. Syst..

[15]  서정헌,et al.  반도체 공정 overview , 2001 .

[16]  A. Knoll,et al.  Intelligent Transportation Systems With The Use of External Infrastructure: A Literature Survey , 2021, ArXiv.

[17]  Cecilia Latotzke,et al.  Efficiency Versus Accuracy: A Review of Design Techniques for DNN Hardware Accelerators , 2021, IEEE Access.

[18]  Lutz Eckstein,et al.  Collective Driving : Cloud Services for Automated Vehicles in UNICARagil , 2019 .