Automatic Label Injection Into Local Infrastructure LiDAR Point Cloud for Training Data Set Generation

The representation of objects in LiDAR point clouds is changed as the height of the mounting position of sensor devices gets increased. Most of the available open datasets for training machine learning based object detectors are generated with vehicle top mounted sensors, thus the detectors trained on such datasets perform weaker when the sensor is observing the scene from a significantly higher viewpoint (e.g. infrastructure sensor). In this paper a novel Automatic Label Injection method is proposed to label the objects in the point cloud of the high-mounted infrastructure LiDAR sensor based on the output of a well performing “trainer” detector deployed at optimal height while considering the uncertainties caused by various factors described in detail throughout the paper. The proposed automatic labeling approach has been validated on a small scale sensor setup in a real-world traffic scenario where accurate differential GNSS reference data where also available for each test vehicle. Furthermore, the concept of a distributed multi-sensor system covering a larger area aimed for automatic dataset generation is also presented. It is shown that a machine learning based detector trained on differential GNSS-based training dataset performs very similarly to the detector retrained on a dataset generated by the proposed Automatic Label Injection technique. According to our results a significant increase in the maximum detection range can be achieved by retraining the detector on viewpoint specific data generated fully automatically by the proposed label injection technique compared to a detector trained on vehicle top mounted sensor data.

[1]  Christian Cress,et al.  A9-Dataset: Multi-Sensor Infrastructure-Based Dataset for Mobility Research , 2022, 2022 IEEE Intelligent Vehicles Symposium (IV).

[2]  Kilian Q. Weinberger,et al.  Learning to Detect Mobile Objects from LiDAR Scans Without Labels , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Cyrill Stachniss,et al.  Automatic Labeling to Generate Training Data for Online LiDAR-Based Moving Object Segmentation , 2022, IEEE Robotics and Automation Letters.

[4]  João L. Monteiro,et al.  Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy , 2021, Inf. Fusion.

[5]  Pál Varga,et al.  Motorway Measurement Campaign to Support R&D Activities in the Field of Automated Driving Technologies , 2021, Sensors.

[6]  Bin Tian,et al.  3D Vehicle Detection With RSU LiDAR for Autonomous Mine , 2021, IEEE Transactions on Vehicular Technology.

[7]  Thomas Funkhouser,et al.  Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Krzysztof Czarnecki,et al.  Canadian Adverse Driving Conditions dataset , 2020, Int. J. Robotics Res..

[9]  Sammy Omari,et al.  One Thousand and One Hours: Self-driving Motion Prediction Dataset , 2020, CoRL.

[10]  Dragomir Anguelov,et al.  Scalability in Perception for Autonomous Driving: Waymo Open Dataset , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Simon Lucey,et al.  Argoverse: 3D Tracking and Forecasting With Rich Maps , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Zsolt Szalay,et al.  A Novel Model Representation Framework for Cooperative Intelligent Transport Systems , 2019, Periodica Polytechnica Transportation Engineering.

[14]  Huosheng Hu,et al.  Automatic Generation of Synthetic LiDAR Point Clouds for 3-D Data Analysis , 2019, IEEE Transactions on Instrumentation and Measurement.

[15]  Jiong Yang,et al.  PointPillars: Fast Encoders for Object Detection From Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Andrea Tagliasacchi,et al.  Sparse Iterative Closest Point , 2013, Comput. Graph. Forum.

[17]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ryan M. Eustice,et al.  Ford Campus vision and lidar data set , 2011, Int. J. Robotics Res..

[19]  Sebastian Thrun,et al.  Towards 3D object recognition via classification of arbitrary object tracks , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  David L. Mills,et al.  Internet time synchronization: the network time protocol , 1991, IEEE Trans. Commun..

[22]  Richard B. Langley,et al.  The UTM Grid System , 2022 .