Automatic Labeling to Generate Training Data for Online LiDAR-Based Moving Object Segmentation

Understanding the scene is key for autonomously navigating vehicles and the ability to segment the surroundings online into moving and non-moving objects is a central ingredient for this task. Often, deep learning-based methods are used to perform moving object segmentation (MOS). The performance of these networks, however, strongly depends on the diversity and amount of labeled training data—information that may be costly to obtain. In this paper, we propose an automatic data labeling pipeline for 3D LiDAR data to save the extensive manual labeling effort and to improve the performance of existing learning-based MOS systems by automatically generating labeled training data. Our proposed approach achieves this by processing the data offline in batches. It first exploits an occupancy-based dynamic object removal to detect possible dynamic objects coarsely. Second, it extracts segments among the proposals and tracks them using a Kalman filter. Based on the tracked trajectories, it labels the actually moving objects such as driving cars and pedestrians as moving. In contrast, the non-moving objects, e.g., parked cars, lamps, roads, or buildings, are labeled as static. We show that this approach allows us to label LiDAR data highly effectively and compare our results to those of other label generation methods. We also train a deep neural network with our auto-generated labels and achieve similar performance compared to the one trained with manual labels on the same data—and an even better performance when using additional datasets with labels generated by our approach. Furthermore, we evaluate our method on multiple datasets using different sensors and our experiments indicate that our method can generate labels in diverse environments.

[1]  Sajid Javed,et al.  Graph Moving Object Segmentation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jinyong Jeong,et al.  MulRan: Multimodal Range Dataset for Urban Place Recognition , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Subrahmanyam Murala,et al.  An End-to-End Edge Aggregation Network for Moving Object Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Leonidas J. Guibas,et al.  KPConv: Flexible and Deformable Convolution for Point Clouds , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Hao Wang,et al.  SpSequenceNet: Semantic Segmentation Network on 4D Point Clouds , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[7]  Cyrill Stachniss,et al.  Fast range image-based segmentation of sparse 3D laser scans for online operation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Arthur Zimek,et al.  Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection , 2015, ACM Trans. Knowl. Discov. Data.

[9]  Cyrill Stachniss,et al.  Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks , 2021, CoRL.

[10]  Cyrill Stachniss,et al.  Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments , 2018, Robotics: Science and Systems.

[11]  Armin B. Cremers,et al.  Laser-based segment classification using a mixture of bag-of-words , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Bruno Vallet,et al.  Street environment change detection from mobile laser scanning point clouds , 2015 .

[13]  Cyrill Stachniss,et al.  Moving Object Segmentation in 3D LiDAR Data: A Learning-Based Approach Exploiting Sequential Data , 2021, IEEE Robotics and Automation Letters.

[14]  Cyrill Stachniss,et al.  Mapping the Static Parts of Dynamic Scenes from 3D LiDAR Point Clouds Exploiting Ground Segmentation , 2021, 2021 European Conference on Mobile Robots (ECMR).

[15]  Cyrill Stachniss,et al.  RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Stefan A. Baur,et al.  SLIM: Self-Supervised LiDAR Scene Flow and Motion Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Andreas Nüchter,et al.  The Peopleremover—Removing Dynamic Objects From 3-D Point Cloud Data by Traversing a Voxel Occupancy Grid , 2018, IEEE Robotics and Automation Letters.

[19]  Hyun Myung,et al.  ERASOR: Egocentric Ratio of Pseudo Occupancy-Based Dynamic Object Removal for Static 3D Point Cloud Map Building , 2021, IEEE Robotics and Automation Letters.

[20]  Leonidas J. Guibas,et al.  FlowNet3D: Learning Scene Flow in 3D Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Cyrill Stachniss,et al.  Multi-Scale Interaction for Real-Time LiDAR Data Segmentation on an Embedded Platform , 2020, IEEE Robotics and Automation Letters.

[22]  Wolfram Burgard,et al.  Autonomous Robot Navigation in Highly Populated Pedestrian Zones , 2015, J. Field Robotics.

[23]  Uwe Stilla,et al.  An Approach to Extract Moving Objects from Mls Data Using a Volumetric Background Representation , 2017 .

[24]  Ayoung Kim,et al.  Remove, then Revert: Static Point cloud Map Construction using Multiresolution Range Images , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Tim D. Barfoot,et al.  Mapless Online Detection of Dynamic Objects in 3D Lidar , 2018, 2019 16th Conference on Computer and Robot Vision (CRV).

[26]  Roland Siegwart,et al.  Dynamic Object Aware LiDAR SLAM based on Automatic Generation of Training Data , 2021, IEEE International Conference on Robotics and Automation.

[27]  Andreas Geiger,et al.  KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D , 2021, ArXiv.

[28]  Eren Erdal Aksoy,et al.  SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds , 2020, ISVC.

[29]  Wolfram Burgard,et al.  Rigid scene flow for 3D LiDAR scans , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Jianren Wang,et al.  3D Multi-Object Tracking: A Baseline and New Evaluation Metrics , 2019 .

[31]  Cyrill Stachniss,et al.  Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset , 2021, Int. J. Robotics Res..

[32]  Cyrill Stachniss,et al.  SuMa++: Efficient LiDAR-based Semantic SLAM , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Hyungjin Kim,et al.  Robust Method for Removing Dynamic Objects from Point Clouds , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Shenhua Hou,et al.  L3-Net: Towards Learning Based LiDAR Localization for Autonomous Driving , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  C. Stachniss,et al.  Learning an Overlap-based Observation Model for 3D LiDAR Localization , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[36]  Cyrill Stachniss,et al.  SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Cyrill Stachniss,et al.  Inferring Objectives in Continuous Dynamic Games from Noise-Corrupted Partial State Observations , 2021, Robotics: Science and Systems.

[38]  Roland Siegwart,et al.  Long-term 3D map maintenance in dynamic environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[40]  Leonidas J. Guibas,et al.  Weakly Supervised Learning of Rigid 3D Scene Flow , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  C. Stachniss,et al.  OverlapNet: a siamese network for computing LiDAR scan similarity with applications to loop closing and localization , 2021, Autonomous Robots.

[42]  Ricardo J. G. B. Campello,et al.  Density-Based Clustering Based on Hierarchical Density Estimates , 2013, PAKDD.