A Technical Survey and Evaluation of Traditional Point Cloud Clustering Methods for LiDAR Panoptic Segmentation

LiDAR panoptic segmentation is a newly proposed technical task for autonomous driving. In contrast to popular end-to-end deep learning solutions, we propose a hybrid method with an existing semantic segmentation network to extract semantic information and a traditional LiDAR point cloud cluster algorithm to split each instance object. We argue geometry-based traditional clustering algorithms are worth being considered by showing a state-of-the-art performance among all published end-to-end deep learning solutions on the panoptic segmentation leaderboard of the SemanticKITTI dataset. To our best knowledge, we are the first to attempt the point cloud panoptic segmentation with clustering algorithms. Therefore, instead of working on new models, we give a comprehensive technical survey in this paper by implementing four typical cluster methods and report their performances on the benchmark. Those four cluster methods are the most representative ones with real-time running speed. They are implemented with C++ in this paper and then wrapped as a python function for seamless integration with the existing deep learning frameworks. We release our code for peer researchers who might be interested in this problem 1.

[1]  Hassan Foroosh,et al.  Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Pengfei Duan,et al.  FISHING Net: Future Inference of Semantic Heatmaps In Grids , 2020, ArXiv.

[3]  Federico Tombari,et al.  Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds , 2020, ArXiv.

[4]  Xiao Zhao,et al.  The connected-component labeling problem: A review of state-of-the-art algorithms , 2017, Pattern Recognit..

[5]  Cyrill Stachniss,et al.  SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Radu Bogdan Rusu,et al.  Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments , 2010, KI - Künstliche Intelligenz.

[7]  Lorenzo Porzi,et al.  Seamless Scene Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Anton Kummert,et al.  FLIC: Fast Lidar Image Clustering , 2020, ICPRAM.

[9]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[10]  You Li,et al.  InsClustering: Instantly Clustering LiDAR Range Measures for Autonomous Vehicle , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[11]  Seung-Woo Seo,et al.  Real-Time and Accurate Segmentation of 3-D Point Clouds Based on Gaussian Process Regression , 2017, IEEE Transactions on Intelligent Transportation Systems.

[12]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Xinge Zhu,et al.  LiDAR-based Panoptic Segmentation via Dynamic Shifting Network , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Thomas A. Funkhouser,et al.  Semantic Scene Completion from a Single Depth Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  A. Yuille,et al.  Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation , 2020, ECCV.

[16]  Thomas S. Huang,et al.  Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Dirk Wollherr,et al.  A clustering method for efficient segmentation of 3D laser data , 2008, 2008 IEEE International Conference on Robotics and Automation.

[18]  Xinge Zhu,et al.  Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation , 2020, ArXiv.

[19]  Xinge Zhu,et al.  Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Cyrill Stachniss,et al.  A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[21]  C. Stachniss,et al.  LiDAR Panoptic Segmentation for Autonomous Driving , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  You Li,et al.  Coarse-to-Fine Segmentation on LiDAR Point Clouds in Spherical Coordinate and Beyond , 2020, IEEE Transactions on Vehicular Technology.

[23]  Wolfram Burgard,et al.  EfficientLPS: Efficient LiDAR Panoptic Segmentation , 2021, IEEE Transactions on Robotics.

[24]  Yuxiang Sun,et al.  A Novel Point Cloud Compression Algorithm Based on Clustering , 2019, IEEE Robotics and Automation Letters.

[25]  Jiong Yang,et al.  PointPillars: Fast Encoders for Object Detection From Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Mohamed Boussaha,et al.  Point Cloud Oversegmentation With Graph-Structured Deep Metric Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Florentin Wörgötter,et al.  Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Carsten Rother,et al.  Panoptic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Anath Fischer,et al.  Graph Based Over-Segmentation Methods for 3D Point Clouds , 2017, Comput. Vis. Image Underst..

[30]  Zhigang Xu,et al.  Fusion of 3D LIDAR and Camera Data for Object Detection in Autonomous Vehicle Applications , 2020, IEEE Sensors Journal.

[31]  Nikolaos Papanikolopoulos,et al.  Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Cyrill Stachniss,et al.  RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Kenji Suzuki,et al.  A Run-Based Two-Scan Labeling Algorithm , 2008, IEEE Transactions on Image Processing.

[34]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[35]  Kesheng Wu,et al.  Optimizing two-pass connected-component labeling algorithms , 2009, Pattern Analysis and Applications.

[36]  Yuxiang Sun,et al.  A Novel Coding Scheme for Large-Scale Point Cloud Sequences Based on Clustering and Registration , 2022, IEEE Transactions on Automation Science and Engineering.

[37]  Cyrill Stachniss,et al.  Fast range image-based segmentation of sparse 3D laser scans for online operation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Martin Buss,et al.  Realtime segmentation of range data using continuous nearest neighbors , 2009, 2009 IEEE International Conference on Robotics and Automation.

[39]  PASS3D: Precise and Accelerated Semantic Segmentation for 3D Point Cloud , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Leonidas J. Guibas,et al.  KPConv: Flexible and Deformable Convolution for Point Clouds , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Silvio Savarese,et al.  A Probabilistic Framework for Real-time 3D Segmentation using Spatial, Temporal, and Semantic Cues , 2016, Robotics: Science and Systems.

[43]  Chunxia Zhao,et al.  Unsupervised Segmentation Of Urban 3D Point Cloud Based On Lidar-image , 2019, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[44]  Cyrill Stachniss,et al.  Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset , 2021, Int. J. Robotics Res..

[45]  Cyrill Stachniss,et al.  SuMa++: Efficient LiDAR-based Semantic SLAM , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).