Region-Based Edge Convolutions With Geometric Attributes for the Semantic Segmentation of Large-Scale 3-D Point Clouds

In this article, we present a semantic segmentation framework for large-scale 3-D point clouds with high spatial resolution. For such data with huge amounts of points, the classification of each individual 3-D point is an intractable task. Instead, we propose to segment the scene into meaningful regions as a first step. Afterward, we classify these segments using a combination of PointNet and geometric deep learning. This two-step approach resembles object-based image analysis. As an additional novelty, we apply surface normalization techniques and enrich features with geometric attributes. Our experiments show the potential of this approach for a variety of outdoor scene analysis tasks. In particular, we are able to reach 89.6% overall accuracy and 64.4% average intersection over union (IoU) in the Semantic3D benchmark. Furthermore, we achieve 66.7% average IoU on Paris-Lille-3D. We also successfully apply our approach to the automatic semantic analysis of forestry data.

[1]  Wuming Zhang,et al.  An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation , 2016, Remote. Sens..

[2]  Joachim Denzler,et al.  Semantic Segmentation of Outdoor Areas Using 3D Moment Invariants and Contextual Cues , 2017, GCPR.

[3]  Yin Zhou,et al.  VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Cewu Lu,et al.  PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation , 2018, ArXiv.

[5]  Ziad Abdeldayem,et al.  Automatic Weighted Splines Filter (AWSF): A New Algorithm for Extracting Terrain Measurements From Raw LiDAR Point Clouds , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  François Goulette,et al.  Semantic Classification of 3D Point Clouds with Multiscale Spherical Neighborhoods , 2018, 2018 International Conference on 3D Vision (3DV).

[7]  Michael Felsberg,et al.  Deep Projective 3D Semantic Segmentation , 2017, CAIP.

[8]  David Moloney,et al.  Classify 3D voxel based point-cloud using convolutional neural network on a neural compute stick , 2017, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[9]  Marc Pollefeys,et al.  Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark , 2017, ArXiv.

[10]  Leonidas J. Guibas,et al.  KPConv: Flexible and Deformable Convolution for Point Clouds , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Ying Li,et al.  TGNet: Geometric Graph CNN on 3-D Point Cloud Segmentation , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[12]  François Goulette,et al.  Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification , 2017, Int. J. Robotics Res..

[13]  H. Spiecker,et al.  Non Destructive Method for Biomass Prediction Combining TLS Derived Tree Volume and Wood Density , 2015 .

[14]  Loic Landrieu,et al.  WEAKLY SUPERVISED SEGMENTATION-AIDED CLASSIFICATION OF URBAN SCENES FROM 3D LIDAR POINT CLOUDS , 2017 .

[15]  Perpetual Hope Akwensi,et al.  An Individual Tree Segmentation Method Based on Watershed Algorithm and Three-Dimensional Spatial Distribution Analysis From Airborne LiDAR Point Clouds , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Li Liu,et al.  Deep Learning for 3D Point Clouds: A Survey , 2020, IEEE transactions on pattern analysis and machine intelligence.

[17]  Chunxiang Wang,et al.  Hierarchical Depthwise Graph Convolutional Neural Network for 3D Semantic Segmentation of Point Clouds , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[18]  Ying Li,et al.  Multi-Scale Point-Wise Convolutional Neural Networks for 3D Object Segmentation From LiDAR Point Clouds in Large-Scale Environments , 2021, IEEE Transactions on Intelligent Transportation Systems.

[19]  S. Reutebuch,et al.  Estimating forest canopy fuel parameters using LIDAR data , 2005 .

[20]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[21]  Yulan Guo,et al.  Global Context Reasoning for Semantic Segmentation of 3D Point Clouds , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[22]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Zhongliang Fu,et al.  MHNet: Multiscale Hierarchical Network for 3D Point Cloud Semantic Segmentation , 2019, IEEE Access.

[24]  Wei Wu,et al.  PointCNN: Convolution On X-Transformed Points , 2018, NeurIPS.

[25]  Jiamao Li,et al.  3D Recurrent Neural Networks with Context Fusion for Point Cloud Semantic Segmentation , 2018, ECCV.

[26]  Alexandre Boulch,et al.  Unstructured Point Cloud Semantic Labeling Using Deep Segmentation Networks , 2017, 3DOR@Eurographics.

[27]  Xiaohua Tong,et al.  Classification of LiDAR Point Clouds Using Supervoxel-Based Detrended Feature and Perception-Weighted Graphical Model , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[28]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  J. Trochta,et al.  3D Forest: An application for descriptions of three-dimensional forest structures using terrestrial LiDAR , 2017, PloS one.

[31]  Konrad Schindler,et al.  FAST SEMANTIC SEGMENTATION OF 3D POINT CLOUDS WITH STRONGLY VARYING DENSITY , 2016 .

[32]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Kurt Keutzer,et al.  SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[36]  Jun Xu,et al.  3D-GIoU: 3D Generalized Intersection over Union for Object Detection in Point Cloud , 2019, Sensors.

[37]  Xiao Xiang Zhu,et al.  A Review of Point Cloud Semantic Segmentation , 2019, ArXiv.

[38]  Federico Tombari,et al.  SHOT: Unique signatures of histograms for surface and texture description , 2014, Comput. Vis. Image Underst..

[39]  Silvio Savarese,et al.  SEGCloud: Semantic Segmentation of 3D Point Clouds , 2017, 2017 International Conference on 3D Vision (3DV).

[40]  François Goulette,et al.  Classification of Point Cloud Scenes with Multiscale Voxel Deep Network , 2018, ArXiv.

[41]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Chengcui Zhang,et al.  A progressive morphological filter for removing nonground measurements from airborne LIDAR data , 2003, IEEE Trans. Geosci. Remote. Sens..

[43]  Gernot Riegler,et al.  OctNet: Learning Deep 3D Representations at High Resolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Jan Dirk Wegner,et al.  Mind the Gap: Modeling Local and Global Context in (Road) Networks , 2014, GCPR.