Local and Global Structure for Urban ALS Point Cloud Semantic Segmentation with Ground-Aware Attention

Interpretation of airborne laser scanning (ALS) point clouds plays a notably role in geo-information production. As a critical step for interpretation, accurate semantic segmentation can considerably broaden various applications of ALS data. However, most existing methods cannot provide precise annotations and high robustness, owing to occlusions, varied point densities, and complex and incomplete object structures. Therefore, we developed a semantic segmentation framework focusing on ALS point clouds. The framework comprises contextual feature extraction from a local neighborhood, scene-aware global information representation, and ground-aware attention module. To verify its effectiveness, comprehensive experiments were conducted on three airborne light detection and ranging (LiDAR) datasets: DublinCity, Dayton Annotated LiDAR Earth Scan (DALES), and DFC2019 datasets. The experimental results demonstrate that the proposed method achieves better segmentation performance than that of some advanced methods. For the DublinCity dataset, our model’s overall accuracy (OA) can be improved to 67.5%, with an average F1(AvgF1) of 37.6%. For the DALES dataset, our method achieved an OA of 96.5% and a mean intersection over union (mIoU) of 77.6%. Our method also achieves a more accurate result on the DFC2019 dataset than that obtained using other models, with an OA of 94.8% and an AvgF1 of 81.4%.

[1]  Cheng Wang,et al.  A GCN-Based Method for Extracting Power Lines and Pylons From Airborne LiDAR Data , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Tengping Jiang,et al.  Hierarchical semantic segmentation of urban scene point clouds via group proposal and graph attention network , 2021, Int. J. Appl. Earth Obs. Geoinformation.

[3]  Qiulei Dong,et al.  GA-NET: Global Attention Network for Point Cloud Semantic Segmentation , 2021, IEEE Signal Processing Letters.

[4]  Rong Huang,et al.  GraNet: Global relation-aware attentional network for semantic segmentation of ALS point clouds , 2021, ISPRS Journal of Photogrammetry and Remote Sensing.

[5]  Xiaojing Yao,et al.  Airborne LiDAR point cloud classification with global-local graph attention convolution neural network , 2021 .

[6]  Elyta Widyaningrum,et al.  Airborne Laser Scanning Point Cloud Classification Using the DGCNN Deep Learning Method , 2021, Remote. Sens..

[7]  Qiang Wang,et al.  Hierarchical Aggregated Deep Features for ALS Point Cloud Classification , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Hongtao Wang,et al.  ALS Point Cloud Classification by Integrating an Improved Fully Convolutional Network into Transfer Learning with Multi-Scale and Multi-View Deep Features , 2020, Sensors.

[9]  Jing Liu,et al.  Hierarchical Instance Recognition of Individual Roadside Trees in Environmentally Complex Urban Areas from UAV Laser Scanning Point Clouds , 2020, ISPRS Int. J. Geo Inf..

[10]  Haitao Guo,et al.  ALS Point Cloud Classification With Small Training Data Set Based on Transfer Learning , 2020, IEEE Geoscience and Remote Sensing Letters.

[11]  Yanjun Su,et al.  A Point-Based Fully Convolutional Neural Network for Airborne LiDAR Ground Point Filtering in Forested Environments , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Min Yu,et al.  Semantic-Based Building Extraction from LiDAR Point Clouds Using Contexts and Optimization in Complex Environment , 2020, Sensors.

[13]  Gui-Song Xia,et al.  A geometry-attentional network for ALS point cloud classification , 2020 .

[14]  Bo Huang,et al.  Feature fusion network based on attention mechanism for 3D semantic segmentation of point clouds , 2020, Pattern Recognit. Lett..

[15]  Rong Huang,et al.  Deep point embedding for urban classification using ALS point clouds: A new perspective from local to global , 2020 .

[16]  Xiangyun Hu,et al.  Semantic Labeling of ALS Point Cloud via Learning Voxel and Pixel Representations , 2020, IEEE Geoscience and Remote Sensing Letters.

[17]  Haifeng Luo,et al.  MS-RRFSegNet: Multiscale Regional Relation Feature Segmentation Network for Semantic Segmentation of Urban Scene Point Clouds , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Nina M. Varney,et al.  DALES: A Large-scale Aerial LiDAR Data Set for Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Cheng Xu,et al.  Large-Scale ALS Data Semantic Classification Integrating Location-Context-Semantics Cues by Higher-Order CRF , 2020, Sensors.

[20]  Jie Zhou,et al.  SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor , 2020, IJCAI.

[21]  A. Markham,et al.  RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Congcong Wen,et al.  DANCE-NET: Density-aware convolution networks with context encoding for airborne LiDAR point cloud classification , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[23]  Xiang Li,et al.  Directionally Constrained Fully Convolutional Neural Network For Airborne Lidar Point Cloud Classification , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[24]  Alexandre Boulch ConvPoint: Continuous convolutions for point cloud processing , 2019, Comput. Graph..

[25]  Xiangyun Hu,et al.  A Graph-Voxel Joint Convolution Neural Network for ALS Point Cloud Segmentation , 2020, IEEE Access.

[26]  Naoto Yokoya,et al.  2019 IEEE GRSS Data Fusion Contest: Large-Scale Semantic 3D Reconstruction [Technical Committees] , 2019 .

[27]  Xuejin Chen,et al.  Ground-Aware Point Cloud Semantic Segmentation for Autonomous Driving , 2019, ACM Multimedia.

[28]  Xiaoqiang Liu,et al.  Hierarchical Classification of Urban ALS Data by Using Geometry and Intensity Information , 2019, Sensors.

[29]  S. M. Iman Zolanvari,et al.  DublinCity: Annotated LiDAR Point Cloud and its Applications , 2019, BMVC.

[30]  Binh-Son Hua,et al.  ShellNet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[31]  Lei Wang,et al.  Appendix for : Graph Attention Convolution for Point Cloud Semantic Segmentation , 2019 .

[32]  Chi-Wing Fu,et al.  PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Leonidas J. Guibas,et al.  KPConv: Flexible and Deformable Convolution for Point Clouds , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[34]  Cheng Wang,et al.  Learning high-level features by fusing multi-view representation of MLS point clouds for 3D object recognition in road environments , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[35]  Hasan Asy'ari Arief,et al.  Addressing Overfitting on Pointcloud Classification using Atrous XCRF , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[36]  Norbert Pfeifer,et al.  Improving LiDAR classification accuracy by contextual label smoothing in post-processing , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[37]  Fuxin Li,et al.  PointConv: Deep Convolutional Networks on 3D Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Vladlen Koltun,et al.  Tangent Convolutions for Dense Prediction in 3D , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Ruibin Zhao,et al.  Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network , 2018, Int. J. Geogr. Inf. Sci..

[40]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Wei Huang,et al.  A Convolutional Neural Network-Based 3D Semantic Labeling Method for ALS Point Clouds , 2017, Remote. Sens..

[43]  Qing Zhu,et al.  Robust point cloud classification based on multi-level semantic relationships for urban scenes , 2017 .

[44]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[45]  Bisheng Yang,et al.  Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data , 2017 .

[46]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Dushyant Rao,et al.  Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Bisheng Yang,et al.  Two-step adaptive extraction method for ground points and breaklines from lidar point clouds , 2016 .

[49]  Yanjun Su,et al.  Improved progressive TIN densification filtering algorithm for airborne LiDAR data in forested areas , 2016 .

[50]  Wuming Zhang,et al.  An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation , 2016, Remote. Sens..

[51]  Zhen Wang,et al.  A Multilevel Point-Cluster-Based Discriminative Feature for ALS Point Cloud Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Stefan Hinz,et al.  Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers , 2015 .

[53]  Cheng Wang,et al.  Automated Extraction of Urban Road Facilities Using Mobile Laser Scanning Data , 2015, IEEE Transactions on Intelligent Transportation Systems.

[54]  S. J. Oude Elberink,et al.  Multiple-entity based classification of airborne laser scanning data in urban areas , 2014 .

[55]  J. Niemeyer,et al.  Contextual classification of lidar data and building object detection in urban areas , 2014 .

[56]  Xiangguo Lin,et al.  SVM-Based Classification of Segmented Airborne LiDAR Point Clouds in Urban Areas , 2013, Remote. Sens..