Airborne Laser Scanning Point Cloud Classification Using the DGCNN Deep Learning Method

Classification of aerial point clouds with high accuracy is significant for many geographical applications, but not trivial as the data are massive and unstructured. In recent years, deep learning for 3D point cloud classification has been actively developed and applied, but notably for indoor scenes. In this study, we implement the point-wise deep learning method Dynamic Graph Convolutional Neural Network (DGCNN) and extend its classification application from indoor scenes to airborne point clouds. This study proposes an approach to provide cheap training samples for point-wise deep learning using an existing 2D base map. Furthermore, essential features and spatial contexts to effectively classify airborne point clouds colored by an orthophoto are also investigated, in particularly to deal with class imbalance and relief displacement in urban areas. Two airborne point cloud datasets of different areas are used: Area-1 (city of Surabaya—Indonesia) and Area-2 (cities of Utrecht and Delft—the Netherlands). Area-1 is used to investigate different input feature combinations and loss functions. The point-wise classification for four classes achieves a remarkable result with 91.8% overall accuracy when using the full combination of spectral color and LiDAR features. For Area-2, different block size settings (30, 50, and 70 m) are investigated. It is found that using an appropriate block size of, in this case, 50 m helps to improve the classification until 93% overall accuracy but does not necessarily ensure better classification results for each class. Based on the experiments on both areas, we conclude that using DGCNN with proper settings is able to provide results close to production.

[1]  James P. Horwath,et al.  Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images , 2020, npj Computational Materials.

[2]  Belén Riveiro,et al.  POINTNET FOR THE AUTOMATIC CLASSIFICATION OF AERIAL POINT CLOUDS , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[3]  Ryosuke Nakamura,et al.  3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration) , 2018, SIGSPATIAL/GIS.

[4]  Petteri Packalen,et al.  Utility of image point cloud data towards generating enhanced multitemporal multisensor land cover maps , 2020, Int. J. Appl. Earth Obs. Geoinformation.

[5]  Alaa Tharwat,et al.  Classification assessment methods , 2020, Applied Computing and Informatics.

[6]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Shi Feng,et al.  Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation , 2019, ICML.

[8]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[9]  Shulin Wang,et al.  Feature selection in machine learning: A new perspective , 2018, Neurocomputing.

[10]  Jan Dirk Wegner,et al.  Large-Scale Semantic 3D Reconstruction: An Adaptive Multi-resolution Model for Multi-class Volumetric Labeling , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  S. Schmohl,et al.  SUBMANIFOLD SPARSE CONVOLUTIONAL NETWORKS FOR SEMANTIC SEGMENTATION OF LARGE-SCALE ALS POINT CLOUDS , 2019 .

[12]  Alfredo Petrosino,et al.  Adjusted F-measure and kernel scaling for imbalanced data learning , 2014, Inf. Sci..

[13]  Weimin Wang,et al.  A Point-Wise LiDAR and Image Multimodal Fusion Network (PMNet) for Aerial Point Cloud 3D Semantic Segmentation , 2019, Remote. Sens..

[14]  Jan Boehm,et al.  A review on deep learning techniques for 3D sensed data classification , 2019, Remote. Sens..

[15]  Pascual Campoy Cervera,et al.  A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles , 2017, J. Sensors.

[16]  Joaquín Martínez-Sánchez,et al.  Road Environment Semantic Segmentation with Deep Learning from MLS Point Cloud Data , 2019, Sensors.

[17]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[18]  C. Qi Deep Learning on Point Sets for 3 D Classification and Segmentation , 2016 .

[19]  Bastian Leibe,et al.  Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[20]  Sebastian Raschka,et al.  An Overview of General Performance Metrics of Binary Classifier Systems , 2014, ArXiv.

[21]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .

[22]  Norbert Pfeifer,et al.  Classification of ALS Point Clouds Using End-to-End Deep Learning , 2019, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science.

[23]  David Masko,et al.  The Impact of Imbalanced Training Data for Convolutional Neural Networks , 2015 .

[24]  Anh Nguyen,et al.  3D point cloud segmentation: A survey , 2013, 2013 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[25]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Md. Zakirul Alam Bhuiyan,et al.  A Survey on Deep Learning in Big Data , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[27]  Franz Rottensteiner,et al.  Building and Road Extraction by LiDAR and Imagery , 2017 .

[28]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Taghi M. Khoshgoftaar,et al.  Survey on deep learning with class imbalance , 2019, J. Big Data.

[30]  Rong Huang,et al.  Deep point embedding for urban classification using ALS point clouds: A new perspective from local to global , 2020 .

[31]  Yaping Lin,et al.  Using Training Samples Retrieved from a Topographic Map and Unsupervised Segmentation for the Classification of Airborne Laser Scanning Data , 2020, Remote. Sens..

[32]  T. B. Alakus,et al.  Comparison of deep learning approaches to predict COVID-19 infection , 2020, Chaos, Solitons & Fractals.

[33]  Wisnu Jatmiko,et al.  Semantic Segmentation on LiDAR Point Cloud in Urban Area using Deep Learning , 2019, 2019 International Workshop on Big Data and Information Security (IWBIS).

[34]  Sarah Filippi,et al.  Interpreting Deep Neural Networks Through Variable Importance , 2019, ArXiv.

[35]  Xiao Xiang Zhu,et al.  A Review of Point Cloud Semantic Segmentation , 2019 .

[36]  Juntao Yang,et al.  A probabilistic graphical model for the classification of mobile LiDAR point clouds , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[37]  Lei Zhang,et al.  A deep manifold learning approach for spatial-spectral classification with limited labeled training samples , 2019, Neurocomputing.

[38]  Wei Wu,et al.  PointCNN: Convolution On X-Transformed Points , 2018, NeurIPS.