Knowledge Enhanced Neural Networks for Point Cloud Semantic Segmentation

Deep learning approaches have sparked much interest in the AI community during the last decade, becoming state-of-the-art in domains such as pattern recognition, computer vision, and data analysis. However, these methods are highly demanding in terms of training data, which is often a major issue in the geospatial and remote sensing fields. One possible solution to this problem comes from the Neuro-Symbolic Integration field (NeSy), where multiple methods have been defined to incorporate background knowledge into the neural network’s learning pipeline. One such method is KENN (Knowledge Enhanced Neural Networks), which injects logical knowledge into the neural network’s structure through additional final layers. Empirically, KENN showed comparable or better results than other NeSy frameworks in various tasks while being more scalable. Therefore, we propose the usage of KENN for point cloud semantic segmentation tasks, where it has immense potential to resolve issues with small sample sizes and unbalanced classes. While other works enforce the knowledge constraints in post-processing, to the best of our knowledge, no previous methods have injected inject such knowledge into the learning pipeline through the use of a NeSy framework. The experiment results over different datasets demonstrate that the introduction of knowledge rules enhances the performance of the original network and achieves state-of-the-art levels of accuracy, even with subideal training data.

[1]  Emile van Krieken,et al.  Refining neural network predictions using background knowledge , 2022, Machine Learning.

[2]  R. Pierdicca,et al.  GeoAI: a review of artificial intelligence approaches for the interpretation of complex geomatics data , 2022, Geoscientific Instrumentation, Methods and Data Systems.

[3]  M. Kada,et al.  ALS POINT CLOUD CLASSIFICATION USING POINTNET++ AND KPCONV WITH PRIOR KNOWLEDGE , 2021, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[4]  Alessandro Golkar,et al.  An Efficient and General Framework for Aerial Point Cloud Classification in Urban Scenarios , 2021, Remote. Sens..

[5]  Frank Boochs,et al.  Object Semantic Segmentation in Point Clouds - Comparison of a Deep Learning and a Knowledge-Based Method , 2021, ISPRS Int. J. Geo Inf..

[6]  J. D. Wegner,et al.  The Hessigheim 3D (H3D) Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Textured Meshes from UAV LiDAR and Multi-View-Stereo , 2021, ArXiv.

[7]  Luciano Serafini,et al.  Logic Tensor Networks , 2020, Artif. Intell..

[8]  Maarten Vergauwen,et al.  Unsupervised reconstruction of Building Information Modeling wall objects from point cloud data , 2020 .

[9]  Roberto Pierdicca,et al.  Comparing Machine and Deep Learning Methods for Large 3D Heritage Semantic Segmentation , 2020, ISPRS Int. J. Geo Inf..

[10]  Zhen Ye,et al.  LASDU: A Large-Scale Aerial LiDAR Dataset for Semantic Labeling in Dense Urban Areas , 2020, ISPRS Int. J. Geo Inf..

[11]  Maarten Vergauwen,et al.  Point Cloud vs. Mesh Features for Building Interior Classification , 2020, Remote. Sens..

[12]  Frank van Harmelen,et al.  Analyzing Differentiable Fuzzy Logic Operators , 2020, Artif. Intell..

[13]  Hsien-I Lin,et al.  Boosting Minority Class Prediction on Imbalanced Point Cloud Data , 2020, Applied Sciences.

[14]  Saifullahi Aminu Bello,et al.  Review: deep learning on 3D point clouds , 2020, Remote. Sens..

[15]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[16]  Frank Boochs,et al.  Automatic Detection of Objects in 3D Point Clouds Based on Exclusively Semantic Guided Processes , 2019, ISPRS Int. J. Geo Inf..

[17]  Xiao Xiang Zhu,et al.  A Review of Point Cloud Semantic Segmentation , 2019, ArXiv.

[18]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[19]  Richard Evans,et al.  Learning Explanatory Rules from Noisy Data , 2017, J. Artif. Intell. Res..

[20]  Michael Weinmann,et al.  A Hybrid Semantic Point Cloud Classification-Segmentation Framework Based on Geometric Features and Semantic Rules , 2017, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science.

[21]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[22]  Marco Gori,et al.  Semantic-based regularization for learning and inference , 2017, Artif. Intell..

[23]  Leonidas J. Guibas,et al.  A scalable active framework for region annotation in 3D shape collections , 2016, ACM Trans. Graph..

[24]  Stefan Hinz,et al.  Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers , 2015 .

[25]  Artur S. d'Avila Garcez,et al.  The Connectionist Inductive Learning and Logic Programming System , 1999, Applied Intelligence.

[26]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[27]  Q. Xia,et al.  Classification method for imbalanced LiDAR point cloud based on stack autoencoder , 2023, Electronic Research Archive.

[28]  Florent Poux,et al.  UNSUPERVISED OBJECT-BASED CLUSTERING IN SUPPORT OF SUPERVISED POINT-BASED 3D POINT CLOUD CLASSIFICATION , 2021, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[29]  J. Niemeyer,et al.  Contextual classification of lidar data and building object detection in urban areas , 2014 .