Large-scale urban point cloud labeling and reconstruction

Abstract The large number of object categories and many overlapping or closely neighboring objects in large-scale urban scenes pose great challenges in point cloud classification. In this paper, a novel framework is proposed for classification and reconstruction of airborne laser scanning point cloud data. To label point clouds, we present a rectified linear units neural network named ReLu-NN where the rectified linear units (ReLu) instead of the traditional sigmoid are taken as the activation function in order to speed up the convergence. Since the features of the point cloud are sparse, we reduce the number of neurons by the dropout to avoid over-fitting of the training process. The set of feature descriptors for each 3D point is encoded through self-taught learning, and forms a discriminative feature representation which is taken as the input of the ReLu-NN. The segmented building points are consolidated through an edge-aware point set resampling algorithm, and then they are reconstructed into 3D lightweight models using the 2.5D contouring method ( Zhou and Neumann, 2010 ). Compared with deep learning approaches, the ReLu-NN introduced can easily classify unorganized point clouds without rasterizing the data, and it does not need a large number of training samples. Most of the parameters in the network are learned, and thus the intensive parameter tuning cost is significantly reduced. Experimental results on various datasets demonstrate that the proposed framework achieves better performance than other related algorithms in terms of classification accuracy and reconstruction quality.

[1]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[2]  Hao Zhang,et al.  SmartBoxes for interactive urban reconstruction , 2010, SIGGRAPH 2010.

[3]  Camillo Ressl,et al.  Digital Photogrammetric Camera Evaluation - Generation of Di- gital Elevation Models , 2010 .

[4]  Daniel Cohen-Or,et al.  Edge-aware point set resampling , 2013, ACM Trans. Graph..

[5]  Zhen Wang,et al.  A Multiscale and Hierarchical Feature Extraction Method for Terrestrial Laser Scanning Point Cloud Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Sagi Filin,et al.  Segmentation of terrestrial laser scanning data using geometry and image information , 2013 .

[7]  Hui Lin,et al.  Semantic decomposition and reconstruction of residential scenes from LiDAR data , 2013, ACM Trans. Graph..

[8]  Norbert Pfeifer,et al.  A Comprehensive Automated 3D Approach for Building Extraction, Reconstruction, and Regularization from Airborne Laser Scanning Point Clouds , 2008, Sensors.

[9]  Marc Pollefeys,et al.  Dense Semantic 3D Reconstruction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  S. J. Oude Elberink,et al.  Multiple-entity based classification of airborne laser scanning data in urban areas , 2014 .

[12]  Zhen Wang,et al.  A Multilevel Point-Cluster-Based Discriminative Feature for ALS Point Cloud Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Jie Shan,et al.  Segmentation and Reconstruction of Polyhedral Building Roofs From Aerial Lidar Point Clouds , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Jing Xiao,et al.  Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification , 2014 .

[15]  G. Vosselman Point cloud segmentation for urban scene classification , 2013 .

[16]  Xiaoxiang Zhu,et al.  Automatic Detection and Reconstruction of 2-D/3-D Building Shapes From Spaceborne TomoSAR Point Clouds , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Christian Früh,et al.  Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images , 2005, International Journal of Computer Vision.

[18]  Bo Du,et al.  A Three-Step Approach for TLS Point Cloud Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[19]  George Vosselman,et al.  Knowledge based reconstruction of building models from terrestrial laser scanning data , 2009 .

[20]  A. Vetrivel,et al.  Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high resolution oblique aerial images, and multiple-kernel-learning , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[21]  Xianfeng Huang,et al.  A Methodology for Automated Segmentation and Reconstruction of Urban 3-D Buildings from ALS Point Clouds , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  Bo Guo,et al.  Discriminative-Dictionary-Learning-Based Multilevel Point-Cluster Features for ALS Point-Cloud Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Primo Zingaretti,et al.  Complete classification of raw LIDAR data and 3D reconstruction of buildings , 2006, Pattern Analysis and Applications.

[24]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[25]  Jiann-Yeou Rau,et al.  Analysis of Oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Hao Liu,et al.  A hierarchical methodology for urban facade parsing from TLS point clouds , 2017 .

[27]  Dimitri Lague,et al.  3D Terrestrial LiDAR data classification of complex natural scenes using a multi-scale dimensionality criterion: applications in geomorphology , 2011, ArXiv.

[28]  Danil V. Prokhorov,et al.  A Convolutional Learning System for Object Classification in 3-D Lidar Data , 2010, IEEE Transactions on Neural Networks.

[29]  Wojciech Matusik,et al.  Capture and modeling of non-linear heterogeneous soft tissue , 2009, ACM Trans. Graph..

[30]  Ruofei Zhong,et al.  Classification of Urban Point Clouds: A Robust Supervised Approach With Automatically Generating Training Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Qi Zhang,et al.  Deep learning-based tree classification using mobile LiDAR data , 2015 .

[33]  Andrzej Borkowski,et al.  3D building reconstruction from ALS data using unambiguous decomposition into elementary structures , 2016 .

[34]  Qingming Huang,et al.  Multi-Level Discriminative Dictionary Learning With Application to Large Scale Image Classification , 2015, IEEE Transactions on Image Processing.

[35]  Daniel G. Aliaga,et al.  A Survey of Urban Reconstruction , 2013, Comput. Graph. Forum.