A multi-faceted CNN architecture for automatic classification of mobile LiDAR data and an algorithm to reproduce point cloud samples for enhanced training

Abstract Mobile Laser Scanning (MLS) data of outdoor environment are typically characterised by occlusion, noise, clutter, large data size and high quantum of information which makes their classification a challenging problem. This paper presents three deep Convolutional Neural Network (CNN) architectures in three dimension (3D), namely single CNN (SCN), multi-faceted CNN (MFC) and MFC with reproduction (MFCR) for automatic classification of MLS data. The MFC uses multiple facets of an MLS sample as inputs to different SCNs, thus providing additional information during classification. The MFC, once trained, is used to reproduce additional samples with the help of existing samples. The reproduced samples are employed to further refine the MFC training parameters, thus giving a new method called MFCR. The three architectures are evaluated on an ensemble of 3D outdoor MLS data consisting of four classes, i.e. tree, pole, house and ground covered with low vegetation along with car samples from KITTI dataset. The total accuracy and kappa values of classifications reached up to (i) 86.0% and 81.3% for the SCN (ii) 94.3% and 92.4% for the MFC and (iii) 96.0% and 94.6% for the MFCR, respectively. The paper has demonstrated the use of multiple facets to significantly improve classification accuracy over the SCN. Finally, a unique approach has been developed for reproduction of samples which has shown potential to improve the accuracy of classification. Unlike previous works on the use of CNN for structured point cloud of indoor objects, this work shows the utility of different proposed CNN architectures for classification of varieties of outdoor objects, viz., tree, pole, house and ground which are captured as unstructured point cloud by MLS.

[1]  N. Haala,et al.  Mobile LiDAR mapping for 3D point cloud collecation in urban areas : a performance test , 2008 .

[2]  Bahman Soheilian,et al.  Road side detection and reconstruction using LIDAR sensor , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[3]  Ulrich Neumann,et al.  Fast and Robust Multi-view 3D Object Recognition in Point Clouds , 2015, 2015 International Conference on 3D Vision.

[4]  Jing Huang,et al.  Point cloud labeling using 3D Convolutional Neural Network , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[5]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[6]  Ajai Kumar Singh,et al.  Identification of pole-like structures from mobile lidar data of complex road environment , 2016 .

[7]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  David A. Clausi,et al.  Sea Ice Concentration Estimation During Melt From Dual-Pol SAR Scenes Using Deep Convolutional Neural Networks: A Case Study , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Reza Bosagh Zadeh,et al.  FusionNet: 3D Object Classification Using Multiple Data Representations , 2016, ArXiv.

[10]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[11]  Aboelmagd Noureldin,et al.  Integrated Indoor Navigation System for Ground Vehicles With Automatic 3-D Alignment and Position Initialization , 2015, IEEE Transactions on Vehicular Technology.

[12]  Sebastian Scherer,et al.  3D Convolutional Neural Networks for landing zone detection from LiDAR , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Bisheng Yang,et al.  A shape-based segmentation method for mobile laser scanning point clouds , 2013 .

[15]  Juha Hyyppä,et al.  Individual tree biomass estimation using terrestrial laser scanning , 2013 .

[16]  Stefan Hinz,et al.  CONTEXTUAL CLASSIFICATION OF POINT CLOUD DATA BY EXPLOITING INDIVIDUAL 3D NEIGBOURHOODS , 2015 .

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  Bisheng Yang,et al.  Hierarchical extraction of urban objects from mobile laser scanning data , 2015 .

[19]  Alexandre Boulch,et al.  SnapNet-R: Consistent 3D Multi-view Semantic Labeling for Robotics , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[20]  Huanxin Zou,et al.  Deep Convolutional Highway Unit Network for SAR Target Classification With Limited Labeled Training Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[21]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.