User exemplar-based building element retrieval from raw point clouds using deep point-level features

Abstract 3D point cloud data can be utilized for site inspection and reverse engineering of building models. However, conventional methods for building element retrieval require a database of 3D CAD or BIM models which are unsuitable for the case of historical buildings without as-planned models or temporary structures that are not in the pre-built model. Thus, this paper proposes a semi-automated method to efficiently retrieve duplicate building elements without these constraints. First, the point cloud is processed with a pre-trained deep feature extractor to generate a 50-dimensional feature vector for each point. Next, the point cloud is segmented through feature clustering and region-growing algorithms, then displayed on a user interface for selection. Lastly, the selected exemplar is provided as input to a peak-finding algorithm to determine positive matches. Experimental results on five different datasets show that the proposed method obtains average rates above 90% for precision and recall.

[1]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Changmin Kim,et al.  Skeleton-based 3D reconstruction of as-built pipelines from laser-scan data , 2013 .

[3]  Jie Gong,et al.  Data processing for real-time construction site spatial modeling , 2008 .

[4]  Frédéric Bosché,et al.  Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction , 2010, Adv. Eng. Informatics.

[5]  Jack Chin Pang Cheng,et al.  An integrated underground utility management and decision support based on BIM and GIS , 2019, Automation in Construction.

[6]  Jie Wu,et al.  Dimensional accuracy and structural performance assessment of spatial structure components using 3D laser scanning , 2018, Automation in Construction.

[7]  Soon-Wook Kwon,et al.  Comparison and utilization of point cloud generated from photogrammetry and laser scanning: 3D world model for smart heavy equipment planning , 2019, Automation in Construction.

[8]  J. J. McArthur,et al.  Automating the creation of building information models for existing buildings , 2019, Automation in Construction.

[9]  Jingdao Chen,et al.  Exemplar-Based Building Element Retrieval from Point Clouds , 2019 .

[10]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jurģis Zemītis,et al.  3D Scanning Data Use for Modular Building Renovation based on BIM Model , 2018 .

[12]  Jingdao Chen,et al.  Unsupervised Recognition of Volumetric Structural Components from Building Point Clouds , 2017 .

[13]  Yong K. Cho,et al.  Region Proposal Mechanism for Building Element Recognition for Advanced Scan-to-BIM Process , 2018 .

[14]  Jingdao Chen,et al.  Multi-View Incremental Segmentation of 3-D Point Clouds for Mobile Robots , 2019, IEEE Robotics and Automation Letters.

[15]  Jingdao Chen,et al.  Principal Axes Descriptor for Automated Construction-Equipment Classification from Point Clouds , 2017, J. Comput. Civ. Eng..

[16]  George Vosselman,et al.  Knowledge based reconstruction of building models from terrestrial laser scanning data , 2009 .

[17]  Nikolaos D. Doulamis,et al.  Evaluation of relevance feedback schemes in content-based in retrieval systems , 2006, Signal Process. Image Commun..

[18]  Pingbo Tang,et al.  Computationally efficient change analysis of piece-wise cylindrical building elements for proactive project control , 2017 .

[19]  Fabio Remondino,et al.  Image‐based 3D Modelling: A Review , 2006 .

[20]  Cyrill Stachniss,et al.  Automated 3D volumetric reconstruction of multiple-room building interiors for as-built BIM , 2018, Adv. Eng. Informatics.

[21]  Jie Gong,et al.  Community-scale multi-level post-hurricane damage assessment of residential buildings using multi-temporal airborne LiDAR data , 2019, Automation in Construction.

[22]  C. Fraser,et al.  Digital camera calibration methods: Considerations and comparisons , 2006 .

[23]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[24]  Hoon Sohn,et al.  Automated Estimation of Reinforced Precast Concrete Rebar Positions Using Colored Laser Scan Data , 2017, Comput. Aided Civ. Infrastructure Eng..

[25]  Markus Vincze,et al.  Ensemble of shape functions for 3D object classification , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[26]  Changmin Kim,et al.  Automated construction progress measurement using a 4D building information model and 3D data , 2013 .

[27]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Cristina Portalés,et al.  Digital Cultural Heritage , 2018, Lecture Notes in Computer Science.

[29]  George Athanasiou,et al.  3D reconstruction of disaster scenes for urban search and rescue , 2018, Multimedia Tools and Applications.

[30]  Jingdao Chen,et al.  Performance evaluation of 3D descriptors for object recognition in construction applications , 2018 .

[31]  Jingdao Chen,et al.  Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction , 2019, J. Comput. Civ. Eng..

[32]  Ulrich Neumann,et al.  SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Mani Golparvar-Fard,et al.  Segmentation of building point cloud models including detailed architectural/structural features and MEP systems , 2015 .

[34]  Gerard Salton,et al.  Improving Retrieval Performance by Relevance Feedback , 1997 .

[35]  Samy Bengio,et al.  Large Scale Online Learning of Image Similarity through Ranking , 2009, IbPRIA.

[36]  Carl T. Haas,et al.  Pipe spool recognition in cluttered point clouds using a curvature-based shape descriptor , 2016 .

[37]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[38]  Jie Gong,et al.  Automated Analysis of Mobile LiDAR Data for Component‐Level Damage Assessment of Building Structures during Large Coastal Storm Events , 2018, Comput. Aided Civ. Infrastructure Eng..

[39]  Higinio González-Jorge,et al.  4-Plane congruent sets for automatic registration of as-is 3D point clouds with 3D BIM models , 2018 .

[40]  Burcu Akinci,et al.  Automatic Creation of Semantically Rich 3D Building Models from Laser Scanner Data , 2013 .

[41]  Qian Wang,et al.  Applications of 3D point cloud data in the construction industry: A fifteen-year review from 2004 to 2018 , 2019, Adv. Eng. Informatics.

[42]  Marinos Ioannides,et al.  In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction , 2014, Multimedia Tools and Applications.

[43]  Marc Pollefeys,et al.  Multi-Label Semantic 3D Reconstruction Using Voxel Blocks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[44]  Allison Kealy,et al.  Shape Grammar Approach to 3D Modeling of Indoor Environments Using Point Clouds , 2019, J. Comput. Civ. Eng..

[45]  Chao Wang,et al.  Automatic BIM component extraction from point clouds of existing buildings for sustainability applications , 2015 .

[46]  Hoon Sohn,et al.  A mirror-aided laser scanning system for geometric quality inspection of side surfaces of precast concrete elements , 2019, Measurement.

[47]  Xie Han,et al.  Point cloud simplification with preserved edge based on normal vector , 2015 .

[48]  J. Pfanzagl,et al.  Studies in the history of probability and statistics XLIV A forerunner of the t-distribution , 1996 .

[49]  Stephen Fai,et al.  Building Information Modeling for Cultural Heritage: The Management of Generative Process for Complex Historical Buildings , 2017, ITN-DCH.

[50]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[51]  David Arditi,et al.  AUTOMATED PROGRESS CONTROL USING LASER SCANNING TECHNOLOGY , 2013 .

[52]  Frédéric Bosché,et al.  Tracking of secondary and temporary objects in structural concrete work , 2014 .

[53]  Danijel Rebolj,et al.  Point cloud quality requirements for Scan-vs-BIM based automated construction progress monitoring , 2017 .

[54]  Carl T. Haas,et al.  Rapid Geometric Modeling for Unstructured Construction Workspaces , 2003 .

[55]  Burcu Akinci,et al.  Automated Recognition of 3D CAD Objects in Site Laser Scans for Project 3D Status Visualization and Performance Control , 2009 .

[56]  S. V. Sreenivasan,et al.  A framework for rapid local area modeling for construction automation , 2002 .