Three-Dimensional Reconstruction of Structural Surface Model of Heritage Bridges Using UAV-Based Photogrammetric Point Clouds

Three-dimensional (3D) digital technology is essential to the maintenance and monitoring of cultural heritage sites. In the field of bridge engineering, 3D models generated from point clouds of existing bridges is drawing increasing attention. Currently, the widespread use of the unmanned aerial vehicle (UAV) provides a practical solution for generating 3D point clouds as well as models, which can drastically reduce the manual effort and cost involved. In this study, we present a semi-automated framework for generating structural surface models of heritage bridges. To be specific, we propose to tackle this challenge via a novel top-down method for segmenting main bridge components, combined with rule-based classification, to produce labeled 3D models from UAV photogrammetric point clouds. The point clouds of the heritage bridge are generated from the captured UAV images through the structure-from-motion workflow. A segmentation method is developed based on the supervoxel structure and global graph optimization, which can effectively separate bridge components based on geometric features. Then, recognition by the use of a classification tree and bridge geometry is utilized to recognize different structural elements from the obtained segments. Finally, surface modeling is conducted to generate surface models of the recognized elements. Experiments using two bridges in China demonstrate the potential of the presented structural model reconstruction method using UAV photogrammetry and point cloud processing in 3D digital documentation of heritage bridges. By using given markers, the reconstruction error of point clouds can be as small as 0.4%. Moreover, the precision and recall of segmentation results using testing date are better than 0.8, and a recognition accuracy better than 0.8 is achieved.

[1]  Marko Kohv,et al.  Assessing multitemporal water‐level changes with uav‐based photogrammetry , 2017 .

[2]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[3]  Cyrill Stachniss,et al.  UAV-based crop and weed classification for smart farming , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Ayman Habib,et al.  Automated Relative Orientation of UAV-Based Imagery in the Presence of Prior Information for the Flight Trajectory , 2016 .

[5]  Siyuan Chen,et al.  UAV Bridge Inspection through Evaluated 3D Reconstructions , 2019, Journal of Bridge Engineering.

[6]  Melba M. Crawford,et al.  Improving Orthorectification of UAV-Based Push-Broom Scanner Imagery Using Derived Orthophotos From Frame Cameras , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[7]  David Zhang,et al.  A survey of graph theoretical approaches to image segmentation , 2013, Pattern Recognit..

[8]  Tian Zhou,et al.  Automated Aerial Triangulation for UAV-Based Mapping , 2018, Remote. Sens..

[9]  I. Colomina,et al.  Unmanned aerial systems for photogrammetry and remote sensing: A review , 2014 .

[10]  Ali Khaloo,et al.  Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling , 2018 .

[11]  Ramesh C. Jain,et al.  Segmentation through Variable-Order Surface Fitting , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Uwe Stilla,et al.  A Voxel- and Graph-Based Strategy for Segmenting Man-Made Infrastructures Using Perceptual Grouping Laws: Comparison and Evaluation , 2018, Photogrammetric Engineering and Remote Sensing.

[13]  Uwe Stilla,et al.  CLASSIFICATION OF MLS POINT CLOUDS IN URBAN SCENES USING DETRENDED GEOMETRIC FEATURES FROM SUPERVOXEL-BASED LOCAL CONTEXTS , 2018, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[14]  Uwe Stilla,et al.  Automatic vehicle extraction from airborne LiDAR data of urban areas aided by geodesic morphology , 2010, Pattern Recognit. Lett..

[15]  Anshuman Bhardwaj,et al.  UAVs as remote sensing platform in glaciology: Present applications and future prospects , 2016 .

[16]  Thomas A. Funkhouser,et al.  Min-cut based segmentation of point clouds , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[17]  Yusheng Xu,et al.  Geometric Primitive Extraction From Point Clouds of Construction Sites Using VGS , 2017, IEEE Geoscience and Remote Sensing Letters.

[18]  Harith T. Al-Jumaily,et al.  Urban Point Cloud Mining Based on Density Clustering and MapReduce , 2017, J. Comput. Civ. Eng..

[19]  Wanshou Jiang,et al.  Efficient SfM for Oblique UAV Images: From Match Pair Selection to Geometrical Verification , 2018, Remote. Sens..

[20]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Pierre Grussenmeyer,et al.  Documentation of heritage buildings using close‐range UAV images: dense matching issues, comparison and case studies , 2017 .

[22]  T. Rabbani,et al.  SEGMENTATION OF POINT CLOUDS USING SMOOTHNESS CONSTRAINT , 2006 .

[23]  George Vosselman,et al.  Evaluating the Societal Impact of Using Drones to Support Urban Upgrading Projects , 2018, ISPRS Int. J. Geo Inf..

[24]  Xiaohua Tong,et al.  A Linear Feature-Based Approach for the Registration of Unmanned Aerial Vehicle Remotely-Sensed Images and Airborne LiDAR Data , 2016, Remote. Sens..

[25]  Yusheng Xu,et al.  Segmentation of building roofs from airborne LiDAR point clouds using robust voxel-based region growing , 2017 .

[26]  Xiaofei He,et al.  Real-Time Object Tracking on a Drone With Multi-Inertial Sensing Data , 2018, IEEE Transactions on Intelligent Transportation Systems.

[27]  Michela Bertolotto,et al.  Octree-based region growing for point cloud segmentation , 2015 .

[28]  Guido Morgenthal,et al.  Framework for automated UAS-based structural condition assessment of bridges , 2019, Automation in Construction.

[29]  Li Li,et al.  PAIRWISE LINKAGE FOR POINT CLOUD SEGMENTATION , 2016 .

[30]  S. M. Jong,et al.  Mapping landslide displacements using Structure from Motion (SfM) and image correlation of multi-temporal UAV photography , 2014 .

[31]  N. Pfeifer,et al.  SEGMENTATION BASED ROBUST INTERPOLATION - A NEW APPROACH TO LASER DATA FILTERING , 2005 .

[32]  Bisheng Yang,et al.  Hierarchical extraction of urban objects from mobile laser scanning data , 2015 .

[33]  Chengcheng Guo,et al.  Illumination-Robust Subpixel Fourier-Based Image Correlation Methods Based on Phase Congruency , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Arko Lucieer,et al.  The Impact of the Calibration Method on the Accuracy of Point Clouds Derived Using Unmanned Aerial Vehicle Multi-View Stereopsis , 2015, Remote. Sens..

[35]  Fabio Remondino,et al.  A REVIEW OFPOINT CLOUDS SEGMENTATION AND CLASSIFICATION ALGORITHMS , 2017 .

[36]  Vincent G. Ambrosia,et al.  Unmanned Aircraft Systems in Remote Sensing and Scientific Research: Classification and Considerations of Use , 2012, Remote. Sens..

[37]  Stefan Hinz,et al.  Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers , 2015 .

[38]  Florentin Wörgötter,et al.  Object Partitioning Using Local Convexity , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Florentin Wörgötter,et al.  Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Yusheng Xu,et al.  Unsupervised Segmentation of Point Clouds From Buildings Using Hierarchical Clustering Based on Gestalt Principles , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[41]  Zoltan-Csaba Marton,et al.  Tutorial: Point Cloud Library: Three-Dimensional Object Recognition and 6 DOF Pose Estimation , 2012, IEEE Robotics & Automation Magazine.

[42]  Carlos Castillo,et al.  Image-based surface reconstruction in geomorphometry - merits, limits and developments , 2016 .

[43]  David H. Roberts,et al.  Operational Framework for Rapid, Very-high Resolution Mapping of Glacial Geomorphology Using Low-cost Unmanned Aerial Vehicles and Structure-from-Motion Approach , 2019, Remote. Sens..

[44]  Giorgos Mallinis,et al.  On the Use of Unmanned Aerial Systems for Environmental Monitoring , 2018, Remote. Sens..

[45]  Geert Verhoeven,et al.  Taking computer vision aloft – archaeological three‐dimensional reconstructions from aerial photographs with photoscan , 2011 .

[46]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[47]  Yusheng Xu,et al.  Reconstruction of scaffolds from a photogrammetric point cloud of construction sites using a novel 3D local feature descriptor , 2018 .

[48]  F. Nex,et al.  UAV for 3D mapping applications: a review , 2014 .

[49]  Abdul Nurunnabi,et al.  Robust Segmentation for Large Volumes of Laser Scanning Three-Dimensional Point Cloud Data , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[50]  Diego González-Aguilera,et al.  Image-Based Modelling from Unmanned Aerial Vehicle (UAV) Photogrammetry: An Effective, Low-Cost Tool for Archaeological Applications , 2015 .

[51]  T. Rakha,et al.  Review of Unmanned Aerial System (UAS) applications in the built environment: Towards automated building inspection procedures using drones , 2018, Automation in Construction.