Workflow for Off-Site Bridge Inspection Using Automatic Damage Detection-Case Study of the Pahtajokk Bridge

For the inspection of structures, particularly bridges, it is becoming common to replace humans with autonomous systems that use unmanned aerial vehicles (UAV). In this paper, a framework for autonomous bridge inspection using a UAV is proposed with a four-step workflow: (a) data acquisition with an efficient UAV flight path, (b) computer vision comprising training, testing and validation of convolutional neural networks (ConvNets), (c) point cloud generation using intelligent hierarchical dense structure from motion (DSfM), and (d) damage quantification. This workflow starts with planning the most efficient flight path that allows for capturing of the minimum number of images required to achieve the maximum accuracy for the desired defect size, then followed by bridge and damage recognition. Three types of autonomous detection are used: masking the background of the images, detecting areas of potential damage, and pixel-wise damage segmentation. Detection of bridge components by masking extraneous parts of the image, such as vegetation, sky, roads or rivers, can improve the 3D reconstruction in the feature detection and matching stages. In addition, detecting damaged areas involves the UAV capturing close-range images of these critical regions, and damage segmentation facilitates damage quantification using 2D images. By application of DSfM, a denser and more accurate point cloud can be generated for these detected areas, and aligned to the overall point cloud to create a digital model of the bridge. Then, this generated point cloud is evaluated in terms of outlier noise, and surface deviation. Finally, damage that has been detected is quantified and verified, based on the point cloud generated using the Terrestrial Laser Scanning (TLS) method. The results indicate this workflow for autonomous bridge inspection has potential.

[1]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[2]  Soojin Cho,et al.  Image‐based concrete crack assessment using mask and region‐based convolutional neural network , 2019, Structural Control and Health Monitoring.

[3]  Ali Khaloo,et al.  Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling , 2018 .

[4]  Higinio González-Jorge,et al.  Validation of terrestrial laser scanning and photogrammetry techniques for the measurement of vertical underclearance and beam geometry in structural inspection of bridges , 2013 .

[5]  L. Taşçi,et al.  Deformation monitoring in steel arch bridges through close-range photogrammetry and the finite element method , 2015, Experimental Techniques.

[6]  David V. Jáuregui,et al.  Static Measurement of Beam Deformations via Close-Range Photogrammetry , 2002 .

[7]  Ajay Singh,et al.  Texture Features based Blur Classification in Barcode Images , 2013 .

[8]  Jian Zhang,et al.  Zernike‐moment measurement of thin‐crack width in images enabled by dual‐scale deep learning , 2018, Comput. Aided Civ. Infrastructure Eng..

[9]  Pedro Arias,et al.  An innovative method for remote measurement of minimum vertical underclearance in routine bridge inspection , 2012 .

[10]  Jenny Harding,et al.  DATA QUALITY IN 3D: GAUGING QUALITY MEASURES FROM USERS’ REQUIREMENTS , 2007 .

[11]  Hojjat Adeli,et al.  A New Approach for Health Monitoring of Structures: Terrestrial Laser Scanning , 2007, Comput. Aided Civ. Infrastructure Eng..

[12]  Peng Zhao,et al.  Damage Classification for Masonry Historic Structures Using Convolutional Neural Networks Based on Still Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[13]  Xuefeng Zhao,et al.  Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network , 2019, Comput. Aided Civ. Infrastructure Eng..

[14]  George Pavlidis,et al.  Multi-image 3D reconstruction data evaluation , 2014 .

[15]  Maziar Jamshidi,et al.  Simultaneous pixel-level concrete defect detection and grouping using a fully convolutional model , 2021, Structural Health Monitoring.

[16]  Weidong Wu,et al.  Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation , 2018, 2018 IEEE International Smart Cities Conference (ISC2).

[17]  Eleni Smyrou,et al.  Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning , 2021, Automation in Construction.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Ting-Hua Yi,et al.  Reviews on innovations and applications in structural health monitoring for infrastructures , 2014 .

[20]  A. Ellmann,et al.  Terrestrial laser scanning for the monitoring of bridge load tests – two case studies , 2018 .

[21]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[22]  Damian Beben,et al.  Displacements Monitoring of Suspension Bridge Using Geodetic Techniques , 2017 .

[23]  Lizhen Zhao,et al.  Application of Inclinometer in Arch Bridge Dynamic Deflection Measurement , 2014 .

[24]  D. Girardeau-Montaut,et al.  CHANGE DETECTION ON POINTS CLOUD DATA ACQUIRED W ITH A GROUND LASER SCANNER , 2005 .

[25]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[26]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[27]  Wei Wang,et al.  Computer vision-based concrete crack detection using U-net fully convolutional networks , 2019, Automation in Construction.

[28]  Dong Yeob Han,et al.  Deformation Measurement of a Railroad Bridge Using a Photogrammetric Board without Control Point Survey , 2018, J. Sensors.

[29]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[30]  Lennart Elfgren,et al.  3D reconstruction of existing concrete bridges using optical methods , 2019, Structure and Infrastructure Engineering.

[31]  João Bento,et al.  Bridge deflection evaluation using strain and rotation measurements , 2013 .

[32]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[33]  Xin Nie,et al.  Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction , 2020, Comput. Aided Civ. Infrastructure Eng..

[34]  Xiao Liang,et al.  Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization , 2018, Comput. Aided Civ. Infrastructure Eng..

[35]  Siu-Wing Cheng,et al.  Denoising a Point Cloud for Surface Reconstruction , 2016, ArXiv.