Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation

We propose coupling the state-of-the-art computer technology deep learning and unmanned aerial vehicles (UAV) to automatically detect and assess the health condition of civil infrastructure such as bridges and pavements. UAV carrying high resolution camera and infrared thermography camera to collect a large amount of image data from the target infrastructure, which serves as inputs of trained deep neural networks for damage classification and condition assessment. Details of the framework that may guide the automation process are explained. We demonstrated preliminary application of using UAV and deep neural network in concrete crack and asphalt pavement distress classification. Challenges and needs for deployment of UAV and deep learning are briefly discussed in the end.

[1]  Abba G. Lichtenstein,et al.  The Silver Bridge Collapse Recounted , 1993 .

[2]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[3]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[4]  Thomas Oommen,et al.  Evaluating the Use of Unmanned Aerial Vehicles for Transportation Purposes , 2015 .

[5]  Yi-Zhou Lin,et al.  Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning , 2017, Comput. Aided Civ. Infrastructure Eng..

[6]  Soroush Mokhtari,et al.  Analytical study of computer vision-based pavement crack quantification using machine learning techniques , 2015 .

[7]  Asce,et al.  2013 report card for America’s infrastructure , 2017 .

[8]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[9]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[10]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[11]  Barritt Lovelace,et al.  Unmanned Aerial Vehicle Bridge Inspection Demonstration Project , 2015 .

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Zheng Wu,et al.  Performance Evaluation of Various Rehabilitation and Preservation Treatments , 2010 .

[14]  Ivan Bartoli,et al.  Bridge related damage quantification using unmanned aerial vehicle imagery , 2016 .

[15]  Ghada S. Moussa,et al.  A New Technique for Automatic Detection and Parameters Estimation of Pavement Crack , 2011 .

[16]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[17]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[18]  王海龙,et al.  Raspberry Pi Model B , 2012 .

[19]  J S Miller,et al.  DISTRESS IDENTIFICATION MANUAL FOR THE LONG-TERM PAVEMENT PERFORMANCE PROGRAM (FOURTH REVISED EDITION) , 2003 .

[20]  Carlo Meghini,et al.  Deep learning for decentralized parking lot occupancy detection , 2017, Expert Syst. Appl..

[21]  Nikolaos Doulamis,et al.  Deep Convolutional Neural Networks for efficient vision based tunnel inspection , 2015, 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[22]  Shamim N. Pakzad,et al.  Structural Damage Detection Using Convolutional Neural Networks , 2017 .

[23]  Jayavardhana Gubbi,et al.  Power infrastructure monitoring and damage detection using drone captured images , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[24]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Elisabeth Anna Malsch,et al.  The Causes of the I-35 West Bridge Collapse , 2011 .

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.