Vision and Deep Learning-Based Algorithms to Detect and Quantify Cracks on Concrete Surfaces from UAV Videos

Immediate assessment of structural integrity of important civil infrastructures, like bridges, hospitals, or dams, is of utmost importance after natural disasters. Currently, inspection is performed manually by engineers who look for local damages and their extent on significant locations of the structure to understand its implication on its global stability. However, the whole process is time-consuming and prone to human errors. Due to their size and extent, some regions of civil structures are hard to gain access for manual inspection. In such situations, a vision-based system of Unmanned Aerial Vehicles (UAVs) programmed with Artificial Intelligence algorithms may be an effective alternative to carry out a health assessment of civil infrastructures in a timely manner. This paper proposes a framework of achieving the above-mentioned goal using computer vision and deep learning algorithms for detection of cracks on the concrete surface from its image by carrying out image segmentation of pixels, i.e., classification of pixels in an image of the concrete surface and whether it belongs to cracks or not. The image segmentation or dense pixel level classification is carried out using a deep neural network architecture named U-Net. Further, morphological operations on the segmented images result in dense measurements of crack geometry, like length, width, area, and crack orientation for individual cracks present in the image. The efficacy and robustness of the proposed method as a viable real-life application was validated by carrying out a laboratory experiment of a four-point bending test on an 8-foot-long concrete beam of which the video is recorded using a camera mounted on a UAV-based, as well as a still ground-based, video camera. Detection, quantification, and localization of damage on a civil infrastructure using the proposed framework can directly be used in the prognosis of the structure’s ability to withstand service loads.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  S. Hashimoto,et al.  Automated Crack Detection for Concrete Surface Image Using Percolation Model and Edge Information , 2006, IECON 2006 - 32nd Annual Conference on IEEE Industrial Electronics.

[3]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[4]  Mohammad R. Jahanshahi,et al.  An innovative methodology for detection and quantification of cracks through incorporation of depth perception , 2011, Machine Vision and Applications.

[5]  Jian Zhang,et al.  Pixel‐level crack delineation in images with convolutional feature fusion , 2018, Structural Control and Health Monitoring.

[6]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[8]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[9]  Ashutosh Bagchi,et al.  Image-based retrieval of concrete crack properties for bridge inspection , 2014 .

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Ioannis Brilakis,et al.  Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation , 2011 .

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[14]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[15]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yongchao Yang,et al.  Dynamic Imaging: Real-Time Detection of Local Structural Damage with Blind Separation of Low-Rank Background and Sparse Innovation , 2016 .

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ye Xia,et al.  Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection , 2020 .

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Jizhong Xiao,et al.  Deep Concrete Inspection Using Unmanned Aerial Vehicle Towards CSSC Database , 2017 .

[21]  Rih-Teng Wu,et al.  A texture‐Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces , 2017, Comput. Aided Civ. Infrastructure Eng..

[22]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[23]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[24]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[25]  Tara C. Hutchinson,et al.  Improved image analysis for evaluating concrete damage , 2006 .

[26]  Soojin Cho,et al.  Image‐based concrete crack assessment using mask and region‐based convolutional neural network , 2019, Structural Control and Health Monitoring.

[27]  ChaYoung-Jin,et al.  Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks , 2017 .

[28]  Wei Wang,et al.  Computer vision-based concrete crack detection using U-net fully convolutional networks , 2019, Automation in Construction.

[29]  Satish Nagarajaiah,et al.  Automatic detection and damage quantification of multiple cracks on concrete surface from video , 2020, International Journal of Sustainable Materials and Structural Systems.

[30]  Li Li,et al.  DeepCrack: A deep hierarchical feature learning architecture for crack segmentation , 2019, Neurocomputing.

[31]  Gang Liu,et al.  Full-scale multi-functional test platform for investigating mechanical performance of track–subgrade systems of high-speed railways , 2020, Railway Engineering Science.