A real-time detection approach for bridge cracks based on YOLOv4-FPM

Abstract In order to realize real-time detection for bridge cracks by unmanned aerial vehicle (UAV), a deep learning model named YOLOv4-FPM is proposed on the basis of the YOLOv4 model. In YOLOv4-FPM, focal loss is used to optimize the loss function, which improves the accuracy and overcomes the challenges of complex background. Pruning algorithm is used to simplify the network and accelerate the detection speed. The multi-scale dataset is used to expand the predictable range of YOLOv4-FPM and enhance its scale robustness. The experimental results show that the mean average precision (mAP) of YOLOv4-FPM is 0.976, which is 0.064 higher than YOLOv4. The size and parameters of the model are reduced to 18.2%, and the model processes in real-time (119FPS) images at 1000 × 1000 pixels, which is 20 times faster than in a recent work. Moreover, it can effectively detect cracks in images of different sizes.

[1]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[2]  F. Nex,et al.  UAV for 3D mapping applications: a review , 2014 .

[3]  Anis Koubaa,et al.  Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3 , 2018, 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS).

[4]  ChaYoung-Jin,et al.  Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks , 2017 .

[5]  Yihong Gong,et al.  Object detection with class aware region proposal network and focused attention objective , 2020, Pattern Recognit. Lett..

[6]  Hyuk-Jae Lee,et al.  Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Michelle Karg,et al.  Saliency-guided region proposal network for CNN based object detection , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[8]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[9]  Zhaozheng Hu,et al.  Grid-based pavement crack analysis using deep learning , 2017, 2017 4th International Conference on Transportation Information and Safety (ICTIS).

[10]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[11]  Wei Li,et al.  Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network , 2019, Automation in Construction.

[12]  Zijun Zhang,et al.  Automatic Detection of Wind Turbine Blade Surface Cracks Based on UAV-Taken Images , 2017, IEEE Transactions on Industrial Electronics.

[13]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[15]  Larry S. Davis,et al.  Face-MagNet: Magnifying Feature Maps to Detect Small Faces , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[16]  Zhiqiang Shen,et al.  Learning Efficient Convolutional Networks through Network Slimming , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Quoc-Lam Nguyen,et al.  Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network , 2018, Automation in Construction.

[19]  Joseph A. Beck,et al.  Computationally Intelligent Image Processing Techniques for Crack Detection in Structural Components from Imaged Data , 2016 .

[20]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[21]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[23]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Arne Schumann,et al.  Deep cross-domain flying object classification for robust UAV detection , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[25]  Naif Alajlan,et al.  Efficient Framework for Palm Tree Detection in UAV Images , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[27]  Dariu Gavrila,et al.  EuroCity Persons: A Novel Benchmark for Person Detection in Traffic Scenes , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mohan Rajesh Elara,et al.  Self-reconfigurable façade-cleaning robot equipped with deep-learning-based crack detection based on convolutional neural networks , 2019 .

[29]  Fan Meng,et al.  A novel YOLOv3-arch model for identifying cholelithiasis and classifying gallstones on CT images , 2019, PloS one.

[30]  Won-Hwa Hong,et al.  Application of Crack Identification Techniques for an Aging Concrete Bridge Inspection Using an Unmanned Aerial Vehicle , 2018, Sensors.

[31]  Sattar Dorafshan,et al.  Evaluation of bridge decks with overlays using impact echo, a deep learning approach , 2020 .

[32]  Mustafa Gul,et al.  Densely connected deep neural network considering connectivity of pixels for automatic crack detection , 2020 .

[33]  Tran Hiep Dinh,et al.  Automatic Crack Detection in Built Infrastructure Using Unmanned Aerial Vehicles , 2017, ArXiv.