Developing a new deep learning CNN model to detect and classify highway cracks

Purpose This paper aims to Test the capabilities/accuracies of four deep learning pre trained convolutional neural network (CNN) models to detect and classify types of highway cracks, as well as developing a new CNN model to maximize the accuracy at different learning rates. Design/methodology/approach A sample of 4,663 images of highway cracks were collected and classified into three categories of cracks, namely, “vertical cracks,” “horizontal and vertical cracks” and “diagonal cracks,” subsequently, using “Matlab” to classify the sample to training (70%) and testing (30%) to apply the four deep learning CNN models and compute their accuracies. After that, developing a new deep learning CNN model to maximize the accuracy of detecting and classifying highway cracks and testing the accuracy using three optimization algorithms at different learning rates. Findings The accuracies result of the four deep learning pre-trained models are above the averages between top-1 and top-5 and the accuracy of classifying and detecting the samples exceeded the top-5 accuracy for the pre-trained AlexNet model around 3% and by 0.2% for the GoogleNet model. The accurate model here is the GoogleNet model as the accuracy is 89.08% and it is higher than AlexNet by 1.26%. While the computed accuracy for the new created deep learning CNN model exceeded all pre-trained models by achieving 97.62% at a learning rate of 0.001 using Adam’s optimization algorithm. Practical implications The created deep learning CNN model will enable users (e.g. highway agencies) to scan a long highway and detect types of cracks accurately in a very short time compared to traditional approaches. Originality/value A new deep learning CNN-based highway cracks detection was developed based on testing four pre-trained CNN models and analyze the capabilities of each model to maximize the accuracy of the proposed CNN.

[1]  Xiangrong Xu,et al.  Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm , 2021 .

[2]  Lukumon O. Oyedele,et al.  Deep learning in the construction industry: A review of present status and future innovations , 2020 .

[3]  Mayrai Gindy,et al.  A computer vision based rebar detection chain for automatic processing of concrete bridge deck GPR data , 2020 .

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  X. Ye,et al.  Structural crack detection using deep learning–based fully convolutional networks , 2019, Advances in Structural Engineering.

[6]  Heng-Da Cheng,et al.  Self-Supervised Structure Learning for Crack Detection Based on Cycle-Consistent Generative Adversarial Networks , 2020, J. Comput. Civ. Eng..

[7]  Chuan-Zhi Dong,et al.  Structural displacement monitoring using deep learning-based full field optical flow methods , 2020, Structure and Infrastructure Engineering.

[8]  Chaobo Zhang,et al.  Concrete bridge surface damage detection using a single‐stage detector , 2019, Comput. Aided Civ. Infrastructure Eng..

[9]  Yunyi Jia,et al.  Automatic detection of moisture damages in asphalt pavements from GPR data with deep CNN and IRS method , 2020 .

[10]  Chuan-Zhi Dong,et al.  A review of computer vision–based structural health monitoring at local and global levels , 2020 .

[11]  Wei Li,et al.  CrackU‐net: A novel deep convolutional neural network for pixelwise pavement crack detection , 2020, Structural Control and Health Monitoring.

[12]  Hui Li,et al.  The State of the Art of Data Science and Engineering in Structural Health Monitoring , 2019, Engineering.

[13]  Niannian Wang,et al.  Automatic damage detection of historic masonry buildings based on mobile deep learning , 2019, Automation in Construction.

[14]  Hai Liu,et al.  Detection and localization of rebar in concrete by deep learning using ground penetrating radar , 2020 .

[15]  Zili Xu,et al.  Damage detection in a novel deep-learning framework: a robust method for feature extraction , 2020, Structural Health Monitoring.

[16]  Sung Ho Hwang,et al.  Estimation of crack width based on shape‐sensitive kernels and semantic segmentation , 2019, Structural Control and Health Monitoring.

[17]  Qipei Mei,et al.  Multi-level feature fusion in densely connected deep-learning architecture and depth-first search for crack segmentation on images collected with smartphones , 2020, Structural Health Monitoring.

[18]  Yang Zhao,et al.  Deep learning-based feature engineering methods for improved building energy prediction , 2019, Applied Energy.

[19]  Alejandro Betancourt,et al.  Automatic detection of building typology using deep learning methods on street level images , 2020 .

[20]  Zhaoyu Su,et al.  Artificial intelligence-empowered pipeline for image-based inspection of concrete structures , 2020 .

[21]  Xiao Liang,et al.  Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization , 2018, Comput. Aided Civ. Infrastructure Eng..

[22]  Amir Hossein Alavi,et al.  New machine learning-based prediction models for fracture energy of asphalt mixtures , 2019, Measurement.

[23]  Jen-Yu Han,et al.  Pavement performance monitoring and anomaly recognition based on crowdsourcing spatiotemporal data , 2019, Automation in Construction.

[24]  Jian Zhang,et al.  Pixel‐level crack delineation in images with convolutional feature fusion , 2018, Structural Control and Health Monitoring.

[25]  Hyung-Jo Jung,et al.  Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing , 2019, Automation in Construction.

[26]  Wentai Lei,et al.  Automatic hyperbola detection and fitting in GPR B-scan image , 2019, Automation in Construction.

[27]  Yongjun Sun,et al.  Statistical investigations of transfer learning-based methodology for short-term building energy predictions , 2020 .

[28]  Young-Jin Cha,et al.  Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer , 2019, Construction and Building Materials.

[29]  Xuefeng Zhao,et al.  Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network , 2019, Comput. Aided Civ. Infrastructure Eng..

[30]  Sung-Han Sim,et al.  Automated bridge component recognition from point clouds using deep learning , 2020, Structural Control and Health Monitoring.

[31]  R. Yin.  The Case Study Crisis: Some Answers , 1981 .

[32]  Sattar Dorafshan,et al.  Evaluation of bridge decks with overlays using impact echo, a deep learning approach , 2020 .

[33]  Dulcy M. Abraham,et al.  Deep Learning-Based Automated Detection of Sewer Defects in CCTV Videos , 2020, J. Comput. Civ. Eng..

[34]  Xiao Liang,et al.  Uncertainty‐assisted deep vision structural health monitoring , 2020, Comput. Aided Civ. Infrastructure Eng..

[35]  Young-Jin Cha,et al.  Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning , 2020 .

[36]  Graziano Fiorillo,et al.  Improving the conversion accuracy between bridge element conditions and NBI ratings using deep convolutional neural networks , 2020 .

[37]  Mohsen Guizani,et al.  Deep Learning for IoT Big Data and Streaming Analytics: A Survey , 2017, IEEE Communications Surveys & Tutorials.

[38]  Mustafa Gul,et al.  A cost effective solution for pavement crack inspection using cameras and deep neural networks , 2020 .

[39]  Wei Lu,et al.  Image-based concrete crack detection in tunnels using deep fully convolutional networks , 2020 .

[40]  Zheng Yi Wu,et al.  Deep learning‐based multi‐class damage detection for autonomous post‐disaster reconnaissance , 2020, Structural Control and Health Monitoring.

[41]  Qiwen Qiu,et al.  Imaging techniques for defect detection of fiber reinforced polymer‐bonded civil infrastructures , 2020, Structural Control and Health Monitoring.

[42]  Jinbo Song,et al.  Weakly supervised network based intelligent identification of cracks in asphalt concrete bridge deck , 2020, Alexandria Engineering Journal.

[43]  Joachim Denzler,et al.  Deep learning and process understanding for data-driven Earth system science , 2019, Nature.

[44]  Hui Li,et al.  Identification framework for cracks on a steel structure surface by a restricted Boltzmann machines algorithm based on consumer‐grade camera images , 2018 .

[45]  F. A. Silva,et al.  Combined mechanical and 3D-microstructural analysis of strain-hardening cement-based composites (SHCC) by in-situ X-ray microtomography , 2020 .

[46]  Masayuki Kohiyama,et al.  Detection method of unlearned pattern using support vector machine in damage classification based on deep neural network , 2020, Structural Control and Health Monitoring.

[47]  Junyu Dong,et al.  An Overview on Data Representation Learning: From Traditional Feature Learning to Recent Deep Learning , 2016, ArXiv.

[48]  Ye Xiaowei,et al.  Structural crack detection using deep learning–based fully convolutional networks: , 2019 .

[49]  Mohammad R. Jahanshahi,et al.  Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection , 2018 .

[50]  Soojin Cho,et al.  Image‐based concrete crack assessment using mask and region‐based convolutional neural network , 2019, Structural Control and Health Monitoring.

[51]  Yun-Kyu An,et al.  Deep super resolution crack network (SrcNet) for improving computer vision–based automated crack detectability in in situ bridges , 2020, Structural Health Monitoring.

[52]  Amir H. Behzadan,et al.  Deep learning for site safety: Real-time detection of personal protective equipment , 2020 .

[53]  Ying Wang,et al.  SHMnet: Condition assessment of bolted connection with beyond human-level performance , 2020, Structural Health Monitoring.

[54]  Xiaochun Luo,et al.  An automatic and non-invasive physical fatigue assessment method for construction workers , 2019, Automation in Construction.

[55]  Seokho Chi,et al.  UAV-RFID Integration for Construction Resource Localization , 2020 .

[56]  Hyoungkwan Kim,et al.  Patch-Based Crack Detection in Black Box Images Using Convolutional Neural Networks , 2019, J. Comput. Civ. Eng..

[57]  Hong Pan,et al.  Deep BBN Learning for Health Assessment toward Decision-Making on Structures under Uncertainties , 2018 .

[58]  Yunfeng Zhang,et al.  Bridge condition rating data modeling using deep learning algorithm , 2020, Structure and Infrastructure Engineering.

[59]  Nhat-Duc Hoang,et al.  Image Processing-Based Classification of Asphalt Pavement Cracks Using Support Vector Machine Optimized by Artificial Bee Colony , 2018, J. Comput. Civ. Eng..

[60]  David A Lange,et al.  Deep Learning-Based Automated Image Segmentation for Concrete Petrographic Analysis , 2020, ArXiv.