A vision-based active learning convolutional neural network model for concrete surface crack detection

Concrete surface crack detection based on computer vision, specifically via a convolutional neural network, has drawn increasing attention for replacing manual visual inspection of bridges and buildings. This article proposes a new framework for this task and a sampling and training method based on active learning to treat class imbalances. In particular, the new framework includes a clear definition of two categories of samples, a relevant sliding window technique, data augmentation and annotation methods. The advantages of this framework are that data integrity can be ensured and a very large amount of annotation work can be saved. Training datasets generated with the proposed sampling and training method not only are representative of the original dataset but also highlight samples that are highly complex, yet informative. Based on the proposed framework and sampling and training strategy, AlexNet is re-tuned, validated, tested and compared with an existing network. The investigation revealed outstanding performances of the proposed framework in terms of the detection accuracy, precision and F1 measure due to its nonlinear learning ability, training dataset integrity and active learning strategy.

[1]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[2]  Ye Xiaowei,et al.  Structural crack detection using deep learning–based fully convolutional networks: , 2019 .

[3]  ByoungChul Ko,et al.  A Brief Review of Facial Emotion Recognition Based on Visual Information , 2018, Sensors.

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ruimao Zhang,et al.  Cost-Effective Active Learning for Deep Image Classification , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Ming-Yu Liu,et al.  Deep Active Learning for Civil Infrastructure Defect Detection and Classification , 2017 .

[8]  Wei Zhang,et al.  Unified Vision‐Based Methodology for Simultaneous Concrete Defect Detection and Geolocalization , 2018, Comput. Aided Civ. Infrastructure Eng..

[9]  Zhen Wang,et al.  An adaptive delay compensation method based on a discrete system model for real-time hybrid simulation , 2020 .

[10]  Billie F. Spencer,et al.  Concrete Crack Assessment Using Digital Image Processing and 3D Scene Reconstruction , 2016, J. Comput. Civ. Eng..

[11]  Mohammed Bennamoun,et al.  A Guide to Convolutional Neural Networks for Computer Vision , 2018, A Guide to Convolutional Neural Networks for Computer Vision.

[12]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[15]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[16]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[17]  ChaYoung-Jin,et al.  Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks , 2017 .

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[20]  Enrique López Droguett,et al.  Convolutional neural networks for automated damage recognition and damage type identification , 2018, Structural Control and Health Monitoring.

[21]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[22]  Sung-Han Sim,et al.  Crack and Noncrack Classification from Concrete Surface Images Using Machine Learning , 2019 .

[23]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[24]  Gao Fan,et al.  Lost data recovery for structural health monitoring based on convolutional neural networks , 2019, Structural Control and Health Monitoring.

[25]  Chang Wang,et al.  Research of Bridge Crack Detecting System Based on Machine Vision , 2012 .

[26]  X. Ye,et al.  Structural crack detection using deep learning–based fully convolutional networks , 2019, Advances in Structural Engineering.

[27]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[28]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[29]  Bin Li,et al.  A survey on instance selection for active learning , 2012, Knowledge and Information Systems.

[30]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[31]  Zhe Wang,et al.  Crack Damage Detection Method via Multiple Visual Features and Efficient Multi-Task Learning Model , 2018, Sensors.

[32]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[33]  X. W. Ye,et al.  A review on deep learning-based structural health monitoring of civil infrastructures , 2019 .

[34]  Hui Li,et al.  Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images , 2019 .