Cost-effective system for detection and quantification of concrete surface cracks by combination of convolutional neural network and image processing techniques

Abstract This study proposed a semi-automated system for crack detection and quantification, based on the combination of a trained convolutional neural network (CNN) and a developed application. Specifically, we tested four commonly used CNNs and determined GoogLeNet for this study. Then, the transfer learning and fully training of GoogLeNet were further tested on our testing dataset and a public dataset. The results show that the transfer learning GoogLeNet has relatively balanced performances on these two datasets, with accuracy of 96.69% and 88.39%, respectively. A new sliding window technique (neighborhood scanning) was proposed and shown almost equivalent performance to the previous dual scanning method. A method for calculating crack width was presented. The average relative error of this method is 14.58% (0.05 mm), i.e., much smaller than the 36.37% (i.e., 0.14 mm) of the previous method. An application was then developed to integrate the proposed methods and other techniques such as edge detectors, boundary tracking, and threshold segmentation to segment, quantify, and analyze cracks. Verifications on 23 untrained raw images (eleven with 10240 × 2048 pixels, twelve with 2592 × 4608 pixels) show that: (1) the developed system and a previous pixel-level segmentation system require an average of 9.48 s and 10.35 s; (2) these two systems show an 80.40% and a 78.64% average Intersection over Union (IoU). Therefore, the proposed system is a cost-effective solution for detecting and analyzing cracks on concrete surfaces considering its practical performance and time cost. Practically, the proposed system could be used to analyze the images collected from onsite inspection or from experiment.

[1]  Benjamin Schrauwen,et al.  Defect Detection in Reinforced Concrete Using Random Neural Architectures , 2014, Comput. Aided Civ. Infrastructure Eng..

[2]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[3]  I-Ming Chen,et al.  Transfer learning on convolutional activation feature as applied to a building quality assessment robot , 2017 .

[4]  Siddhartha Kumar Khaitan,et al.  Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection , 2017 .

[5]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[6]  Eduardo Zalama Casanova,et al.  Road Crack Detection Using Visual Features Extracted by Gabor Filters , 2014, Comput. Aided Civ. Infrastructure Eng..

[7]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[8]  Kaige Zhang,et al.  Unified Approach to Pavement Crack and Sealed Crack Detection Using Preclassification Based on Transfer Learning , 2018, J. Comput. Civ. Eng..

[9]  Devin K. Harris,et al.  Robust Pixel-Level Crack Detection Using Deep Fully Convolutional Neural Networks , 2019, J. Comput. Civ. Eng..

[10]  Young-Jin Cha,et al.  Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning , 2020 .

[11]  Jian Zhang,et al.  Pixel‐level crack delineation in images with convolutional feature fusion , 2018, Structural Control and Health Monitoring.

[12]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[13]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[14]  Hongguang Li,et al.  Pixel-Wise Crack Detection Using Deep Local Pattern Predictor for Robot Application , 2018, Sensors.

[15]  Ezzatollah Salari,et al.  Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification , 2010, Comput. Aided Civ. Infrastructure Eng..

[16]  John E. Taylor,et al.  Special Issue on Computational Approaches to Understand and Reduce Energy Consumption in the Built Environment , 2014 .

[17]  Kristin J. Dana,et al.  Automated Crack Detection on Concrete Bridges , 2016, IEEE Transactions on Automation Science and Engineering.

[18]  Hojjat Adeli,et al.  Enhanced probabilistic neural network with local decision circles: A robust classifier , 2010, Integr. Comput. Aided Eng..

[19]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[20]  Hojjat Adeli,et al.  Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings , 2007 .

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[23]  Ashutosh Bagchi,et al.  Image-based retrieval of concrete crack properties for bridge inspection , 2014 .

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Ashutosh Bagchi,et al.  Image based retrieval of concrete crack properties , 2012 .

[26]  Pingrang Wang,et al.  Comparison analysis on present image-based crack detection methods in concrete structures , 2010, 2010 3rd International Congress on Image and Signal Processing.

[27]  Abdenour Nazef,et al.  Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road Assessment , 2016 .

[28]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[29]  Jin H. Huang,et al.  Detection of cracks using neural networks and computational mechanics , 2002 .

[30]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[31]  Xianqiang Yang,et al.  A Modified CenterNet for Crack Detection of Sanitary Ceramics , 2020, IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society.

[32]  Alejandro F. Frangi,et al.  Model-based quantitation of 3-D magnetic resonance angiographic images , 1999, IEEE Transactions on Medical Imaging.

[33]  Rafael C. Gonzalez.  Digital Image Processing, 4Th Edition , 1977 .

[34]  Horst-Michael Groß,et al.  How to get pavement distress detection ready for deep learning? A systematic approach , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[35]  Pradeep Kumar,et al.  Flaw detection in radiographic weldment images using morphological watershed segmentation technique , 2009 .

[36]  Xuefeng Zhao,et al.  Image-Based Concrete Crack Detection Using Convolutional Neural Network and Exhaustive Search Technique , 2019, Advances in Civil Engineering.

[37]  Yozo Fujino,et al.  Concrete Crack Detection by Multiple Sequential Image Filtering , 2012, Comput. Aided Civ. Infrastructure Eng..

[38]  Mani Golparvar-Fard,et al.  Image-Based Automated 3D Crack Detection for Post-disaster Building Assessment , 2014, J. Comput. Civ. Eng..

[39]  David Lattanzi,et al.  Robust Automated Concrete Damage Detection Algorithms for Field Applications , 2014, J. Comput. Civ. Eng..

[40]  J. Nathan Kutz,et al.  Computer Vision Based Inspection Approach to Predict Damage State and Load Level for RC Members , 2017 .

[41]  Kelvin C. P. Wang,et al.  Wavelet-Based Pavement Distress Image Edge Detection with À Trous Algorithm , 2007 .

[42]  Young-Jin Cha,et al.  Vision-based detection of loosened bolts using the Hough transform and support vector machines , 2016 .

[43]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[44]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[45]  Joan-Lluís Zamora i Mestre,et al.  Image Post-Processing Method for Quantification of Cracking in RC Precast Beams under Bending , 2018, Buildings.

[46]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[47]  J. Nathan Kutz,et al.  Data-driven vision-based inspection for reinforced concrete beams and slabs: Quantitative damage and load estimation , 2018, Automation in Construction.