Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network

The spatial characteristics of cracks are significant indicators to assess and evaluate the health of existing buildings and infrastructures. However, the current manual crack description method is time consuming and labor consuming. To improve the efficiency of crack inspection, advanced computer vision‐based techniques have been utilized to detect cracks automatically at image level and grid‐cell level. But existing crack detections are of (high specificity) low generality and inefficient, in terms that conventional approaches are unable to identify and measure diverse cracks concurrently at pixel level. Therefore, this research implements a novel deep learning technique named fully convolutional network (FCN) to address this problem. First, FCN is trained by feeding multiple types of cracks to semantically identify and segment pixel‐wise cracks at different scales. Then, the predicted crack segmentations are represented by single‐pixel width skeletons to quantitatively measure the morphological features of cracks, providing valuable crack indicators for assessment in practice, such as crack topology, crack length, max width, and mean width. To validate the prediction, the predicted segmentations are compared with recent advanced method for crack recognition and ground truth. For crack segmentation, the accuracy, precision, recall, and F1 score are 97.96%, 81.73%, 78.97%, and 79.95%, respectively. For crack length, the relative measurement error varies from −48.03% to 177.79%, meanwhile that ranges from −13.27% to 24.01% for crack width. The results show that FCN is feasible and sufficient for crack identification and measurement. Although the accuracy is not as high as CrackNet because of three types of errors, the prediction has been increased to pixel level and the training time has been dramatically decreased to several per cents of previous methods due to the novel end‐to‐end structure of FCN, which combines typical convolutional neural networks and deconvolutional layers.

[1]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[2]  Yi-Zhou Lin,et al.  Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning , 2017, Comput. Aided Civ. Infrastructure Eng..

[3]  Tsai-Yun Liao,et al.  On‐Line Vehicle Routing Problems for Carbon Emissions Reduction , 2017, Comput. Aided Civ. Infrastructure Eng..

[4]  Jerome F. Hajjar,et al.  Laser-based surface damage detection and quantification using predicted surface properties , 2017 .

[5]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[6]  Hojjat Adeli,et al.  A novel machine learning‐based algorithm to detect damage in high‐rise building structures , 2017 .

[7]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[8]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[9]  Rih-Teng Wu,et al.  A texture‐Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces , 2017, Comput. Aided Civ. Infrastructure Eng..

[10]  Kelvin C. P. Wang,et al.  The Fast Prefix Coding Algorithm (FPCA) for 3D Pavement Surface Data Compression , 2017, Comput. Aided Civ. Infrastructure Eng..

[11]  Hojjat Adeli,et al.  Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete , 2017 .

[12]  Oral Büyüköztürk,et al.  Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters , 2017 .

[13]  Mu-Chun Su,et al.  A self organizing map optimization based image recognition and processing model for bridge crack inspection , 2017 .

[14]  Boguslaw Cyganek,et al.  Image recognition with deep neural networks in presence of noise - Dealing with and taking advantage of distortions , 2017, Integr. Comput. Aided Eng..

[15]  Leonardo Franco,et al.  Layer multiplexing FPGA implementation for deep back-propagation learning , 2017, Integr. Comput. Aided Eng..

[16]  Young-Jin Cha,et al.  Vision-based detection of loosened bolts using the Hough transform and support vector machines , 2016 .

[17]  Kelvin C. P. Wang,et al.  Efficient System of Cracking-Detection Algorithms with 1-mm 3D-Surface Models and Performance Measures , 2016 .

[18]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[19]  Yichang James Tsai,et al.  Enhanced Crack Segmentation Algorithm Using 3D Pavement Data , 2016, J. Comput. Civ. Eng..

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Abdenour Nazef,et al.  Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road Assessment , 2016 .

[22]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[23]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Gang Li,et al.  Long-distance precision inspection method for bridge cracks with image processing , 2014 .

[27]  Eduardo Zalama Casanova,et al.  Road Crack Detection Using Visual Features Extracted by Gabor Filters , 2014, Comput. Aided Civ. Infrastructure Eng..

[28]  Ashutosh Bagchi,et al.  Image-based retrieval of concrete crack properties for bridge inspection , 2014 .

[29]  Xiaoming Sun,et al.  A Pavement Crack Detection Method Combining 2D with 3D Information Based on Dempster‐Shafer Theory , 2014, Comput. Aided Civ. Infrastructure Eng..

[30]  Abraham Mathew,et al.  AUTOMATED VISUAL INSPECTION OF PAVEMENT CRACK DETECTION AND CHARACTERIZATION , 2014 .

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Burcin Becerik-Gerber,et al.  Unsupervised Approach for Autonomous Pavement-Defect Detection and Quantification Using an Inexpensive Depth Sensor , 2013, J. Comput. Civ. Eng..

[33]  Wenbin Ouyang,et al.  Pavement cracking measurements using 3D laser-scan images , 2013 .

[34]  Vikram Pakrashi,et al.  Texture Analysis Based Damage Detection of Ageing Infrastructural Elements , 2013, Comput. Aided Civ. Infrastructure Eng..

[35]  Yun Yong Kim,et al.  Automated image processing technique for detecting and analysing concrete surface cracks , 2013 .

[36]  Sami F. Masri,et al.  Adaptive vision-based crack detection using 3D scene reconstruction for condition assessment of structures , 2012 .

[37]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[38]  Yozo Fujino,et al.  Concrete Crack Detection by Multiple Sequential Image Filtering , 2012, Comput. Aided Civ. Infrastructure Eng..

[39]  P. Correia,et al.  Automatic Road Pavement Crack Detection using SVM , 2012 .

[40]  P. K. Ramakrishnan,et al.  AUTOMATIC DETECTION OF CRACKS IN PAVEMENTS USING EDGE DETECTION OPERATOR , 2012 .

[41]  Ignacio Parra,et al.  Adaptive Road Crack Detection System by Pavement Classification , 2011, Sensors.

[42]  Fereidoon Moghadas Nejad,et al.  An optimum feature extraction method based on Wavelet-Radon Transform and Dynamic Neural Network for pavement distress classification , 2011, Expert Syst. Appl..

[43]  Hyeong Gveong Moon,et al.  Inteligent crack detecting algorithm on the concrete crack image using neural network , 2011 .

[44]  Ezzatollah Salari,et al.  Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification , 2010, Comput. Aided Civ. Infrastructure Eng..

[45]  Paulo Lobato Correia,et al.  Automatic road crack segmentation using entropy and image dynamic thresholding , 2009, 2009 17th European Signal Processing Conference.

[46]  I. Iervolino,et al.  Computer Aided Civil and Infrastructure Engineering , 2009 .

[47]  Nii O. Attoh-Okine,et al.  Evaluating Pavement Cracks with Bidimensional Empirical Mode Decomposition , 2008, EURASIP J. Adv. Signal Process..

[48]  Shuji Hashimoto,et al.  Image‐Based Crack Detection for Real Concrete Surfaces , 2008 .

[49]  Dan Givoli,et al.  XFEM‐based crack detection scheme using a genetic algorithm , 2007 .

[50]  Chang-Soo Han,et al.  Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel , 2007 .

[51]  Peggy Subirats,et al.  Automation of Pavement Surface Crack Detection using the Continuous Wavelet Transform , 2006, 2006 International Conference on Image Processing.

[52]  Paul Fieguth,et al.  Automated detection of cracks in buried concrete pipe images , 2006 .

[53]  Bugao Xu,et al.  Automatic inspection of pavement cracking distress , 2005, SPIE Optics + Photonics.

[54]  Byoung Jik Lee,et al.  Position‐Invariant Neural Network for Digital Pavement Crack Analysis , 2004 .

[55]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[56]  Shuji Hashimoto,et al.  Accurate extraction and measurement of fine cracks from concrete block surface image , 2002, IEEE 2002 28th Annual Conference of the Industrial Electronics Society. IECON 02.

[57]  Jin H. Huang,et al.  Detection of cracks using neural networks and computational mechanics , 2002 .

[58]  Tony R. Martinez,et al.  The need for small learning rates on large problems , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[59]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[60]  Heng-Da Cheng,et al.  Novel Approach to Pavement Cracking Detection Based on Fuzzy Set Theory , 1999 .

[61]  Rangasami L. Kashyap,et al.  Building Skeleton Models via 3-D Medial Surface/Axis Thinning Algorithms , 1994, CVGIP Graph. Model. Image Process..

[62]  Mohamed S Kaseko,et al.  A neural network-based methodology for pavement crack detection and classification , 1993 .

[63]  S A Velinsky,et al.  Histogram‐Based Approach for Automated Pavement‐Crack Sensing , 1992 .

[64]  Ching Y. Suen,et al.  A fast parallel algorithm for thinning digital patterns , 1984, CACM.

[65]  C. J. Hilditch.  Comparison of thinning algorithms on a parallel processor , 1983, Image Vis. Comput..