Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network

Deep learning‐based structural damage detection methods overcome the limitation of inferior adaptability caused by extensively varying real‐world situations (e.g., lighting and shadow changes). However, most deep learning‐based methods detect structural damage at the image level and grid‐cell level. To provide pixel‐level detection of multiple damages, a Fully Convolutional Network (FCN)‐based multiple damages detection method for concrete structure is proposed. To realize this method, a database of 2,750 images (with 504 × 376 pixels) including crack, spalling, efflorescence, and hole images in concrete structure is built, and the four damages included in those images are labeled manually. Then, the architecture of the FCN is modified, trained, validated, and tested using this database. A strategy of model‐based transfer learning is used to initialize the parameters of the FCN during the training process. The results show 98.61% pixel accuracy (PA), 91.59% mean pixel accuracy (MPA), 84.53% mean intersection over union (MIoU), and 97.34% frequency weighted intersection over union (FWIoU). Subsequently, the robustness and adaptability of the trained FCN model is tested and the damage is extracted, where damage areas are provided according to a calibrated relation between the ratio (the pixel area and true area of the detected object) and the distance from the smartphone to the concrete surface using a laser range finder. A comparative study is conducted to examine the performance of the proposed FCN‐based approach using a SegNet‐based method. The results show that the proposed method substantiates quite better performance and can indeed detect multiple concrete damages at the pixel level in realistic situations.

[1]  Jian Zhang,et al.  Pixel‐level crack delineation in images with convolutional feature fusion , 2018, Structural Control and Health Monitoring.

[2]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[3]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[4]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[5]  Yicheng Li,et al.  A Fast Detection Method via Region‐Based Fully Convolutional Neural Networks for Shield Tunnel Lining Defects , 2018, Comput. Aided Civ. Infrastructure Eng..

[6]  Kelvin C. P. Wang,et al.  Pavement Crack Width Measurement Based on Laplace's Equation for Continuity and Unambiguity , 2018, Comput. Aided Civ. Infrastructure Eng..

[7]  Hojjat Adeli,et al.  A novel unsupervised deep learning model for global and local health condition assessment of structures , 2018 .

[8]  Yi-Zhou Lin,et al.  Structural Damage Detection with Automatic Feature‐Extraction through Deep Learning , 2017, Comput. Aided Civ. Infrastructure Eng..

[9]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[10]  Xuefeng Zhao,et al.  A Method of Crack Detection Based on Convolutional Neural Networks , 2017 .

[11]  Hanqing Lu,et al.  Stacked Deconvolutional Network for Semantic Segmentation , 2017, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[12]  Hojjat Adeli,et al.  A novel machine learning‐based algorithm to detect damage in high‐rise building structures , 2017 .

[13]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[14]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[15]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[16]  Hojjat Adeli,et al.  Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete , 2017 .

[17]  Young-Jin Cha,et al.  Vision-based detection of loosened bolts using the Hough transform and support vector machines , 2016 .

[18]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  F. Yang,et al.  Road crack detection using deep convolutional neural network , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  H. Adeli,et al.  Synchrosqueezed wavelet transform-fractality model for locating, detecting, and quantifying damage in smart highrise building structures , 2015 .

[24]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[25]  Reinhold Huber-Mörk,et al.  Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images , 2014, ISVC.

[26]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[30]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Eduardo Zalama Casanova,et al.  Road Crack Detection Using Visual Features Extracted by Gabor Filters , 2014, Comput. Aided Civ. Infrastructure Eng..

[32]  Benjamin Schrauwen,et al.  Defect Detection in Reinforced Concrete Using Random Neural Architectures , 2014, Comput. Aided Civ. Infrastructure Eng..

[33]  Vikram Pakrashi,et al.  Texture Analysis Based Damage Detection of Ageing Infrastructural Elements , 2013, Comput. Aided Civ. Infrastructure Eng..

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[36]  Luh-Maan Chang,et al.  Support-vector-machine-based method for automated steel bridge rust assessment , 2012 .

[37]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[38]  Yozo Fujino,et al.  Concrete Crack Detection by Multiple Sequential Image Filtering , 2012, Comput. Aided Civ. Infrastructure Eng..

[39]  Ignacio Parra,et al.  Adaptive Road Crack Detection System by Pavement Classification , 2011, Sensors.

[40]  Christian Koch,et al.  Pothole detection in asphalt pavement images , 2011, Adv. Eng. Informatics.

[41]  Ezzatollah Salari,et al.  Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification , 2010, Comput. Aided Civ. Infrastructure Eng..

[42]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[43]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[44]  Paulo Lobato Correia,et al.  Automatic road crack segmentation using entropy and image dynamic thresholding , 2009, 2009 17th European Signal Processing Conference.

[45]  Shuji Hashimoto,et al.  Image‐Based Crack Detection for Real Concrete Surfaces , 2008 .

[46]  Hojjat Adeli,et al.  Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings , 2007 .

[47]  Peggy Subirats,et al.  Automation of Pavement Surface Crack Detection using the Continuous Wavelet Transform , 2006, 2006 International Conference on Image Processing.

[48]  Yoshihiko Hamamoto,et al.  A Method for Crack Detection on a Concrete Structure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[49]  Bugao Xu,et al.  Automatic inspection of pavement cracking distress , 2005, SPIE Optics + Photonics.

[50]  Byoung Jik Lee,et al.  Position‐Invariant Neural Network for Digital Pavement Crack Analysis , 2004 .

[51]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[52]  Benjamin A. Graybeal,et al.  Visual Inspection of Highway Bridges , 2002 .

[53]  Tony R. Martinez,et al.  The need for small learning rates on large problems , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[54]  Osama Moselhi,et al.  Classification of Defects in Sewer Pipes Using Neural Networks , 2000 .

[55]  Heng-Da Cheng,et al.  Novel Approach to Pavement Cracking Detection Based on Fuzzy Set Theory , 1999 .

[56]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[57]  S A Velinsky,et al.  Histogram‐Based Approach for Automated Pavement‐Crack Sensing , 1992 .

[58]  Boguslaw Cyganek,et al.  Image recognition with deep neural networks in presence of noise - Dealing with and taking advantage of distortions , 2017, Integr. Comput. Aided Eng..

[59]  Qingquan Li,et al.  An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection , 2017, Image Vis. Comput..

[60]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[61]  Paul Fieguth,et al.  Automated detection of cracks in buried concrete pipe images , 2006 .

[62]  Djemel Ziou,et al.  Edge Detection Techniques-An Overview , 1998 .