Pixel‐level multicategory detection of visible seismic damage of reinforced concrete components

The detection of visible damage (i.e., cracking, concrete spalling and crushing, reinforcement exposure, buckling and fracture) plays a key role in postearthquake safety assessment of reinforced concrete (RC) building structures. In this study, a novel approach based on computer‐vision techniques was developed for pixel‐level multicategory detection of visible seismic damage of RC components. A semantic segmentation database was constructed from test photos of RC structural components. Series of datasets were generated from the constructed database by applying image transformations and data‐balancing techniques at the sample and pixel levels. A deep convolutional network architecture was designed for pixel‐level detection of visible damage. Two sets of parameters were optimized separately, one to detect cracks and the other to detect all other types of damage. A postprocessing technique for crack detection was developed to refine crack boundaries, and thus improve the accuracy of crack characterization. Finally, the proposed vision‐based approach was applied to test photos of a beam‐to‐wall joint specimen. The results demonstrate the accuracy of the vision‐based approach to detect damage, and its high potential to estimate seismic damage states of RC components.

[1]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[2]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[3]  Shivprakash Iyer,et al.  Segmentation of Pipe Images for Crack Detection in Buried Sewers , 2006, Comput. Aided Civ. Infrastructure Eng..

[4]  Martin J. Alperen.  The Federal Emergency Management Agency , 2011 .

[5]  Hyoungkwan Kim,et al.  Encoder–decoder network for pixel‐level road crack detection in black‐box images , 2019, Comput. Aided Civ. Infrastructure Eng..

[6]  Young-Jin Cha,et al.  SDDNet: Real-Time Crack Segmentation , 2020, IEEE Transactions on Industrial Electronics.

[7]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[8]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[9]  Hojjat Adeli,et al.  Enhanced probabilistic neural network with local decision circles: A robust classifier , 2010, Integr. Comput. Aided Eng..

[10]  Dimos Polyzois,et al.  Deep learning-based automatic volumetric damage quantification using depth camera , 2019, Automation in Construction.

[11]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[12]  Qian Wang,et al.  DeepCrack: Learning Hierarchical Convolutional Features for Crack Detection , 2019, IEEE Transactions on Image Processing.

[13]  Mohammad R. Jahanshahi,et al.  An innovative methodology for detection and quantification of cracks through incorporation of depth perception , 2011, Machine Vision and Applications.

[14]  João Paulo Papa,et al.  FEMa: a finite element machine for fast learning , 2019, Neural Computing and Applications.

[15]  Yang Liu,et al.  Deep Learning-Based Fully Automated Pavement Crack Detection on 3D Asphalt Surfaces with an Improved CrackNet , 2018, J. Comput. Civ. Eng..

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Paulo Lobato Correia,et al.  Supervised strategies for cracks detection in images of road pavement flexible surfaces , 2008, 2008 16th European Signal Processing Conference.

[19]  Vikram Pakrashi,et al.  Regionally Enhanced Multiphase Segmentation Technique for Damaged Surfaces , 2014, Comput. Aided Civ. Infrastructure Eng..

[20]  X. Ji,et al.  Seismic behavior and strength capacity of steel coupling beam-to-SRC wall joints , 2019 .

[21]  Shirley J. Dyke,et al.  Visual data classification in post-event building reconnaissance , 2018 .

[22]  Billie F. Spencer,et al.  Towards Automated Post-Earthquake Inspections with Deep Learning-based Condition-Aware Models , 2018, ArXiv.

[23]  Xiao Liang,et al.  Uncertainty‐assisted deep vision structural health monitoring , 2020, Comput. Aided Civ. Infrastructure Eng..

[24]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ezzatollah Salari,et al.  Beamlet Transform‐Based Technique for Pavement Crack Detection and Classification , 2010, Comput. Aided Civ. Infrastructure Eng..

[26]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[27]  Heng-Da Cheng,et al.  Real-Time Image Thresholding Based on Sample Space Reduction and Interpolation Approach , 2003 .

[28]  Yun-Kyu An,et al.  Automated crack evaluation of a high‐rise bridge pier using a ring‐type climbing robot , 2020, Comput. Aided Civ. Infrastructure Eng..

[29]  Hojjat Adeli,et al.  A dynamic ensemble learning algorithm for neural networks , 2019, Neural Computing and Applications.

[30]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[31]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[32]  Wei Zhang,et al.  Unified Vision‐Based Methodology for Simultaneous Concrete Defect Detection and Geolocalization , 2018, Comput. Aided Civ. Infrastructure Eng..

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Satoshi Tanaka,et al.  RECOGNITION OF DAMAGED BUILDING USING DEEP LEARNING BASED ON AERIAL AND LOCAL PHOTOS TAKEN AFTER THE 1995 KOBE EARTHQUAKE , 2018 .

[36]  Hoang-Nam Nguyen,et al.  An Automatic Approach for Accurate Edge Detection of Concrete Crack Utilizing 2D Geometric Features of Crack , 2014, J. Signal Process. Syst..

[37]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[38]  Basic Procedures Manual.  EVALUATION OF EARTHQUAKE DAMAGED CONCRETE AND MASONRY WALL BUILDINGS , 1998 .

[39]  Yoshihiko Hamamoto,et al.  A robust automatic crack detection method from noisy concrete surfaces , 2011, Machine Vision and Applications.

[40]  Xinzheng Lu,et al.  Automated regional seismic damage assessment of buildings using an unmanned aerial vehicle and a convolutional neural network , 2020 .

[41]  Craig D. Comartin,et al.  Evaluation of Earthquake-Damaged Concrete and Masonry Wall Buildings , 2000 .

[42]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Hojjat Adeli,et al.  A New Neural Dynamic Classification Algorithm , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Yozo Fujino,et al.  Concrete Crack Detection by Multiple Sequential Image Filtering , 2012, Comput. Aided Civ. Infrastructure Eng..

[45]  Reginald DesRoches,et al.  Machine Vision-Enhanced Postearthquake Inspection , 2011, J. Comput. Civ. Eng..

[46]  Rih-Teng Wu,et al.  A texture‐Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces , 2017, Comput. Aided Civ. Infrastructure Eng..

[47]  Shuji Hashimoto,et al.  Image‐Based Crack Detection for Real Concrete Surfaces , 2008 .

[48]  Romain Huret,et al.  5. Federal Emergency Management Agency , 2010 .

[49]  Xiao Liang,et al.  Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization , 2018, Comput. Aided Civ. Infrastructure Eng..

[50]  Luh-Maan Chang,et al.  Support-vector-machine-based method for automated steel bridge rust assessment , 2012 .

[51]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[52]  Jian Zhang,et al.  Zernike‐moment measurement of thin‐crack width in images enabled by dual‐scale deep learning , 2018, Comput. Aided Civ. Infrastructure Eng..

[53]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[54]  Xiaodong Ji,et al.  Vision-based measurements of deformations and cracks for RC structure tests , 2020 .

[55]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[56]  Hiroyuki Chida,et al.  Study on image diagnosis of timber houses damaged by earthquake using deep learning , 2021, JAPAN ARCHITECTURAL REVIEW.

[57]  이선영,et al.  Histogram Equalization , 2012, Encyclopedia of Biometrics.