Multiscale and Adversarial Learning-Based Semi-Supervised Semantic Segmentation Approach for Crack Detection in Concrete Structures

Typically, the operational lifetime of underground concrete structures is several decades. At present, many such structures are approaching their original life expectancy. In this stage, the essential functionality of the structures may be considerably degraded, leading to various safety hazards such as collapse roof and tunnel flooding. In general, to overcome such problems, the maintenance of underground structures has been conducted through manual subjective inspections so far. However, recently, several objective inspection technologies have been actively developed by fusing artificial intelligence and imaging techniques recently. In particular, deep learning algorithms have been developed to detect concrete cracks, based on a large amount of data for supervised learning, including numerous labeled images. Such data acquisition requires considerable time and effort. To reduce these costs, in this study, multiscale and adversarial learning techniques were applied to realize crack detection. A total of 1,200 labeled data and 3,000 unlabeled data were used to implement and verify the proposed method. The multiscale segmentation neural network, discriminator neural network, and adversarial learning technique were used to realize accurate crack detection, enhance the learning performance, and ensure the efficiency of training data, respectively. The resulting algorithm had a pixel accuracy, mean intersection over union, frequency weighted intersection over union, and F1 score of 98.176%, 88.936%, 96.525%, and 88.789%, respectively. The proposed technique can be used to examine the conditions to ensure the safe maintenance of aging structures.

[1]  Soojin Cho,et al.  Image‐based concrete crack assessment using mask and region‐based convolutional neural network , 2019, Structural Control and Health Monitoring.

[2]  Tarek Zayed,et al.  Machine vision-based model for spalling detection and quantification in subway networks , 2017 .

[3]  Chih-Chen Chang,et al.  Localization and Quantification of Concrete Spalling Defects Using Terrestrial Laser Scanning , 2015, J. Comput. Civ. Eng..

[4]  Umapada Pal,et al.  Automatic Bridge Crack Detection - A Texture Analysis-Based Approach , 2014, ANNPR.

[5]  Weihua Sheng,et al.  Developing a crack inspection robot for bridge maintenance , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Hui-li Zhao,et al.  Improvement of canny algorithm based on pavement edge detection , 2010, 2010 3rd International Congress on Image and Signal Processing.

[7]  Henry A. Russell,et al.  INSPECTION POLICY AND PROCEDURES FOR RAIL TRANSIT TUNNELS AND UNDERGROUND STRUCTURES , 1997 .

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Ã.F. Ãzgenel,et al.  Performance Comparison of Pretrained Convolutional Neural Networks on Crack Detection in Buildings , 2018 .

[10]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Xiaoou Tang,et al.  Mix-and-Match Tuning for Self-Supervised Semantic Segmentation , 2017, AAAI.

[13]  Peggy Subirats,et al.  Automation of Pavement Surface Crack Detection using the Continuous Wavelet Transform , 2006, 2006 International Conference on Image Processing.

[14]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[15]  Xin Feng,et al.  Pavement distress detection and classification with automated image processing , 2011, Proceedings 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE).

[16]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[17]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[18]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[20]  Shamim N. Pakzad,et al.  Automated Detection of Corrosion Damage in Power Transmission Lattice Towers Using Image Processing , 2017 .

[21]  Sungroh Yoon,et al.  FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Soojin Cho,et al.  Automated Vision-Based Detection of Cracks on Concrete Surfaces Using a Deep Learning Technique , 2018, Sensors.

[23]  Shuang Wang,et al.  Automatic Pixel-Level Crack Detection on Dam Surface Using Deep Convolutional Network , 2020, Sensors.

[24]  Concetto Spampinato,et al.  Semi Supervised Semantic Segmentation Using Generative Adversarial Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Pedro Arias,et al.  Automatic segmentation of road overpasses and detection of mortar efflorescence using mobile LiDAR data , 2013 .

[26]  Eduardo Zalama Casanova,et al.  Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[27]  Weihua Sheng,et al.  A Robotic Crack Inspection and Mapping System for Bridge Deck Maintenance , 2014, IEEE Transactions on Automation Science and Engineering.

[28]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yun Liu,et al.  Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring , 2014, Sensors.

[30]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[31]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Vikram Pakrashi,et al.  Regionally Enhanced Multiphase Segmentation Technique for Damaged Surfaces , 2014, Comput. Aided Civ. Infrastructure Eng..

[33]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[34]  Robert J. Thomas,et al.  SDNET2018: An annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks , 2018, Data in brief.

[35]  Yozo Fujino,et al.  Recent research and development programs for infrastructures maintenance, renovation and management in Japan , 2020, Structure and Infrastructure Engineering.

[36]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Khurram Kamal,et al.  Detection of pavement cracks using tiled fuzzy Hough transform , 2017, J. Electronic Imaging.

[38]  Ming-Hsuan Yang,et al.  Adversarial Learning for Semi-supervised Semantic Segmentation , 2018, BMVC.

[39]  Jian Wan,et al.  Semi-Supervised Semantic Segmentation Using Adversarial Learning for Pavement Crack Detection , 2020, IEEE Access.

[40]  Chang-Soo Han,et al.  Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel , 2007 .

[41]  Cordelia Schmid,et al.  Accurate Object Localization with Shape Masks , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.