Automatic Crack Detection on Road Pavements Using Encoder-Decoder Architecture

Automatic crack detection from images is an important task that is adopted to ensure road safety and durability for Portland cement concrete (PCC) and asphalt concrete (AC) pavement. Pavement failure depends on a number of causes including water intrusion, stress from heavy loads, and all the climate effects. Generally, cracks are the first distress that arises on road surfaces and proper monitoring and maintenance to prevent cracks from spreading or forming is important. Conventional algorithms to identify cracks on road pavements are extremely time-consuming and high cost. Many cracks show complicated topological structures, oil stains, poor continuity, and low contrast, which are difficult for defining crack features. Therefore, the automated crack detection algorithm is a key tool to improve the results. Inspired by the development of deep learning in computer vision and object detection, the proposed algorithm considers an encoder-decoder architecture with hierarchical feature learning and dilated convolution, named U-Hierarchical Dilated Network (U-HDN), to perform crack detection in an end-to-end method. Crack characteristics with multiple context information are automatically able to learn and perform end-to-end crack detection. Then, a multi-dilation module embedded in an encoder-decoder architecture is proposed. The crack features of multiple context sizes can be integrated into the multi-dilation module by dilation convolution with different dilatation rates, which can obtain much more cracks information. Finally, the hierarchical feature learning module is designed to obtain a multi-scale features from the high to low- level convolutional layers, which are integrated to predict pixel-wise crack detection. Some experiments on public crack databases using 118 images were performed and the results were compared with those obtained with other methods on the same images. The results show that the proposed U-HDN method achieves high performance because it can extract and fuse different context sizes and different levels of feature maps than other algorithms.

[1]  Ying Chen,et al.  Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement , 2020, Coatings.

[2]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Johannes Fürnkranz,et al.  Large-Scale Multi-label Text Classification - Revisiting Neural Networks , 2013, ECML/PKDD.

[4]  Qingquan Li,et al.  3D laser imaging and sparse points grouping for pavement crack detection , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[5]  Paulo Lobato Correia,et al.  Road Surface Crack Detection using a Light Field Camera , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[6]  Peggy Subirats,et al.  Automation of Pavement Surface Crack Detection using the Continuous Wavelet Transform , 2006, 2006 International Conference on Image Processing.

[7]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[8]  Hojjat Adeli,et al.  Machine Learning: Neural Networks, Genetic Algorithms, and Fuzzy Systems , 1994 .

[9]  Yong Hu,et al.  A novel LBP based methods for pavement crack detection , 2009 .

[10]  Qian Wang,et al.  DeepCrack: Learning Hierarchical Convolutional Features for Crack Detection , 2019, IEEE Transactions on Image Processing.

[11]  José Manuel Menéndez,et al.  A Simplified Computer Vision System for Road Surface Inspection and Maintenance , 2016, IEEE Transactions on Intelligent Transportation Systems.

[12]  Chang-Soo Han,et al.  Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel , 2007 .

[13]  Jeong Ho Lee,et al.  Bridge inspection robot system with machine vision , 2009 .

[14]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[15]  Xin Zhou,et al.  The Aircraft Skin Crack Inspection Based on Different-Source Sensors and Support Vector Machines , 2016 .

[16]  Hojjat Adeli,et al.  Neural Networks in Civil Engineering: 1989–2000 , 2001 .

[17]  Giuseppe Loprencipe,et al.  Ride Quality Due to Road Surface Irregularities: Comparison of Different Methods Applied on a Set of Real Road Profiles , 2017 .

[18]  Hojjat Adeli,et al.  Perceptron Learning in Engineering Design , 2008 .

[19]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[20]  Giuseppe Loprencipe,et al.  Technical and Economic Criteria to Select Pavement Surfaces of Port Handling Plants , 2019, Coatings.

[21]  Jian Zhou,et al.  Wavelet-based pavement distress detection and evaluation , 2003 .

[22]  Gordon Morison,et al.  A Convolutional Neural Network for Pavement Surface Crack Segmentation Using Residual Connections and Attention Gating , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[23]  Edward Sazonov,et al.  Wireless intelligent sensor network for autonomous structural health monitoring , 2004, SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[24]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[25]  Keke Zhang,et al.  YOLOv3-Lite: A Lightweight Crack Detection Network for Aircraft Structure Based on Depthwise Separable Convolutions , 2019, Applied Sciences.

[26]  Paulo Lobato Correia,et al.  Road surface crack detection: Improved segmentation with pixel-based refinement , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[27]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[28]  Guiyuan Jiang,et al.  Automatic Pixel-Level Pavement Crack Detection Using Information of Multi-Scale Neighborhoods , 2018, IEEE Access.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  ChaYoung-Jin,et al.  Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks , 2017 .

[31]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[32]  G. Loprencipe,et al.  Development of a road asset management system in Kazakhstan , 2017 .

[33]  Qingquan Li,et al.  Novel Approach to Pavement Image Segmentation Based on Neighboring Difference Histogram Method , 2008, 2008 Congress on Image and Signal Processing.

[34]  L. Yang,et al.  Ongoing tests and improvements of the MPS algorithm for the automatic crack detection within grey level pavement images , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[35]  Gordon Morison,et al.  A Deep Convolutional Neural Network for Semantic Pixel-Wise Segmentation of Road and Pavement Surface Cracks , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[36]  Paola Di Mascio,et al.  Implementation of a pavement management system for maintenance and rehabilitation of airport surfaces , 2019, Case Studies in Construction Materials.

[37]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[38]  Weihua Sheng,et al.  A Robotic Crack Inspection and Mapping System for Bridge Deck Maintenance , 2014, IEEE Transactions on Automation Science and Engineering.

[39]  Giuseppe Loprencipe,et al.  Analysis of Pavement Condition Survey Data for Effective Implementation of a Network Level Pavement Management Program for Kazakhstan , 2019, Sustainability.

[40]  Qingquan Li,et al.  Path voting based pavement crack detection from laser range images , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[41]  G. M. Sayeed Ahmed,et al.  Experimental Evaluation of Interfacial Surface Cracks in Friction Welded Dissimilar Metals through Image Segmentation Technique (IST) , 2018, Materials.

[42]  Giuseppe Loprencipe,et al.  A Specified Procedure for Distress Identification and Assessment for Urban Road Surfaces Based on PCI , 2017 .

[43]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[44]  Binqiang Chen,et al.  Health State Monitoring of Bladed Machinery with Crack Growth Detection in BFG Power Plant Using an Active Frequency Shift Spectral Correction Method , 2017, Materials.

[45]  Yu Liu,et al.  Automatic Pavement Crack Detection by Multi-Scale Image Fusion , 2019, IEEE Transactions on Intelligent Transportation Systems.

[46]  Hongjun Chen,et al.  Navigating a Miniature Crawler Robot for Engineered Structure Inspection , 2008, IEEE Transactions on Automation Science and Engineering.

[47]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[48]  Fan Yang,et al.  Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection , 2019, IEEE Transactions on Intelligent Transportation Systems.

[49]  Xiang Bai,et al.  Richer Convolutional Features for Edge Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[51]  Asim Karim,et al.  Neural Network Model for Optimization of Cold-Formed Steel Beams , 1997 .

[52]  Zhun Fan,et al.  Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network , 2018, ArXiv.

[53]  Yanliang Gu,et al.  Automatic Crack Detection and Segmentation Using a Hybrid Algorithm for Road Distress Analysis , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[54]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[55]  Paulo Lobato Correia,et al.  CrackIT — An image processing toolbox for crack detection and characterization , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[56]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[57]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[58]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[59]  Andrzej Rybarczyk,et al.  Asphalt surfaced pavement cracks detection based on histograms of oriented gradients , 2015, 2015 22nd International Conference Mixed Design of Integrated Circuits & Systems (MIXDES).

[60]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[62]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[63]  Manuel Avila,et al.  Free-form anisotropy: A new method for crack detection on pavement surface images , 2011, 2011 18th IEEE International Conference on Image Processing.

[64]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[65]  Yuji Iwahori,et al.  A Method of Data Augmentation for Classifying Road Damage Considering Influence on Classification Accuracy , 2019, KES.

[66]  Nii O. Attoh-Okine,et al.  Evaluating Pavement Cracks with Bidimensional Empirical Mode Decomposition , 2008, EURASIP J. Adv. Signal Process..

[67]  Alistair A. Young,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2017, MICCAI 2017.

[68]  Anthony J. Yezzi,et al.  Detecting Curves with Unknown Endpoints and Arbitrary Topology Using Minimal Paths , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Charles R. Farrar,et al.  STRUCTURAL HEALTH MONITORING AT LOS ALAMOS NATIONAL LABORATORY , 2000 .

[70]  Hojjat Adeli,et al.  An Adaptive Conjugate Gradient Neural Network–Wavelet Model for Traffic Incident Detection , 2000 .

[71]  M. Y. Shahin,et al.  Pavement Management for Airports, Roads, and Parking Lots , 2006 .

[72]  Bo Shaobo,et al.  Pavement Crack Detection and Analysis for High-grade Highway , 2007, 2007 8th International Conference on Electronic Measurement and Instruments.

[73]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  W. Chiu,et al.  Experimental and Computational Studies on the Scattering of an Edge-Guided Wave by a Hidden Crack on a Racecourse Shaped Hole , 2017, Materials.

[75]  Jérôme Idier,et al.  Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection , 2016, IEEE transactions on intelligent transportation systems (Print).

[76]  Paulo Lobato Correia,et al.  Automatic road crack segmentation using entropy and image dynamic thresholding , 2009, 2009 17th European Signal Processing Conference.

[77]  Hui-li Zhao,et al.  Improvement of canny algorithm based on pavement edge detection , 2010, 2010 3rd International Congress on Image and Signal Processing.

[78]  Giuseppe Cantisani,et al.  Evaluation methods for improving surface geometry of concrete floors: A case study , 2015 .

[79]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[80]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[81]  Stephan Dahlke,et al.  Coorbit Theory, Multi--Modulation Frames, and the Concept of Joint Sparsity for Medical Multichannel Data Analysis , 2008, EURASIP J. Adv. Signal Process..

[82]  Richard Kronland-Martinet,et al.  A real-time algorithm for signal analysis with the help of the wavelet transform , 1989 .