Sample and Structure-Guided Network for Road Crack Detection

As an indispensable task for traffic management department, road maintenance has attracted much attention during the last decade due to the rapid development of traffic network. As is known, crack is the early form of many road damages, and repair it in time can significantly save the maintenance cost. In this case, how to detect crack regions quickly and accurately becomes a huge demand. Actually, many image processing technique based methods have been proposed for crack detection, but their performances can not meet our expectations. The reason is that, most of these methods use bottom features such as color and texture to detect the cracks, which are easily influenced by the varied conditions such as light and shadow. Inspired by the great successes of machine learning and artificial intelligence, this paper presents a sample and structure guided network for detecting road cracks. Specifically, the proposed network is based on U-Net architecture, which remains the details from input to output by using skip connection strategy. Then, because the scale of crack samples is much smaller than that of non-crack ones, directly using the conventional cross entropy loss can not optimize the network effectively. In this case, the Focal loss is utilized to address the model optimization problem. Additionally, we incorporate the self-attention strategy into the proposed network, which enhances its stability by encoding the 2-order information among different local regions into the final features. Finally, we test the proposed method on four datasets, three public ones with labels and a photographed one without labels, to validate its effectiveness. It is noteworthy that, for the photographed dataset, we design a series of image processing strategies such as contrast enhancement to improve the generalization capability of the proposed method.

[1]  Sylvie Chambon,et al.  Automatic Road Defect Detection by Textural Pattern Recognition Based on AdaBoost , 2012, Comput. Aided Civ. Infrastructure Eng..

[2]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[3]  Dominique Barba,et al.  A Markov random field for rectilinear structure extraction in pavement distress image analysis , 1995, Proceedings., International Conference on Image Processing.

[4]  T. Saar,et al.  Automatic Asphalt pavement crack detection and classification using Neural Networks , 2010, 2010 12th Biennial Baltic Electronics Conference.

[5]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[6]  Paulo Lobato Correia,et al.  Automatic road crack segmentation using entropy and image dynamic thresholding , 2009, 2009 17th European Signal Processing Conference.

[7]  Hui-li Zhao,et al.  Improvement of canny algorithm based on pavement edge detection , 2010, 2010 3rd International Congress on Image and Signal Processing.

[8]  Yongsheng Dong,et al.  Superpixel-Based Foreground Extraction With Fast Adaptive Trimaps , 2018, IEEE Transactions on Cybernetics.

[9]  Manuel Avila,et al.  Free-form anisotropy: A new method for crack detection on pavement surface images , 2011, 2011 18th IEEE International Conference on Image Processing.

[10]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[11]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[12]  Anthony J. Yezzi,et al.  Detecting Curves with Unknown Endpoints and Arbitrary Topology Using Minimal Paths , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Xiaoqiang Lu,et al.  Muti-stage learning for gender and age prediction , 2019, Neurocomputing.

[14]  S. Chambon,et al.  Automatic Road Pavement Assessment with Image Processing: Review and Comparison , 2011 .

[15]  Xiaoqiang Lu,et al.  Robust Space–Frequency Joint Representation for Remote Sensing Image Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Xuelong Li,et al.  3G structure for image caption generation , 2019, Neurocomputing.

[17]  Dacheng Tao,et al.  Patch Alignment Manifold Matting , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Yong Hu,et al.  A novel LBP based methods for pavement crack detection , 2009 .

[19]  Qingquan Li,et al.  Novel Approach to Pavement Image Segmentation Based on Neighboring Difference Histogram Method , 2008, 2008 Congress on Image and Signal Processing.

[20]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification Using Rearranged Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[22]  Yung-Yu Chuang,et al.  Weakly Supervised Salient Object Detection by Learning A Classifier-Driven Map Generator , 2019, IEEE Transactions on Image Processing.

[23]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[25]  Kyoungson Jhang,et al.  CNN Training for Face Photo based Gender and Age Group Prediction with Camera , 2019, 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[26]  Xiaoqiang Lu,et al.  Deep object tracking with multi-modal data , 2016, 2016 International Conference on Computer, Information and Telecommunication Systems (CITS).

[27]  Jérôme Idier,et al.  Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection , 2016, IEEE transactions on intelligent transportation systems (Print).

[28]  Xuelong Li,et al.  Robust object tracking via diverse templates , 2016, 2016 International Conference on Computer, Information and Telecommunication Systems (CITS).

[29]  Xiaoqiang Lu,et al.  Vision-to-Language Tasks Based on Attributes and Attention Mechanism , 2019, IEEE Transactions on Cybernetics.

[30]  Shuicheng Yan,et al.  Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Nikolaos Doulamis,et al.  Deep Convolutional Neural Networks for efficient vision based tunnel inspection , 2015, 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[32]  Xiaoqiang Lu,et al.  Hierarchical and Robust Convolutional Neural Network for Very High-Resolution Remote Sensing Object Detection , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Jie Fang,et al.  GAN and DCN Based Multi-step Supervised Learning for Image Semantic Segmentation , 2018, PRCV.

[34]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[35]  Xuelong Li,et al.  Convolutional Edge Constraint-Based U-Net for Salient Object Detection , 2019, IEEE Access.

[36]  Yanliang Gu,et al.  Automatic Crack Detection and Segmentation Using a Hybrid Algorithm for Road Distress Analysis , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[37]  Shaocheng Tong,et al.  Fuzzy-Based Multierror Constraint Control for Switched Nonlinear Systems and Its Applications , 2019, IEEE Transactions on Fuzzy Systems.

[38]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Zhiwen Yu,et al.  Quantization-based clustering algorithm , 2010, Pattern Recognit..

[41]  Yan-Jun Liu,et al.  Adaptive neural network-based control for a class of nonlinear pure-feedback systems with time-varying full state constraints , 2018, IEEE/CAA Journal of Automatica Sinica.

[42]  Weihua Sheng,et al.  Developing a crack inspection robot for bridge maintenance , 2011, 2011 IEEE International Conference on Robotics and Automation.

[43]  Zhiwen Yu,et al.  A Bayesian Model for Crowd Escape Behavior Detection , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[44]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[46]  Naoki Tanaka,et al.  A Crack Detection Method in Road Surface Images Using Morphology , 1998, MVA.