A neural network ensemble method for effective crack segmentation using fully convolutional networks and multi-scale structured forests

Crack image segmentation has recently become a major research topic in nondestructive inspection. However, the image segmentation methods are not robust to variations such as illumination, weather, noise and the segmentation accuracy which cannot meet the requirements of practical applications. Therefore, a neural network ensemble method is proposed for effective crack segmentation in this paper, which consists of fully convolution networks (FCN) and multi-scale structured forests for edge detection (SFD). In order to improve the accuracy of crack segmentation and reduce the error mark under complex background, a new network model based on FCN model is proposed to address the problems that lose local information and the capacity of partial refinement, which are frequently encountered in FCN model in the crack segmentation. In addition, SFD is combined with the half-reconstruction method of anti-symmetrical bi-orthogonal wavelet to overcome the limitation of crack edge detection. Finally, the result of the two maps is merged after resizing to the original image dimensions. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than certain existing methods for crack segmentation.

[1]  S. M. Elaraby,et al.  Welding defect detection from radiography images with a cepstral approach , 2011 .

[2]  Aysin Ertüzün,et al.  An efficient method for texture defect detection: sub-band domain co-occurrence matrices , 2000, Image Vis. Comput..

[3]  Umapada Pal,et al.  Automatic Bridge Crack Detection - A Texture Analysis-Based Approach , 2014, ANNPR.

[4]  Desire Sidibé,et al.  Multiple features extraction for timber defects detection and classification using SVM , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[5]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[6]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[7]  Dongming Zhang,et al.  Deep learning based image recognition for crack and leakage defects of metro shield tunnel , 2018, Tunnelling and Underground Space Technology.

[8]  Somying Thainimit,et al.  Crack detection on unwashed eggs using image processing , 2017 .

[9]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[11]  Chuang Zhang,et al.  Image Enhancement Based on Anti-symmetrical Biorthogonal Wavelet Recontruction: Image Enhancement Based on Anti-symmetrical Biorthogonal Wavelet Recontruction , 2010 .

[12]  Yang Liu,et al.  Deep Learning-Based Fully Automated Pavement Crack Detection on 3D Asphalt Surfaces with an Improved CrackNet , 2018, J. Comput. Civ. Eng..

[13]  Nicola Ancona,et al.  Rail corrugation detection by Gabor filtering , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[14]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  He,et al.  A Method of Cracks Image Segmentation Based on the Means of Multiple Thresholds , 2012 .

[17]  Sung-Han Sim,et al.  Comparative analysis of image binarization methods for crack identification in concrete structures , 2017 .

[18]  Yankun Peng,et al.  A machine vision system for identification of micro-crack in egg shell , 2012 .

[19]  Jaime A. Camelio,et al.  Real-time fault detection in manufacturing environments using face recognition techniques , 2010, Journal of Intelligent Manufacturing.

[20]  Bin Lin,et al.  Automated inspection of engineering ceramic grinding surface damage based on image recognition , 2013 .

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[23]  Anan Banharnsakun,et al.  Hybrid ABC-ANN for pavement surface distress detection and classification , 2017, Int. J. Mach. Learn. Cybern..

[24]  Qingguo Tian,et al.  A fast adaptive crack detection algorithm based on a double-edge extraction operator of FSM , 2019, Construction and Building Materials.

[25]  Yoshihiko Hamamoto,et al.  A robust automatic crack detection method from noisy concrete surfaces , 2011, Machine Vision and Applications.

[26]  Mohammed Bennamoun,et al.  Optimal Gabor filters for textile flaw detection , 2002, Pattern Recognit..

[27]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  N. Radhakrishnan,et al.  Surface Defect Detection of Rubber Oil Seals Based on Texture Analysis , 2013 .

[29]  Shuji Hashimoto,et al.  Fast crack detection method for large-size concrete surface images using percolation-based image processing , 2010, Machine Vision and Applications.

[30]  Pitikhate Sooraksa,et al.  Rapid detection of hairline cracks on the surface of piezoelectric ceramics , 2013 .

[31]  Ling Wang,et al.  Crack detection in magnetic tile images using nonsubsampled shearlet transform and envelope gray level gradient , 2017 .

[32]  Javier Gámez García,et al.  A machine vision system for defect characterization on transparent parts with non-plane surfaces , 2010, Machine Vision and Applications.

[33]  Eric Backer,et al.  In memoriam King-Sun FU (1930-1985) , 1985, Pattern Recognit. Lett..

[34]  Liqiang Nie,et al.  Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm , 2016, Neurocomputing.

[35]  Yigang He,et al.  A cost-effective and automatic surface defect inspection system for hot-rolled flat steel , 2016 .

[36]  Fan Meng,et al.  Pavement Crack Detection Using Sketch Token , 2018, ITQM.

[37]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  A. Kehoe,et al.  Image processing for industrial radiographic inspection : image enhancement , 1990 .

[39]  Siddhartha Kumar Khaitan,et al.  Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection , 2017 .

[40]  Rachel A. Tomlinson,et al.  An autonomous surface discontinuity detection and quantification method by digital image correlation and phase congruency , 2017 .

[41]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  S. Divya,et al.  Mathematical morphology and bottom-hat filtering approach for crack detection on relay surfaces , 2013, INTERNATIONAL CONFERENCE ON SMART STRUCTURES AND SYSTEMS - ICSSS'13.

[43]  J. M. Palomares,et al.  Efficient pavement crack detection and classification , 2017, EURASIP Journal on Image and Video Processing.

[44]  ChaYoung-Jin,et al.  Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks , 2017 .

[45]  Xinnan Fan,et al.  A novel underwater dam crack detection algorithm based on sonar images , 2016 .

[46]  Cao Vu Dung,et al.  Autonomous concrete crack detection using deep fully convolutional neural network , 2019, Automation in Construction.

[47]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[48]  B. Suvdaa,et al.  Steel Surface Defects Detection and Classification Using SIFT and Voting Strategy , 2012 .

[49]  Nhat-Duc Hoang,et al.  Image Processing-Based Classification of Asphalt Pavement Cracks Using Support Vector Machine Optimized by Artificial Bee Colony , 2018, J. Comput. Civ. Eng..

[50]  Tony Lindeberg,et al.  An automatic assessment scheme for steel quality inspection , 2000, Machine Vision and Applications.

[51]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..