Automatic pixel-wise detection of evolving cracks on rock surface in video data

Abstract Accurately detecting the presence and evolving boundaries of cracks on rock surfaces is critical for understanding the behavior of crack evolutions and facture mechanism of rock and rock-like material, which could cause engineering disasters if proper operation were not taken to deal with the evolving cracks. In this paper, we investigate the problem of vision-based automatic detection of cracks on rock surface at pixel-level, which is a preliminary step of crack evolution analysis. We build a Split Hopkinson Pressure Bar (SHPB) system to simulate the crack evolution process and capture the process as video data using a high frame camera, where a dataset of evolving cracks is created consisting of rock crack images that are manually labeled in pixel-level granularity. We propose a two-stage method to detect cracks in video data: the first stage employs Convolution Neural Network (CNN) based deep learning method to obtain preliminary results for each image frame while the second stage relies on novel variant Bayesian Inference to further refine the detection results. Specifically, in the first stage, a variant of U-Net model (denoted as CrackUNet) is developed to obtain intermediate classifications (crack or non-crack) that can better combine with other processing techniques for further improvement. Then in the second stage, a novel Spatial-Temporal Bayesian Inference (STBI) method is developed to further improve detection accuracy by taking advantages of the spatial and temporal correlations of the evolving cracks in video data. Experimental results show that the proposed method outperforms all the baselines.

[1]  S A Velinsky,et al.  Histogram‐Based Approach for Automated Pavement‐Crack Sensing , 1992 .

[2]  Shuangjiu Xiao,et al.  Sketch-Based Image Retrieval via Compact Binary Codes Learning , 2018, ICONIP.

[3]  ArenaAlessio,et al.  A new computational approach to cracks quantification from 2D image analysis , 2014 .

[4]  Chiun-lin Wu,et al.  Image analysis method for crack distribution and width estimation for reinforced concrete structures , 2018, Automation in Construction.

[5]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[7]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[8]  Eduardo Zalama Casanova,et al.  Road Crack Detection Using Visual Features Extracted by Gabor Filters , 2014, Comput. Aided Civ. Infrastructure Eng..

[9]  Wenzong Wang,et al.  Segment-based pavement crack quantification , 2019, Automation in Construction.

[10]  J. Lees,et al.  Experimental investigation of crack propagation and crack branching in lightly reinforced concrete beams using digital image correlation , 2017 .

[11]  Heng-Da Cheng,et al.  Real-Time Image Thresholding Based on Sample Space Reduction and Interpolation Approach , 2003 .

[12]  Wei Yao,et al.  Dynamic rock tests using split Hopkinson （Kolsky） bar system - A review , 2015 .

[13]  Jérôme Idier,et al.  Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection , 2016, IEEE transactions on intelligent transportation systems (Print).

[14]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[16]  Takayuki Okatani,et al.  A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks , 2019, Automation in Construction.

[17]  Jung-Wuk Hong,et al.  A vision-based approach for autonomous crack width measurement with flexible kernel , 2020 .

[18]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[19]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Nhat-Duc Hoang,et al.  Automatic detection of asphalt pavement raveling using image texture based feature extraction and stochastic gradient descent logistic regression , 2019, Automation in Construction.

[21]  Robert M. Haralick,et al.  Feature normalization and likelihood-based similarity measures for image retrieval , 2001, Pattern Recognit. Lett..

[22]  Xin Jiang,et al.  Crack Detection from the Slope of the Mode Shape Using Complex Continuous Wavelet Transform , 2012, Comput. Aided Civ. Infrastructure Eng..

[23]  Fan Xi,et al.  Detection crack in image using Otsu method and multiple filtering in image processing techniques , 2016 .

[24]  Manfred Borovcnik,et al.  A Probabilistic Perspective , 1991 .

[25]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[26]  Yoshihiko Hamamoto,et al.  A robust automatic crack detection method from noisy concrete surfaces , 2011, Machine Vision and Applications.

[27]  Qiang Li,et al.  Matched Filtering Algorithm for Pavement Cracking Detection , 2013 .

[28]  Kourosh Shahriar,et al.  Experimental and numerical study of crack propagation and coalescence in pre-cracked rock-like disks , 2014 .

[29]  Gang Li,et al.  Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine , 2017 .

[30]  Dimos Polyzois,et al.  Deep learning-based automatic volumetric damage quantification using depth camera , 2019, Automation in Construction.

[31]  I. Maglogiannis,et al.  Ratsnake: A Versatile Image Annotation Tool with Application to Computer-Aided Diagnosis , 2014, TheScientificWorldJournal.

[32]  Deyu Meng,et al.  Few-Example Object Detection with Model Communication , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Jian Zhao,et al.  Full-field measurement and fracture characterisations of rocks under dynamic loads using high-speed three-dimensional digital image correlation , 2018 .

[34]  Ole Winther,et al.  Convolutional LSTM Networks for Subcellular Localization of Proteins , 2015, AlCoB.

[35]  Qingquan Li,et al.  CrackTree: Automatic crack detection from pavement images , 2012, Pattern Recognit. Lett..

[36]  Wei Wang,et al.  Computer vision-based concrete crack detection using U-net fully convolutional networks , 2019, Automation in Construction.

[37]  Masato Ukai.  Development of Image Processing Technique for Detection of Tunnel Wall Deformation Using Continuously Scanned Image , 2000 .

[38]  Yu Wu,et al.  Progressive Learning for Person Re-Identification With One Example , 2019, IEEE Transactions on Image Processing.

[39]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[40]  Guiyuan Jiang,et al.  Automatic Pixel-Level Pavement Crack Detection Using Information of Multi-Scale Neighborhoods , 2018, IEEE Access.

[41]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[42]  Mohammad Sohel Rahman,et al.  MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation , 2019, Neural Networks.

[43]  Sherif Yehia,et al.  PCA-Based algorithm for unsupervised bridge crack detection , 2006, Adv. Eng. Softw..

[44]  Zhihua Luo,et al.  Extraction of microcracks in rock images based on heuristic graph searching and application , 2015, Comput. Geosci..

[45]  Mohammad R. Jahanshahi,et al.  An innovative methodology for detection and quantification of cracks through incorporation of depth perception , 2011, Machine Vision and Applications.

[46]  Zijun Zhang,et al.  Automatic Detection of Wind Turbine Blade Surface Cracks Based on UAV-Taken Images , 2017, IEEE Transactions on Industrial Electronics.

[47]  Mahmoud Dhimish,et al.  Ultrafast High-Resolution Solar Cell Cracks Detection Process , 2020, IEEE Transactions on Industrial Informatics.

[48]  Yun Liu,et al.  Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring , 2014, Sensors.

[49]  Alessio Arena,et al.  A new computational approach to cracks quantification from 2D image analysis: Application to micro-cracks description in rocks , 2014, Comput. Geosci..

[50]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[51]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[52]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  M. Khanzadi,et al.  A Brief Review and a New Graph-Based Image Analysis for Concrete Crack Quantification , 2019 .

[54]  Rih-Teng Wu,et al.  A texture‐Based Video Processing Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces , 2017, Comput. Aided Civ. Infrastructure Eng..

[55]  Kristin J. Dana,et al.  Automated Crack Detection on Concrete Bridges , 2016, IEEE Transactions on Automation Science and Engineering.

[56]  Jian Zhao,et al.  A Review of Dynamic Experimental Techniques and Mechanical Behaviour of Rock Materials , 2014, Rock Mechanics and Rock Engineering.

[57]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[58]  Kelwin Fernandes,et al.  Pavement pathologies classification using graph-based features , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[59]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[60]  Bernt Schiele,et al.  Semantic Projection Network for Zero- and Few-Label Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).