Bottom-up image detection of water channel slope damages based on superpixel segmentation and support vector machine

Abstract The operation of water supply channels is threatened by the occasionally occurred slope damages. Timely detection of their occurrence is critical for the rapid enforcement of mitigation measures. However, current practices based on routine inspection and structural heath monitoring are inefficient, laborious and tend to be biased. As an attempt to address the limitations, this paper proposes a bottom-up image detection approach for slope damages, which includes four steps, i.e. superpixel segmentation, feature handcrafting, superpixel classification based on support vector machine (SVM), and slope damage recognition. The approach employs a bottom-up strategy to infer the upper-level slope condition from the classification results of individual superpixels in the bottom level. Experiments were conducted to demonstrate the effectiveness of the approach. The handcrafted feature “LBP + HSV” was demonstrated to be effective in characterizing the image features of slope damages. An SVM model with “LBP + HSV” as input can reliably identify the slope condition in superpixels. Based on the SVM model, the bottom-up strategy achieved high recognition performance, of which the overall accuracy can be up to 91.7%. The proposed approach has potential to facilitate the early and comprehensive awareness of slope damages along the entire route of water channel by the integration with unmanned aerial vehicles.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Zhao Zhang,et al.  Dynamic BIM-augmented UAV safety inspection for water diversion project , 2019, Comput. Ind..

[3]  Donghai Liu,et al.  An integrated visualization framework to support whole-process management of water pipeline safety , 2018 .

[4]  Paul Fieguth,et al.  Automated detection of cracks in buried concrete pipe images , 2006 .

[5]  Eung-kon Kim,et al.  Building crack inspection using small UAV , 2015, 2015 17th International Conference on Advanced Communication Technology (ICACT).

[6]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[7]  Russell M. Mersereau,et al.  Critical Assessment of Pavement Distress Segmentation Methods , 2010 .

[8]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[9]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[10]  Markus König,et al.  GPU-Enabled Pavement Distress Image Classification in Real Time , 2017, J. Comput. Civ. Eng..

[11]  Shuang Wang,et al.  Structural Damage Detection using Deep Convolutional Neural Network and Transfer Learning , 2019, KSCE Journal of Civil Engineering.

[12]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[13]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[14]  Rama Chellappa,et al.  Entropy-Rate Clustering: Cluster Analysis via Maximizing a Submodular Function Subject to a Matroid Constraint , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Mahmoud R. Halfawy,et al.  Automated defect detection in sewer closed circuit television images using histograms of oriented gradients and support vector machine , 2014 .

[16]  Zhiliang Ma,et al.  Construction automation and robotics for high-rise buildings over the past decades: A comprehensive review , 2019, Adv. Eng. Informatics.

[17]  Antonio Sgorbissa,et al.  Post‐disaster assessment with unmanned aerial vehicles: A survey on practical implementations and research approaches , 2018, J. Field Robotics.

[18]  Bugao Xu,et al.  Automatic inspection of pavement cracking distress , 2006, J. Electronic Imaging.

[19]  Amir H. Behzadan,et al.  Convolutional neural networks for object detection in aerial imagery for disaster response and recovery , 2020, Adv. Eng. Informatics.

[20]  Jeong Ho Lee,et al.  Bridge inspection robot system with machine vision , 2009 .

[21]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[22]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted Via Energy-Driven Sampling , 2012, International Journal of Computer Vision.

[23]  Sherif Yehia,et al.  PCA-Based algorithm for unsupervised bridge crack detection , 2006, Adv. Eng. Softw..

[24]  Jack Chin Pang Cheng,et al.  Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques , 2018, Automation in Construction.

[25]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[26]  Nathaniel J. C. Libatique,et al.  UAV aerial imaging applications for post-disaster assessment, environmental management and infrastructure development , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).

[27]  Matti Pietikäinen,et al.  Rotation-invariant texture classification using feature distributions , 2000, Pattern Recognit..

[28]  Jitendra Malik,et al.  Learning a classification model for segmentation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Ioannis Brilakis,et al.  Multi-classifier for reinforced concrete bridge defects , 2019, Automation in Construction.

[30]  Xiao Liang,et al.  Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization , 2018, Comput. Aided Civ. Infrastructure Eng..

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[33]  Shirley J. Dyke,et al.  Autonomous image localization for visual inspection of civil infrastructure , 2017 .

[34]  Dongho Kang,et al.  Autonomous UAVs for Structural Health Monitoring Using Deep Learning and an Ultrasonic Beacon System with Geo‐Tagging , 2018, Comput. Aided Civ. Infrastructure Eng..

[35]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[36]  H. Kalke,et al.  Support vector machine learning applied to digital images of river ice conditions , 2018, Cold Regions Science and Technology.

[37]  Ming-Der Yang,et al.  Automated diagnosis of sewer pipe defects based on machine learning approaches , 2008, Expert Syst. Appl..

[38]  Donghai Liu,et al.  Registering georeferenced photos to a building information model to extract structures of interest , 2019, Adv. Eng. Informatics.

[39]  Song Zhu,et al.  Fast superpixel segmentation by iterative edge refinement , 2015 .

[40]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Mani Golparvar-Fard,et al.  Image-Based Automated 3D Crack Detection for Post-disaster Building Assessment , 2014, J. Comput. Civ. Eng..

[42]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[44]  Won-Hwa Hong,et al.  Application of Crack Identification Techniques for an Aging Concrete Bridge Inspection Using an Unmanned Aerial Vehicle , 2018, Sensors.