Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques

Abstract Sanitary sewer systems are designed to collect and transport sanitary wastewater and stormwater. Pipe inspection is important in identifying both the type and location of pipe defects to maintain the normal sewer operations. Closed-circuit television (CCTV) has been commonly utilized for sewer pipe inspection. Currently, interpretation of the CCTV images is mostly conducted manually to identify the defect type and location, which is time-consuming, labor-intensive and inaccurate. Conventional computer vision techniques are explored for automated interpretation of CCTV images, but such process requires large amount of image pre-processing and the design of complex feature extractor for certain cases. In this study, an automated approach is developed for detecting sewer pipe defects based on a deep learning technique namely faster region-based convolutional neural network (faster R-CNN). The detection model is trained using 3000 images collected from CCTV inspection videos of sewer pipes. After training, the model is evaluated in terms of detection accuracy and computation cost using mean average precision (mAP), missing rate, detection speed and training time. The proposed approach is demonstrated to be applicable for detecting sewer pipe defects accurately with high accuracy and fast speed. In addition, a new model is constructed and several hyper-parameters are adjusted to study the influential factors of the proposed approach. The experiment results demonstrate that dataset size, initialization network type and training mode, and network hyper-parameters have influence on model performance. Specifically, the increase of dataset size and convolutional layers can improve the model accuracy. The adjustment of hyper-parameters such as filter dimensions or stride values contributes to higher detection accuracy, achieving an mAP of 83%. The study lays the foundation for applying deep learning techniques in sewer pipe defect detection as well as addressing similar issues for construction and facility management.

[1]  William T. Freeman,et al.  Latent hierarchical structural learning for object detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Tonghua Su,et al.  Faster R-CNN based autonomous navigation for vehicles in warehouse , 2017, 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[4]  Xiaochun Luo,et al.  Detecting non-hardhat-use by a deep learning method from far-field surveillance videos , 2018 .

[5]  Shivprakash Iyer,et al.  Segmentation of Pipe Images for Crack Detection in Buried Sewers , 2006, Comput. Aided Civ. Infrastructure Eng..

[6]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[7]  Fereidoon Moghadas Nejad,et al.  Image Based Techniques for Crack Detection, Classification and Quantification in Asphalt Pavement: A Review , 2017 .

[8]  Tarek Zayed,et al.  Automated defect detection tool for closed circuit television (cctv) inspected sewer pipelines , 2018 .

[9]  Huaizu Jiang,et al.  Face Detection with the Faster R-CNN , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[10]  Gang Li,et al.  Long-distance precision inspection method for bridge cracks with image processing , 2014 .

[11]  Paul W. Fieguth,et al.  Morphological segmentation and classification of underground pipe images , 2006, Machine Vision and Applications.

[12]  Samir R. Ibadov,et al.  Algorithm for detecting violations of traffic rules based on computer vision approaches , 2017 .

[13]  Paul Fieguth,et al.  Automated detection of cracks in buried concrete pipe images , 2006 .

[14]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[15]  Chunsun Zhang,et al.  An Unmanned Aerial Vehicle‐Based Imaging System for 3D Measurement of Unpaved Road Surface Distresses 1 , 2012, Comput. Aided Civ. Infrastructure Eng..

[16]  Xu Wang,et al.  Traffic Signs Detection Based on Faster R-CNN , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW).

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[19]  Ikhlas Abdel-Qader,et al.  ANALYSIS OF EDGE-DETECTION TECHNIQUES FOR CRACK IDENTIFICATION IN BRIDGES , 2003 .

[20]  Shivprakash Iyer,et al.  A robust approach for automatic detection and segmentation of cracks in underground pipeline images , 2005, Image Vis. Comput..

[21]  Jin-Hee Lee,et al.  ResNet-Based Vehicle Classification and Localization in Traffic Surveillance Systems , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Zhao Lin,et al.  A modified faster R-CNN based on CFAR algorithm for SAR ship detection , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[23]  Chang-Soo Han,et al.  Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel , 2007 .

[24]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[25]  Hubo Cai,et al.  Automatic Detection of Pavement Surface Defects Using Consumer Depth Camera , 2014 .

[26]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Hyuk-Jin Yoon,et al.  Analysis of Crack Image Recognition Characteristics in Concrete Structures Depending on the Illumination and Image Acquisition Distance through Outdoor Experiments , 2016, Sensors.

[28]  Paul Fieguth,et al.  Segmentation of buried concrete pipe images , 2006 .

[29]  Kristin J. Dana,et al.  Automated Crack Detection on Concrete Bridges , 2016, IEEE Transactions on Automation Science and Engineering.

[30]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Takashi Matsumoto,et al.  Development of an Automatic Detector of Cracks in Concrete Using Machine Learning , 2017 .

[32]  Chun Liu,et al.  Automatic quantification of crack patterns by image processing , 2013, Comput. Geosci..

[33]  Weihong Deng,et al.  Very deep convolutional neural network based image classification using small training sample size , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Shuji Hashimoto,et al.  Image‐Based Crack Detection for Real Concrete Surfaces , 2008 .

[36]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[37]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[38]  Xinkai Wu,et al.  Car Detection from Low-Altitude UAV Imagery with the Faster R-CNN , 2017 .

[39]  Liang Lin,et al.  Is Faster R-CNN Doing Well for Pedestrian Detection? , 2016, ECCV.

[40]  Yi Fang,et al.  UAV Low Altitude Photogrammetry for Power Line Inspection , 2016, ISPRS Int. J. Geo Inf..

[41]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[42]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[43]  Mahmoud R. Halfawy,et al.  Efficient Algorithm for Crack Detection in Sewer Images from Closed-Circuit Television Inspections , 2014 .

[44]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[45]  Nikolaos Doulamis,et al.  Deep Convolutional Neural Networks for efficient vision based tunnel inspection , 2015, 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[46]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[47]  Sidney Nascimento Givigi,et al.  Automatic Crack Detection and Measurement Based on Image Analysis , 2016, IEEE Transactions on Instrumentation and Measurement.

[48]  Dan Zecha,et al.  A closer look: Small object detection in faster R-CNN , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[49]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[50]  Ashutosh Bagchi,et al.  Image-based retrieval of concrete crack properties for bridge inspection , 2014 .

[51]  Tara C. Hutchinson,et al.  Image-Based Framework for Concrete Surface Crack Monitoring and Quantification , 2010 .

[52]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[53]  Vipin Chaudhary,et al.  Intervertebral disc detection in X-ray images using faster R-CNN , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[54]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[55]  Xin Yang,et al.  Real-time vehicle detection and tracking in video based on faster R-CNN , 2017 .

[56]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .