3D InspectionNet: a deep 3D convolutional neural networks based approach for 3D defect detection on concrete columns

Deep learning-based defect feature recognition from 2D image datasets, has recently been a very active research area and deep Convolutional Neural Networks have brought breakthroughs toward object detection and recognition. Due to CNN’s outstanding performance, several recent studies applied it for defect detection in either routine or post-earthquake infrastructure inspections and have reported competitive performance and potential toward automating infrastructure safety assessment. Despite their benefits, the majority of 2D approaches do not leverage or provide 3D depth information directly from the content of the images and as a result do not enable the 3D measurement of severity of these defects. With the increased popularity of 3D scanning and reconstruction technologies, there is pressing need for defect recognition models that operate on 3D data. In this paper, a novel framework using Deep 3D Convolutional Neural networks (3DCNNs) termed 3D InspectionNet is introduced to learn 3D defects features from an artificially generated 3D dataset, intended to mimic defects on the surface of concrete columns such as either cracks or spalls. InspectionNet has the capability of learning the distribution of complex defect features from a large 3D dataset, and distinguishing defects features. For training 3D InspectionNet, a large simulated 3D defect dataset of 3D CAD models was automatically constructed with labeled defect features. The proposed framework can distinguish defect features from the geometric data such as voxels with a high accuracy. The results of this preliminary work demonstrate and emphasize the feasibility and potentials of this approach for 3D defect detection in automated inspection applications.

[1]  Reinhold Huber-Mörk,et al.  Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images , 2014, ISVC.

[2]  Carl T. Haas,et al.  A man-machine balanced rapid object model for automation of pavement crack sealing and maintenance , 2002 .

[3]  Ioannis Brilakis,et al.  Innovative Stereo Vision-Based Approach to Generate Dense Depth Map of Transportation Infrastructure , 2011 .

[4]  Abdolreza Joghataie,et al.  Neuro-Skins: Dynamics, Plasticity and Effect of Neuron Type and Cell Size on Their Response , 2019, Neural Processing Letters.

[5]  Jeong Ho Lee,et al.  Bridge inspection robot system with machine vision , 2009 .

[6]  Devin K. Harris,et al.  Automated construction of bridge condition inventory using natural language processing and historical inspection reports , 2019, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[7]  AhmedMahmoud,et al.  Toward low-cost 3D automatic pavement distress surveying: the close range photogrammetry approach , 2011 .

[8]  Mohammad R. Jahanshahi,et al.  An innovative methodology for detection and quantification of cracks through incorporation of depth perception , 2011, Machine Vision and Applications.

[9]  Feniosky Peña-Mora,et al.  Integrated Sequential As-Built and As-Planned Representation with D4AR Tools in Support of Decision-Making Tasks in the AEC/FM Industry , 2011 .

[10]  Sami F. Masri,et al.  A new methodology for non-contact accurate crack width measurement through photogrammetry for automated structural safety evaluation , 2013 .

[11]  Noah Snavely.  Photo Tourism : Exploring image collections in 3D , 2006 .

[12]  Ioannis Brilakis,et al.  Machine Vision-Based Concrete Surface Quality Assessment , 2010 .

[13]  Paul W. Fieguth,et al.  Morphological segmentation and classification of underground pipe images , 2006, Machine Vision and Applications.

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sherif Yehia,et al.  PCA-Based algorithm for unsupervised bridge crack detection , 2006, Adv. Eng. Softw..

[16]  T. Saar,et al.  Automatic Asphalt pavement crack detection and classification using Neural Networks , 2010, 2010 12th Biennial Baltic Electronics Conference.

[17]  Prakhar Jaiswal,et al.  FeatureNet: Machining feature recognition based on 3D Convolution Neural Network , 2018, Comput. Aided Des..

[18]  Yun Yong Kim,et al.  Automated image processing technique for detecting and analysing concrete surface cracks , 2013 .

[19]  P. Rivard,et al.  Damage assessment for concrete structure using image processing techniques on acoustic borehole imagery , 2009 .

[20]  Devin K. Harris,et al.  Robust Pixel-Level Crack Detection Using Deep Fully Convolutional Neural Networks , 2019, J. Comput. Civ. Eng..

[21]  James H. Garrett,et al.  Visual Pattern Recognition Supporting Defect Reporting and Condition Assessment of Wastewater Collection Systems , 2009 .

[22]  Mani Golparvar-Fard,et al.  Remote assessment of pre- And post-disaster critical physical infrastructures using mobile workstation chariot and D4AR models , 2019 .

[23]  Mani Golparvar-Fard,et al.  Pothole Properties Measurement through Visual 2D Recognition and 3D Reconstruction , 2012 .

[24]  Mojtaba Farrokh,et al.  Adaptive simulation of hysteresis using neuro-Madelung model , 2016 .

[25]  Abdolreza Joghataie,et al.  Transforming Results from Model to Prototype of Concrete Gravity Dams Using Neural Networks , 2011 .

[26]  Abdolreza Joghataie,et al.  Nonlinear Analysis of Concrete Gravity Dams by Neural Networks , 2009 .

[27]  Andrea Bergamini,et al.  TECHNICAL DESIGN NOTE: High precision measurement of surface cracks using an optical system , 2009 .

[28]  Toshio Abe,et al.  System integration of road-crack evaluation system , 1993, Electronic Imaging.

[29]  Min C. Shin,et al.  Detection of cracks in nuclear power plant using spatial-temporal grouping of local patches , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[30]  Yoshihiko Hamamoto,et al.  A Robust Method for Automatically Detecting Cracks on Noisy Concrete Surfaces , 2009, IEA/AIE.

[31]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[32]  Reginald DesRoches,et al.  Machine Vision Enhanced Post-Earthquake Inspection , 2011 .

[33]  Devin K. Harris,et al.  Increasing the robustness of material-specific deep learning models for crack detection across different materials , 2020 .

[34]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[35]  Mani Golparvar-Fard,et al.  Automated Progress Monitoring Using Unordered Daily Construction Photographs and IFC-Based Building Information Models , 2015, J. Comput. Civ. Eng..

[36]  Mohammad R. Jahanshahi,et al.  Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection , 2018 .

[37]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[38]  Silvio Savarese,et al.  Application of D4AR - A 4-Dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication , 2009, J. Inf. Technol. Constr..

[39]  Bo Zhang,et al.  A portable digital detection technique of building surface crack , 2011, Applied Optics and Photonics China.

[40]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[41]  Satoshi Abe,et al.  A High-Speed Image Processor for Detection of Pavement Cracks , 1992, MVA.

[42]  Sami F. Masri,et al.  Adaptive vision-based crack detection using 3D scene reconstruction for condition assessment of structures , 2012 .

[43]  Kelvin C. P. Wang,et al.  Classification of Pavement Surface Distress with an Embedded Neural Net Chip , 1998 .

[44]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[45]  Abdolreza Joghataie,et al.  Modeling Hysteretic Deteriorating Behavior Using Generalized Prandtl Neural Network , 2015 .

[46]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Mohamed S Kaseko,et al.  COMPARISON OF TRADITIONAL AND NEURAL CLASSIFIERS FOR PAVEMENT-CRACK DETECTION. , 1994 .

[48]  Ioannis Brilakis,et al.  Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation , 2011 .

[49]  Christian Koch,et al.  Automated Pothole Distress Assessment Using Asphalt Pavement Video Data , 2013 .

[50]  Nasser Kehtarnavaz,et al.  Image-based expert-system approach to distress detection on CRC pavement , 1994 .

[51]  Reginald DesRoches,et al.  Machine Vision-Enhanced Postearthquake Inspection , 2011, J. Comput. Civ. Eng..

[52]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).