CorrDetector: A Framework for Structural Corrosion Detection from Drone Images using Ensemble Deep Learning

In this paper, we propose a new technique that applies automated image analysis in the area of structural corrosion monitoring and demonstrate improved efficacy compared to existing approaches. Structural corrosion monitoring is the initial step of the risk-based maintenance philosophy and depends on an engineer’s assessment regarding the risk of building failure balanced against the fiscal cost of maintenance. This introduces the opportunity for human error which is further complicated when restricted to assessment using drone captured images for those areas not reachable by humans due to many background noises. The importance of this problem has promoted an active research community aiming to support the engineer through the use of artificial intelligence (AI) image analysis for corrosion detection. In this paper, we advance this area of research with the development of a framework, CorrDetector. CorrDetector uses a novel ensemble deep learning approach underpinned by convolutional neural networks (CNNs) for structural identification and corrosion feature extraction. We provide an empirical evaluation using real-world images of a complicated structure (e.g. telecommunication tower) captured by drones, a typical scenario for engineers. Our study demonstrates that the ensemble approach of CorrDetector significantly outperforms the state-of-the-art in terms of classi∗Corresponding Author Email addresses: fforkan@swin.edu.au (Abdur Rahim Mohammad Forkan), ykang@swin.edu.au (Yong-Bin Kang), pjayaraman@swin.edu.au (Prem Prakash Jayaraman), Kewen.Liao@acu.edu.au (Kewen Liao), rkaul@swin.edu.au (Rohit Kaul), graham.morgan@newcastle.ac.uk (Graham Morgan), raj.ranjan@ncl.ac.uk (Rajiv Ranjan), samir@robonomics.ai (Samir Sinha) Preprint submitted to Expert Systems with Applications July 13, 2021 ar X iv :2 10 2. 04 68 6v 1 [ cs .C V ] 9 F eb 2 02 1

[1]  Margarita R. Gamarra Acosta,et al.  An innovative image-processing model for rust detection using Perlin Noise to simulate oxide textures , 2014 .

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Ying Gu,et al.  A novel hybrid approach for crack detection , 2020, Pattern Recognit..

[4]  Prem Prakash Jayaraman,et al.  Healthcare 4.0: A review of frontiers in digital health , 2019, WIREs Data Mining Knowl. Discov..

[5]  Gorjan Alagic,et al.  #p , 2019, Quantum information & computation.

[6]  Reza Javaherdashti,et al.  How corrosion affects industry and life , 2000 .

[7]  Ramana M. Pidaparti,et al.  Structural Corrosion Health Assessment using Computational Intelligence Methods , 2007 .

[8]  Carlos Fernández-Isla,et al.  Automated Visual Inspection of Ship Hull Surfaces Using the Wavelet Transform , 2013 .

[9]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[10]  P. Alam.  ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[11]  Thomas S. Huang,et al.  A framework for grid-based image retrieval , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[12]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[13]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[14]  J. Telegdi,et al.  Microbiologically influenced corrosion (MIC) , 2017 .

[15]  Mohammad R. Jahanshahi,et al.  Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection , 2018 .

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Vikram Pakrashi,et al.  ROC dependent event isolation method for image processing based assessment of corroded harbour structures , 2010 .

[18]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  I. Maglogiannis,et al.  Ratsnake: A Versatile Image Annotation Tool with Application to Computer-Aided Diagnosis , 2014, TheScientificWorldJournal.

[20]  B. F. Spencer,et al.  Vision-based Structural Inspection using Multiscale Deep Convolutional Neural Networks , 2018, ArXiv.

[21]  Kamilia Kamardin,et al.  Offline Signature Verification using Deep Learning Convolutional Neural Network (CNN) Architectures GoogLeNet Inception-v1 and Inception-v3 , 2019, Procedia Computer Science.

[22]  Francisco Bonnin-Pascual,et al.  Corrosion Detection for Automated Visual Inspection , 2014 .

[23]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[24]  Luca Petricca,et al.  CORROSION DETECTION USING A.I. : A COMPARISON OF STANDARD COMPUTER VISION TECHNIQUES AND DEEP LEARNING MODEL , 2016 .

[25]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[26]  P. Alam.  ‘N’ , 2021, Composites Engineering: An A–Z Guide.

[27]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[28]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[29]  Mingyang Li,et al.  Ensemble deep learning for automated visual classification using EEG signals , 2020, Pattern Recognit..

[30]  S. F. Masri,et al.  Effect of Color Space, Color Channels, and Sub-Image Block Size on the Performance of Wavelet-Based Texture Analysis Algorithms: An Application to Corrosion Detection on Steel Structures , 2013 .

[31]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Long Chen,et al.  A survey on deep learning methods for scene flow estimation , 2020, Pattern Recognit..

[33]  Ahmad Afshar,et al.  Automatic detection and classification of the ceramic tiles' surface defects , 2017, Pattern Recognit..

[34]  Changmin Kim,et al.  Rapid and automated determination of rusted surface areas of a steel bridge for robotic maintenance systems , 2014 .

[35]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[36]  Aboelmagd Noureldin,et al.  Wavelet Transform for Structural Health Monitoring: A Compendium of Uses and Features , 2006 .