Deep learning-based visual crack detection using Google Street View images

In this study, the utility of using Google Street View (GSV) for evaluating the quality of pavement is investigated. A convolutional neural network (CNN) is developed to perform image classification on GSV pavement images. Pavement images are extracted from GSV and then divided into smaller image patches to form data sets. Each image patch is visually classified into different categories of pavement cracks based on the standard practice. A comparative study of pavement quality assessment is conducted between the results of the CNN classified image patches obtained from GSV and those from a sophisticated commercial visual inspection company. The result of the comparison indicates the feasibility and effectiveness of using GSV images for pavement evaluation. The trained network is then tested on a new data set. This study shows that the designed CNN helps classify the pavement images into different defined crack categories.

[1]  Tan Liu,et al.  A Review of Machine Vision-Based Structural Health Monitoring: Methodologies and Applications , 2016, J. Sensors.

[2]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[3]  Dongming Zhang,et al.  Deep learning based image recognition for crack and leakage defects of metro shield tunnel , 2018, Tunnelling and Underground Space Technology.

[4]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  H. Abdi,et al.  Principal component analysis , 2010 .

[7]  Eugenio Culurciello,et al.  An Analysis of Deep Neural Network Models for Practical Applications , 2016, ArXiv.

[8]  Hyeonjoon Moon,et al.  Background Information of Deep Learning for Structural Engineering , 2017 .

[9]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[10]  Li Li,et al.  Automated Pavement Distress Detection and Deterioration Analysis Using Street View Map , 2020, IEEE Access.

[11]  Ignacio Parra,et al.  Adaptive Road Crack Detection System by Pavement Classification , 2011, Sensors.

[12]  Mubarak Shah,et al.  Accurate Image Localization Based on Google Maps Street View , 2010, ECCV.

[13]  Michał Grochowski,et al.  Data augmentation for improving deep learning in image classification problem , 2018, 2018 International Interdisciplinary PhD Workshop (IIPhDW).

[14]  Shengen Yan,et al.  Deep Image: Scaling up Image Recognition , 2015, ArXiv.

[15]  Christoph Mertz,et al.  Vision for road inspection , 2014, IEEE Winter Conference on Applications of Computer Vision.

[16]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[17]  David C. Hogg,et al.  Deeper Networks for Pavement Crack Detection , 2017 .

[18]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[19]  Julien O. Teitler,et al.  Using Google Street View to audit neighborhood environments. , 2011, American journal of preventive medicine.

[20]  Paulo Trigueiros,et al.  High Resolution Trichromatic Road Surface Scanning with a Line Scan Camera and Light Emitting Diode Lighting for Road-Kill Detection , 2016, Sensors.

[21]  Yong Jae Lee,et al.  Style-Aware Mid-level Representation for Discovering Visual Connections in Space and Time , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Ji Yeong Lee,et al.  Intelligent Bridge Inspection Using Remote Controlled Robot and Image Processing Technique , 2011 .

[23]  Carlo Luschi,et al.  Revisiting Small Batch Training for Deep Neural Networks , 2018, ArXiv.

[24]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[25]  John S. Zelek,et al.  Fully Automated Road Defect Detection Using Street View Images , 2017, 2017 14th Conference on Computer and Robot Vision (CRV).

[26]  Yaroslav Bulatov,et al.  Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks , 2013, ICLR.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Aaron Shapiro,et al.  Street-level: Google Street View’s abstraction by datafication , 2018, New Media Soc..

[29]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[32]  Marc Maguire,et al.  Bridge inspection: human performance, unmanned aerial systems and automation , 2018, Journal of Civil Structural Health Monitoring.

[33]  David Lattanzi,et al.  Review of Robotic Infrastructure Inspection Systems , 2017 .

[34]  Enrico Zio,et al.  Artificial intelligence for fault diagnosis of rotating machinery: A review , 2018, Mechanical Systems and Signal Processing.

[35]  Gaurav S. Sukhatme,et al.  Multi-image stitching and scene reconstruction for evaluating defect evolution in structures , 2011 .

[36]  Min Zhang,et al.  Research on Baidu Street View Road Crack Information Extraction Based on Deep Learning Method , 2020 .

[37]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[38]  James M. W. Brownjohn,et al.  Review of machine-vision based methodologies for displacement measurement in civil structures , 2018 .

[39]  Michal Havlena,et al.  From Google Street View to 3D city models , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.