Detection and localization of potholes in thermal images using deep neural networks

A pothole is a depression caused on roads due to seepage of water into soil structure or weight of continuously moving traffic. This not only damages the suspension of the vehicles but is also a prime reason for road accidents worldwide. This necessitates the need to develop an efficient automatic pothole detection system which can assist concerned authorities for timely repair and maintenance of the roads. This paper proposes a novel approach of bounding box based pothole localization from thermal images using deep neural networks. The modified ResNet34-single shot multibox detector gives an average precision of 74.53% whereas modified ResNet50-RetinaNet model provides 91.15% precision. The results obtained by the proposed modified ResNet50-RetinaNet model are the state-of-the-art results for localization of potholes using thermal images. In real-world scenarios such a system can assist relevant authorities to judge the severity of road damage and take appropriate effective measures accordingly.

[1]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Ken Turkowski,et al.  Filters for common resampling tasks , 1990 .

[3]  Ming Liu,et al.  Road Damage Detection Based on Unsupervised Disparity Map Segmentation , 2019, IEEE Transactions on Intelligent Transportation Systems.

[4]  Jangwoo Kwon,et al.  Detection of Potholes Using a Deep Convolutional Neural Network , 2018, J. Univers. Comput. Sci..

[5]  Taehyeong Kim,et al.  Review and Analysis of Pothole Detection Methods , 2014 .

[6]  Khurram Kamal,et al.  Metrology and visualization of potholes using the microsoft kinect sensor , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[7]  Trung-Nghia Le,et al.  Anabranch network for camouflaged object segmentation , 2019, Comput. Vis. Image Underst..

[8]  Youngtae Jo,et al.  Pothole Detection System Using a Black-box Camera , 2015, Sensors.

[9]  Bo Shen,et al.  Fuzzy-Logic-Based Control, Filtering, and Fault Detection for Networked Systems: A Survey , 2015 .

[10]  Christian Koch,et al.  Pothole detection in asphalt pavement images , 2011, Adv. Eng. Informatics.

[11]  Rakesh Chandmal Sharma,et al.  Pothole Detection and Warning System for Indian Roads , 2019, Lecture Notes in Mechanical Engineering.

[12]  Ryan Newton,et al.  The pothole patrol: using a mobile sensor network for road surface monitoring , 2008, MobiSys '08.

[13]  Y. Inoue,et al.  An Acid–Base Controlled Molecular Switch. syn–anti Conformational Switching in a μ-oxo Bis(Iron Porphyrin) , 2003 .

[14]  Wilhelm Burger,et al.  Principles of Digital Image Processing - Core Algorithms , 2009, Undergraduate Topics in Computer Science.

[15]  Muhammad Haroon Yousaf,et al.  Computer vision based detection and localization of potholes in asphalt pavement images , 2016, 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[16]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[17]  Bugao Xu,et al.  A real-time 3D scanning system for pavement distortion inspection , 2009 .

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Kasun De Zoysa,et al.  Automated Pothole Detection System , 2008 .

[20]  Dongmahn Seo,et al.  Detecting a pothole using deep convolutional neural network models for an adaptive shock observing in a vehicle driving , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[21]  Kelvin C. P. Wang,et al.  Experimentation of 3D Pavement Imaging through Stereovision , 2007 .

[22]  Ling Shao,et al.  Camouflaged Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Ioannis Pitas,et al.  Pothole Detection Based on Disparity Transformation and Road Surface Modeling , 2020, IEEE Transactions on Image Processing.

[24]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Varun Gupta,et al.  Convolutional neural networks based potholes detection using thermal imaging , 2019, J. King Saud Univ. Comput. Inf. Sci..

[26]  Hrishikesh Venkataraman,et al.  PotSense: Pothole Detection on Indian Roads using Smartphone Sensors , 2020 .

[27]  Reinhard Klette,et al.  A Multi-frame Stereo Vision-Based Road Profiling Technique for Distress Analysis , 2018, 2018 15th International Symposium on Pervasive Systems, Algorithms and Networks (I-SPAN).

[28]  Chi-Chun Lo,et al.  A Real-Time Pothole Detection Approach for Intelligent Transportation System , 2015 .

[29]  J. Zico Kolter,et al.  Intelligent Pothole Detection and Road Condition Assessment , 2017, ArXiv.

[30]  Girts Strazdins,et al.  Real time pothole detection using Android smartphones with accelerometers , 2011, 2011 International Conference on Distributed Computing in Sensor Systems and Workshops (DCOSS).