Comparison of CNN-Based Models for Pothole Detection in Real-World Adverse Conditions: Overview and Evaluation

Potholes pose a significant problem for road safety and infrastructure. They can cause damage to vehicles and present a risk to pedestrians and cyclists. The ability to detect potholes in real time and with a high level of accuracy, especially under different lighting conditions, is crucial for the safety of road transport participants and the timely repair of these hazards. With the increasing availability of cameras on vehicles and smartphones, there is a growing interest in using computer vision techniques for this task. Convolutional neural networks (CNNs) have shown great potential for object detection tasks, including pothole detection. This study provides an overview of computer vision algorithms used for pothole detection. Experimental results are then used to evaluate the performance of the latest CNN-based models for pothole detection in different real-world road conditions, including rain, sunset, evening, and night, as well as clean conditions. The models evaluated in this study include both conventional and the newest architectures from the region-based CNN (R-CNN) and You Only Look Once (YOLO) families. The YOLO models demonstrated a faster detection response and higher accuracy in detecting potholes under clear, rain, sunset, and evening conditions. R-CNN models, on the other hand, performed better in the worse-visibility conditions at night. This study provides valuable insights into the performance of different CNN models for pothole detection in real road conditions and may assist in the selection of the most appropriate model for a specific application.

[1]  Gaoang Wang,et al.  Sustainable Road Pothole Detection: A Crowdsourcing Based Multi-Sensors Fusion Approach , 2023, Sustainability.

[2]  Haichao Zhang,et al.  TRD-YOLO: A Real-Time, High-Performance Small Traffic Sign Detection Algorithm , 2023, Sensors.

[3]  T. Tak,et al.  Image-Based Pothole Detection Using Multi-Scale Feature Network and Risk Assessment , 2023, Electronics.

[4]  Xiangsuo Fan,et al.  Adverse Weather Target Detection Algorithm Based on Adaptive Color Levels and Improved YOLOv5 , 2022, Sensors.

[5]  L. Li,et al.  YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications , 2022, ArXiv.

[6]  Yunyi Jia,et al.  Cloud-Based Collaborative Road-Damage Monitoring with Deep Learning and Smartphones , 2022, Sustainability.

[7]  H. Liao,et al.  YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Habeeb Salaudeen,et al.  Pothole Detection Using Image Enhancement GAN and Object Detection Network , 2022, Electronics.

[9]  Wenshuo Wang,et al.  Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms , 2022, Transportation Safety and Environment.

[10]  Saket S. Chaturvedi,et al.  Pay "Attention" to Adverse Weather: Weather-aware Attention-based Object Detection , 2022, 2022 26th International Conference on Pattern Recognition (ICPR).

[11]  Kuan-Hsien Liu,et al.  Half Wavelet Attention on M-Net+ for Low-Light Image Enhancement , 2022, 2022 IEEE International Conference on Image Processing (ICIP).

[12]  Jesús Requena-Carrión,et al.  A Novel Road Maintenance Prioritisation System Based on Computer Vision and Crowdsourced Reporting , 2022, J. Sens. Actuator Networks.

[13]  Jianke Zhu,et al.  Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions , 2021, AAAI.

[14]  Khaled R. Ahmed,et al.  Smart Pothole Detection Using Deep Learning Based on Dilated Convolution , 2021, Sensors.

[15]  Dong-Eun Lee,et al.  Application of Various YOLO Models for Computer Vision-Based Real-Time Pothole Detection , 2021, Applied Sciences.

[16]  Rami Alkhatib,et al.  Real-Time Pothole Detection Using Deep Learning , 2021, ArXiv.

[17]  Yu-Chen Lin,et al.  Implementation of Pavement Defect Detection System on Edge Computing Platform , 2021, Applied Sciences.

[18]  Arcot Sowmya,et al.  Context-Enhanced Representation Learning for Single Image Deraining , 2021, International Journal of Computer Vision.

[19]  Tieniu Tan,et al.  Selective Wavelet Attention Learning for Single Image Deraining , 2021, International Journal of Computer Vision.

[20]  Wenhan Yang,et al.  Benchmarking Low-Light Image Enhancement and Beyond , 2021, International Journal of Computer Vision.

[21]  Yi Jiang,et al.  Sparse R-CNN: End-to-End Object Detection with Learnable Proposals , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Chao Wu,et al.  An Automated Machine-Learning Approach for Road Pothole Detection Using Smartphone Sensor Data , 2020, Sensors.

[23]  Carlos Pena-Caballero,et al.  Real-Time Road Hazard Information System , 2020, Infrastructures.

[24]  Raoul de Charette,et al.  Rain Rendering for Evaluating and Improving Robustness to Bad Weather , 2020, International Journal of Computer Vision.

[25]  Alberto Ochoa-Zezzatti,et al.  An Asphalt Damage Dataset and Detection System Based on RetinaNet for Road Conditions Assessment , 2020, Applied Sciences.

[26]  Fahad Shahbaz Khan,et al.  D2Det: Towards High Quality Object Detection and Instance Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Lei Xiang,et al.  Multi-Scale Boosted Dehazing Network With Dense Feature Fusion , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[29]  Petr Novotny,et al.  Designating Regional Elements System in a Critical Infrastructure System in the Context of the Czech Republic , 2020, Syst..

[30]  Marin Marinov,et al.  A Study on Recent Developments and Issues with Obstacle Detection Systems for Automated Vehicles , 2020, Sustainability.

[31]  Hanshen Chen,et al.  Pothole detection using location-aware convolutional neural networks , 2020, International Journal of Machine Learning and Cybernetics.

[32]  J. Čelko,et al.  The Economically Active People in the Transport Process , 2019, IOP Conference Series: Materials Science and Engineering.

[33]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[34]  Feifan Lv,et al.  Attention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset , 2019, International Journal of Computer Vision.

[35]  Nuno Vasconcelos,et al.  Cascade R-CNN: High Quality Object Detection and Instance Segmentation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Wei Jiang,et al.  Convolutional neural network for pothole detection in asphalt pavement , 2019, Road Materials and Pavement Design.

[37]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[38]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Yoshihide Sekimoto,et al.  Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images , 2018, Comput. Aided Civ. Infrastructure Eng..

[40]  Li Tao,et al.  LLCNN: A convolutional neural network for low-light image enhancement , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[41]  Jie Ma,et al.  MSR-net: Low-light Image Enhancement Using Deep Convolutional Network , 2017, ArXiv.

[42]  Varun P. Gopi,et al.  Vehicle Vibration Signal Processing for Road Surface Monitoring , 2017, IEEE Sensors Journal.

[43]  Marcin Staniek,et al.  Stereo Vision Method Application to Road Inspection , 2017 .

[44]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[45]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[46]  Chi-Chun Lo,et al.  A Real-Time Pothole Detection Approach for Intelligent Transportation System , 2015 .

[47]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[50]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Naim Dahnoun,et al.  An efficient algorithm for pothole detection using stereo vision , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[52]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  E. Salari,et al.  Pavement pothole detection and severity measurement using laser imaging , 2011, 2011 IEEE INTERNATIONAL CONFERENCE ON ELECTRO/INFORMATION TECHNOLOGY.

[54]  Matús Chochlík,et al.  Dynamic Architecture for Analytical Its Services , 2010, Communications - Scientific letters of the University of Zilina.

[55]  Christopher K. I. Williams,et al.  International Journal of Computer Vision manuscript No. (will be inserted by the editor) The PASCAL Visual Object Classes (VOC) Challenge , 2022 .

[56]  Kelvin C. P. Wang,et al.  Experimentation of 3D Pavement Imaging through Stereovision , 2007 .

[57]  J. K. Liu,et al.  Detection of Pavement Distresses Using 3D Laser Scanning Technology , 2005 .

[58]  Xavier Baró,et al.  CodaLab Competitions: An Open Source Platform to Organize Scientific Challenges , 2023, J. Mach. Learn. Res..

[59]  Hongli Liu,et al.  CEAM-YOLOv7: Improved YOLOv7 Based on Channel Expansion and Attention Mechanism for Driver Distraction Behavior Detection , 2022, IEEE Access.

[60]  J. Čelko,et al.  Morning modal split model of economically active people in Zilina region , 2021, Transportation Research Procedia.

[61]  Huixuan Fu,et al.  Improved YOLOv4 Marine Target Detection Combined with CBAM , 2021, Symmetry.

[62]  M. Lacinák Resilience of the Smart Transport System – Risks and Aims , 2021, Transportation Research Procedia.

[63]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[64]  Moni Naor,et al.  Computer Vision – ECCV 2016 , 2016, Lecture Notes in Computer Science.

[65]  R. Hicks PAY ATTENTION! , 2015, Physician leadership journal.

[66]  David J. Fleet,et al.  Computer Vision – ECCV 2014 , 2014, Lecture Notes in Computer Science.