RCNet: road classification convolutional neural networks for intelligent vehicle system

Vision-based techniques for intelligent vehicles in heterogeneous road environments are gaining significant attention from researchers and industrialists. Unfortunately, the mechanisms in this domain suffer from limited performance due to scene complexity, varying road structure, and improper illumination conditions. These challenging situations may lead an intelligent vehicle into dangerous situations such as collisions or road accidents and may cause higher mortality. The application of intelligent methods and other machine learning techniques for road surface classification is little explored in the existing literature. Thus, we propose a convolutional neural network-based road classification network (RCNet) for the accurate classification of road surfaces. This procedure includes the classification of five major categories of road surfaces: curvy, dry, ice, rough, and wet roads. The experimental results reveal the behavior of the proposed RCNet under various optimizer techniques. The standard performance evaluation measures have been used to test and validate the proposed method on the Oxford RobotCar dataset. RCNet achieves classification accuracy, precision, and sensitivity of 99.90%, and 99.97% of specificity. Results of implemented work are significantly higher than available state-of-the-art techniques and show accurate and effective performance in the complex road environment.

[1]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[2]  Abolfazl Abdollahi,et al.  Semi automatic road extraction from digital images , 2017 .

[3]  Toby P. Breckon,et al.  Automatic Road Environment Classification , 2011, IEEE Transactions on Intelligent Transportation Systems.

[4]  Assessment of groundwater suitability for irrigation in Madinah City, Saudi Arabia , 2015, Arabian Journal of Geosciences.

[5]  Myungchan Roh,et al.  Drivable Road Detection with 3D Point Clouds Based on the MRF for Intelligent Vehicle , 2013, FSR.

[6]  A. Cass,et al.  My experiences with kidney care: A qualitative study of adults in the Northern Territory of Australia living with chronic kidney disease, dialysis and transplantation , 2019, PloS one.

[7]  Junyu Gao,et al.  Embedding structured contour and location prior in siamesed fully convolutional networks for road detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[9]  Ariel Monteserin,et al.  A deep learning approach to automatic road surface monitoring and pothole detection , 2019, Personal and Ubiquitous Computing.

[10]  Guosheng Lin,et al.  Discriminative Training of Deep Fully Connected Continuous CRFs With Task-Specific Loss , 2016, IEEE Transactions on Image Processing.

[11]  Jie Shen,et al.  Road surface condition classification using deep learning , 2019, J. Vis. Commun. Image Represent..

[12]  Kazuhide Okamoto,et al.  Vision-Based Autonomous Path Following Using a Human Driver Control Model With Reliable Input-Feature Value Estimation , 2019, IEEE Transactions on Intelligent Vehicles.

[13]  Dinesh Singh,et al.  Deep Spatio-Temporal Representation for Detection of Road Accidents Using Stacked Autoencoder , 2019, IEEE Transactions on Intelligent Transportation Systems.

[14]  Naijie Gu,et al.  An Encoder-Decoder Based Convolution Neural Network (CNN) for Future Advanced Driver Assistance System (ADAS) , 2017 .

[15]  Junqi Guo,et al.  A novel FMH model for road extraction from high-resolution remote sensing images in urban areas , 2018, IIKI.

[16]  Sivakumar Rajagopal,et al.  A robust anisotropic diffusion filter with low arithmetic complexity for images , 2019, EURASIP J. Image Video Process..

[17]  Patrik Jonsson,et al.  Road Surface Status Classification Using Spectral Analysis of NIR Camera Images , 2015, IEEE Sensors Journal.

[18]  Bo Peng,et al.  Road Segmentation of UAV RS Image Using Adversarial Network with Multi-Scale Context Aggregation , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[19]  Zhitao Xiao,et al.  Combining CNN and MRF for road detection , 2017, Comput. Electr. Eng..

[20]  Marcin Ochman,et al.  Hybrid approach to road detection in front of the vehicle , 2019, IFAC-PapersOnLine.

[21]  Brijesh Verma,et al.  A Neural Ensemble Approach for Segmentation and Classification of Road Images , 2014, ICONIP.

[22]  Zhenmin Tang,et al.  Deep representation learning for road detection using Siamese network , 2018, Multimedia Tools and Applications.

[23]  Yuri Álvarez López,et al.  Synthetic Aperture Radar Imaging System for Landmine Detection Using a Ground Penetrating Radar on Board a Unmanned Aerial Vehicle , 2018, IEEE Access.

[24]  Wu Jun,et al.  Combining CNN and MRF for Road Detection , 2017 .

[25]  Faouzi Alaya Cheikh,et al.  Road surface condition detection utilizing resonance frequency and optical technologies , 2019 .

[26]  Madjid Tavana,et al.  Autonomous vehicles: challenges, opportunities, and future implications for transportation policies , 2016, Journal of Modern Transportation.

[27]  Yanbing Xue,et al.  Semantic image segmentation with fused CNN features , 2017 .

[28]  James M. Rehg,et al.  Vision-Based High-Speed Driving With a Deep Dynamic Observer , 2018, IEEE Robotics and Automation Letters.

[29]  Kang Ryoung Park,et al.  Enhanced Detection and Recognition of Road Markings Based on Adaptive Region of Interest and Deep Learning , 2019, IEEE Access.

[30]  Seoung Bum Kim,et al.  Machine Learning Approaches to Estimate Road Surface Temperature Variation along Road Section in Real-Time for Winter Operation , 2020, Int. J. Intell. Transp. Syst. Res..

[31]  Bernt Schiele,et al.  Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[33]  Mingwu Ren,et al.  Road Boundaries Detection based on Modified Occupancy Grid Map Using Millimeter-wave Radar , 2020, Mob. Networks Appl..

[34]  Mingguo Zhao,et al.  Laser-Based Obstacle Avoidance and Road Quality Detection for Autonomous Bicycles , 2015 .

[35]  Yanjun Huang,et al.  Speed independent road classification strategy based on vehicle response: Theory and experimental validation , 2019, Mechanical Systems and Signal Processing.

[36]  Siddharth Agarwal,et al.  Ground-Edge-Based LIDAR Localization Without a Reflectivity Calibration for Autonomous Driving , 2017, IEEE Robotics and Automation Letters.

[37]  James H. Elder,et al.  Road Segmentation for Classification of Road Weather Conditions , 2016, ECCV Workshops.

[38]  Jian Yang,et al.  Importance-Aware Semantic Segmentation for Autonomous Vehicles , 2019, IEEE Transactions on Intelligent Transportation Systems.

[39]  Hafiz Malik,et al.  LiDAR Data Integrity Verification for Autonomous Vehicle , 2019, IEEE Access.

[40]  Minkwang Lee,et al.  Road boundary detection and tracking for structured and unstructured roads using a 2D lidar sensor , 2014 .

[41]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Cheng-Lin Liu,et al.  Discriminative Feature Selection via Employing Smooth and Robust Hinge Loss , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Junyong You Weather Data Integrated Mask R-CNN for Automatic Road Surface Condition Monitoring , 2019, 2019 IEEE Visual Communications and Image Processing (VCIP).

[44]  Nhat-Duc Hoang,et al.  A novel method for asphalt pavement crack classification based on image processing and machine learning , 2019, Engineering with Computers.

[45]  Mehdi Mokhtarzade,et al.  A new method for road detection in urban areas using high-resolution satellite images and Lidar data based on fuzzy nearest-neighbor classification and optimal features , 2016, Arabian Journal of Geosciences.

[46]  Shoaib Ehsan,et al.  Multiscale image denoising using goodness-of-fit test based on EDF statistics , 2019, PloS one.

[47]  Nan Yang,et al.  A review of road extraction from remote sensing images , 2016 .

[48]  Any Gupta,et al.  A Framework for Camera-Based Real-Time Lane and Road Surface Marking Detection and Recognition , 2018, IEEE Transactions on Intelligent Vehicles.

[49]  Vinicius F. S. Mota,et al.  A participatory sensing framework to classify road surface quality , 2019, Journal of Internet Services and Applications.

[50]  R. Liebich,et al.  Impact detection using a machine learning approach and experimental road roughness classification , 2019, Mechanical Systems and Signal Processing.

[51]  Hao Xu,et al.  Lane change identification and prediction with roadside LiDAR data , 2020 .

[52]  Sukhan Lee,et al.  Frontiers of Intelligent Autonomous Systems , 2013, Frontiers of Intelligent Autonomous Systems.

[53]  Fereidoon Moghadas Nejad,et al.  Image Based Techniques for Crack Detection, Classification and Quantification in Asphalt Pavement: A Review , 2017 .