Identification of Road-Surface Type Using Deep Neural Networks for Friction Coefficient Estimation

Nowadays, vehicles have advanced driver-assistance systems which help to improve vehicle safety and save the lives of drivers, passengers and pedestrians. Identification of the road-surface type and condition in real time using a video image sensor, can increase the effectiveness of such systems significantly, especially when adapting it for braking and stability-related solutions. This paper contributes to the development of the new efficient engineering solution aimed at improving vehicle dynamics control via the anti-lock braking system (ABS) by estimating friction coefficient using video data. The experimental research on three different road surface types in dry and wet conditions has been carried out and braking performance was established with a car mathematical model (MM). Testing of a deep neural networks (DNN)-based road-surface and conditions classification algorithm revealed that this is the most promising approach for this task. The research has shown that the proposed solution increases the performance of ABS with a rule-based control strategy.

[1]  Feng Xiao,et al.  Lane Detection Algorithm for Intelligent Vehicles in Complex Road Conditions and Dynamic Environments , 2019, Sensors.

[2]  P. S. Heyns,et al.  Application of an ANN-based methodology for road surface condition identification on mining vehicles and roads , 2014 .

[3]  Carmelo D'Agostino,et al.  From manual to automatic pavement distress detection and classification , 2017, 2017 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS).

[4]  Wuwei Chen,et al.  Curve recognition algorithm based on edge point curvature voting , 2020 .

[5]  Tomoo Kamakura,et al.  Identification of winter tires using vibration signals generated on the road surface , 2014 .

[6]  Sohini Roy Chowdhury,et al.  Methods and systems for generating and using a road friction estimate based on camera image signal processing , 2019 .

[7]  Klaus Augsburg,et al.  Robust Continuous Wheel Slip Control With Reference Adaptation: Application to the Brake System With Decoupled Architecture , 2018, IEEE Transactions on Industrial Informatics.

[8]  Gentiane Venture,et al.  Modeling and Identification of Passenger Car Dynamics Using Robotics Formalism , 2006, IEEE Transactions on Intelligent Transportation Systems.

[9]  Sangram Patil,et al.  Surface prediction and control algorithms for anti-lock brake system , 2012 .

[10]  Stratis Kanarachos,et al.  Machine Learning Algorithms for Wet Road Surface Detection Using Acoustic Measurements , 2019, 2019 IEEE International Conference on Mechatronics (ICM).

[11]  Torsten Sattler,et al.  Obstacle detection for self-driving cars using only monocular cameras and wheel odometry , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Guillaume Saint-Pierre,et al.  How to predict real road state from vehicle embedded camera? , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[13]  Antonio Jesús Guerra Fernández,et al.  A Procedure for Determining Tire-Road Friction Characteristics Using a Modification of the Magic Formula Based on Experimental Results , 2018, Sensors.

[14]  Henri Briand,et al.  One application of neural networks for detection of defects using video data bases: identification of road distresses , 1997, Database and Expert Systems Applications. 8th International Conference, DEXA '97. Proceedings.

[15]  Alexandre Santos,et al.  A New Approach on Communications Architectures for Intelligent Transportation Systems , 2017, FNC/MobiSPC.

[16]  Joachim Rudolph,et al.  Vehicle state estimation for anti-lock control with nonlinear observer , 2015 .

[17]  Mats Jonasson,et al.  Machine Learning Models for Road Surface and Friction Estimation using Front-Camera Images , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[18]  Burhan Ergen,et al.  Recognition of Road Type and Quality for Advanced Driver Assistance Systems with Deep Learning , 2018, Elektronika ir Elektrotechnika.

[19]  Gaetano Assanto,et al.  A near-infrared optoelectronic approach to detection of road conditions , 2013 .

[20]  Danwei Wang,et al.  Vehicle motion control with subsystem prioritization , 2015 .

[21]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[22]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[23]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[24]  Xinming Huang,et al.  End-to-end learning for lane keeping of self-driving cars , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[25]  Yang Song,et al.  Rapid Estimation of Road Friction for Anti-Skid Autonomous Driving , 2020, IEEE Transactions on Intelligent Transportation Systems.

[26]  Eldar Sabanovic,et al.  TECHNOLOGICAL MEASURES OF FOREFRONT ROAD IDENTIFICATION FOR VEHICLE COMFORT AND SAFETY IMPROVEMENT , 2019 .

[27]  C. Asensio,et al.  On-board wet road surface identification using tyre/road noise and Support Vector Machines , 2014 .

[28]  Nikolai Smolyanskiy,et al.  On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Arto J. Niskanen,et al.  Three Three-Axis IEPE Accelerometers on the Inner Liner of a Tire for Finding the Tire-Road Friction Potential Indicators † , 2015, Sensors.

[30]  Oluremi Olatunbosun,et al.  A Strain-Based Method to Detect Tires’ Loss of Grip and Estimate Lateral Friction Coefficient from Experimental Data by Fuzzy Logic for Intelligent Tire Development , 2018, Sensors.

[31]  Hsin Guan,et al.  Road surface condition identification approach based on road characteristic value , 2014 .

[32]  Stratis Kanarachos,et al.  Road Friction Virtual Sensing: A Review of Estimation Techniques with Emphasis on Low Excitation Approaches , 2017 .

[33]  Danwei Wang,et al.  Vehicle dynamics control with energy recuperation based on control allocation for independent wheel motors and brake system , 2013 .

[34]  Paulo Lobato Correia,et al.  Identifying and retrieving distress images from road pavement surveys , 2008, 2008 15th IEEE International Conference on Image Processing.

[35]  Francisco Angel Moreno,et al.  The Málaga urban dataset: High-rate stereo and LiDAR in a realistic urban scenario , 2014, Int. J. Robotics Res..

[36]  Hiroyuki Hisamatsu,et al.  Evaluation of a Bicycle-Mounted Ultrasonic Distance Sensor for Monitoring Road Surface Condition , 2015, 2015 7th International Conference on Computational Intelligence, Communication Systems and Networks.

[37]  Klaus Augsburg,et al.  Extending the HSRI tyre model for large inflation pressure changes , 2017 .

[38]  P. S. Heyns,et al.  An overview of the neural network based technique for monitoring of road condition via reconstructe4d road profiles , 2008 .

[39]  Sibel Arslan,et al.  The effects of OFDM design parameters on the V2X communication performance: A survey , 2017, Veh. Commun..

[40]  Gang Shen.  Road crack detection based on video image processing , 2016, 2016 3rd International Conference on Systems and Informatics (ICSAI).

[41]  P. Schalk Els,et al.  ABS braking on rough terrain , 2018, Journal of Terramechanics.

[42]  Jiang Yu Zheng,et al.  Modeling Weather and Illuminations in Driving Views Based on Big-Video Mining , 2018, IEEE Transactions on Intelligent Vehicles.

[43]  Henrikas Sivilevičius,et al.  Magic Formula Tyre Model Application for a Tyre-Ice Interaction , 2017 .