Evidential Data Fusion for Characterization of Pavement Surface Conditions during Winter Using a Multi-Sensor Approach

The role of a service that is dedicated to road weather analysis is to issue forecasts and warnings to users regarding roadway conditions, thereby making it possible to anticipate dangerous traffic conditions, especially during the winter period. It is important to define pavement conditions at all times. In this paper, a new data acquisition approach is proposed that is based upon the analysis and combination of two sensors in real time by nanocomputer. The first sensor is a camera that records images and videos of the road network. The second sensor is a microphone that records the tire–pavement interaction, to characterize each surface’s condition. The two low-cost sensors were fed to different deep learning architectures that are specialized in surface state analysis; the results were combined using an evidential theory-based data fusion approach. This study is a proof of concept, to test an evidential approach for improving classification with deep learning, applied to only two sensors; however, one could very well add more sensors and make the nanocomputers communicate together, to analyze a larger urban environment.

[1]  Ching-Haur Chang,et al.  A lane detection approach based on intelligent vision , 2015, Comput. Electr. Eng..

[2]  C. Asensio,et al.  Identification and mapping of asphalt surface deterioration by tyre-pavement interaction noise measurement , 2019 .

[3]  Jean Dezert,et al.  Applications and Advances of DSmT for Information Fusion , 2004 .

[4]  Wilhelm Stork,et al.  Road surface classification for extended floating car data , 2014, 2014 IEEE International Conference on Vehicular Electronics and Safety.

[5]  Jiandong Zhao,et al.  Road Surface State Recognition Based on SVM Optimization and Image Segmentation Processing , 2017 .

[6]  Erik Marchi,et al.  Detecting Road Surface Wetness from Audio: A Deep Learning Approach , 2015, 2016 23rd International Conference on Pattern Recognition (ICPR).

[7]  Arthur P. Dempster,et al.  A Generalization of Bayesian Inference , 1968, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[8]  N. Abbas,et al.  Développement de modèles de fusion et de classification contextuelle d'images satellitaires par la théorie de l'évidence et la théorie du raisonnement plausible et paradoxal , 2009 .

[9]  Tan Li,et al.  A state-of-the-art review of measurement techniques on tire–pavement interaction noise , 2018, Measurement.

[10]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Faouzi Alaya Cheikh,et al.  Road surface condition detection utilizing resonance frequency and optical technologies , 2019 .

[12]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[13]  DSmT-Based Fusion Strategy for Human Activity Recognition in Body Sensor Networks , 2020 .

[14]  Shaun S. Wulff,et al.  Multilevel weather detection based on images: a machine learning approach with histogram of oriented gradient and local binary pattern-based features , 2021, J. Intell. Transp. Syst..

[15]  Wei Dai,et al.  Very deep convolutional neural networks for raw waveforms , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Thierry Denoeux,et al.  Fusion of evidential CNN classifiers for image classification , 2021, BELIEF.

[17]  Thierry Denoeux,et al.  An evidential classifier based on Dempster-Shafer theory and deep learning , 2021, Neurocomputing.

[18]  Gurjit Singh Walia,et al.  Intelligent fire-detection model using statistical color models data fusion with Dezert–Smarandache method , 2013 .

[19]  Nabil Khoderagha.  Outil innovant pour la gestion des routes , 2019 .

[20]  Sylvie Le Hégarat-Mascle,et al.  Evidential framework for data fusion in a multi-sensor surveillance system , 2015, Eng. Appl. Artif. Intell..

[21]  S. Dimter,et al.  Application of GIS technology in pavement management systems , 2019, Journal of the Croatian Association of Civil Engineers.

[22]  M. Jeon,et al.  Classification of road surface status using a 94 GHz dual-channel polarimetric radiometer , 2012 .

[23]  Hideyuki Nomura,et al.  Detection of Road Surface States from Tire Noise Using Neural Network Analysis , 2010 .

[24]  Fereidoon Moghadas Nejad,et al.  Evaluation of pavement surface drainage using an automated image acquisition and processing system , 2018 .

[25]  Hideyuki Nomura,et al.  Detection of Road Surface Conditions Using Tire Noise from Vehicles , 2009 .

[26]  Xiaohui Liu,et al.  Real-time traffic sign recognition from video by class-specific discriminative features , 2010, Pattern Recognit..

[27]  Edwin Simonnet,et al.  Réseaux de neurones profonds appliqués à la compréhension de la parole. (Deep learning applied to spoken langage understanding) , 2019 .

[28]  P. Smets.  Data fusion in the transferable belief model , 2000, Proceedings of the Third International Conference on Information Fusion.

[29]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[30]  Patrik Jonsson,et al.  Road Surface Status Classification Using Spectral Analysis of NIR Camera Images , 2015, IEEE Sensors Journal.

[31]  Krzysztof Czarnecki,et al.  Canadian Adverse Driving Conditions dataset , 2020, Int. J. Robotics Res..

[32]  Thomas Beckenbauer,et al.  Investigating generation mechanisms of tyre/road noise by speed exponent analysis , 2017 .

[33]  Dempster–Shafer Theory for Modeling and Treating Uncertainty in IoT Applications Based on Complex Event Processing , 2021, Sensors.