Deep Learning and Spatial Statistics for Determining Road Surface Condition

Machine Learning (ML), and especially Deep Learning (DL) methods, have evolved rapidly over last years and showed remarkable advances in research areas such as computer vision and natural language processing; however, there are still engineering applications in industries such as transportation where DL methods have not been applied yet or that can be benefited from an integrated approach using DL in addition to other methods. For countries in Northern latitudes one of such applications is Monitoring Road Surface Condition (RSC) during the Winter season for improving road safety and road maintenance operations. In this study we introduce a novel approach for monitoring of RSC that integrates DL methods and Spatial Statistics (SS) to simultaneously process data from roadside cameras and weather stations to determine automatically the category of snow coverage at sample locations across a region of interest. Our approach integrates the advantages of SS for interpolating spatial variables and the strengths of DL for Computer Vision tasks, particularly for image classification. On one hand SS models serve to understand the spatial autocorrelation of random variables and to determine their expected values in unsampled locations based on a number of near observations. On the other hand, DL models extract relevant patterns from a large number of training images and learn a mapping from input images to a set of predefined labels. We implement and evaluate our approach using data collected in the province of Ontario during the 2017-2018 Winter season. Specifically, we included data from three separate sources, Environment Canada (EC) Weather stations, Road Weather Information System (RWIS) stations, and roadside cameras from the Ministry of Transportation of Ontario (MTO). To the best of our knowledge, this is the first study that integrates both DL and SS techniques for processing the three data sources with the goal of monitoring RSC. The DL models we implement and compared are Inception, Inception-Resnet, Xception, DenseNet, MobileNetv2, and NASNet. All of these models have achieved remarkable results for image classification in well-known benchmarks. The SS models we evaluate are Ordinary Kriging (OK), Radial Basis Functions (RBF), and Inverse Distance Weigthed (IDW). The first provides a comprehensive understanding of the spatial autocorrelation for each particular variable, while the second and third allow a faster implementation. Our integrated approach works by combining the output feature vector from the DL model with the interpolated values from the SS model to output a more robust prediction of RSC for the locations of interest.