Providentia - A Large Scale Sensing System for the Assistance of Autonomous Vehicles

The environmental perception of autonomous vehicles is not only limited by physical sensor ranges and algorithmic performance, but also occlusions degrade their understanding of the current traffic situation. This poses a great threat for safety, limits their driving speed and can lead to inconvenient maneuvers that decrease their acceptance. Intelligent Transportation Systems can help to alleviate these problems. By providing autonomous vehicles with additional detailed information about the current traffic in form of a digital model of their world, i.e. a digital twin, an Intelligent Transportation System can fill in the gaps in the vehicle's perception and enhance its field of view. However, detailed descriptions of implementations of such a system and working prototypes demonstrating its feasibility are scarce. In this work, we propose a hardware and software architecture to build such a reliable Intelligent Transportation System. We have implemented this system in the real world and show that it is able to create an accurate digital twin of an extended highway stretch. Furthermore, we provide this digital twin to an autonomous vehicle and demonstrate how it extends the vehicle's perception beyond the limits of its on-board sensors.

[1]  S.S. Blackman,et al.  Multiple hypothesis tracking for multiple target tracking , 2004, IEEE Aerospace and Electronic Systems Magazine.

[2]  Ronald P. S. Mahler,et al.  Optimal/robust distributed data fusion: a unified approach , 2000, SPIE Defense + Commercial Sensing.

[3]  Xiaogang Wang,et al.  Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-Temporal Path Proposals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Alois Knoll,et al.  Graph Neural Networks for Modelling Traffic Participant Interaction , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[5]  Milos Vasic,et al.  A Collaborative Sensor Fusion Algorithm for Multi-object Tracking Using a Gaussian Mixture Probability Hypothesis Density Filter , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[6]  José M. F. Moura,et al.  Understanding Traffic Density from Large-Scale Web Camera Data , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ling Shao,et al.  Viewpoint-Aware Attentive Multi-view Inference for Vehicle Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Gereon Hinz,et al.  Designing a far-reaching view for highway traffic scenarios with 5G-based intelligent infrastructure , 2017 .

[9]  Lijun Yu,et al.  Traffic Danger Recognition With Surveillance Cameras Without Training Data , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[10]  J. Miller,et al.  Vehicle-to-vehicle-to-infrastructure (V2V2I) intelligent transportation system architecture , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[11]  Stanley T. Birchfield,et al.  A Taxonomy and Analysis of Camera Calibration Methods for Traffic Monitoring Applications , 2010, IEEE Transactions on Intelligent Transportation Systems.

[12]  Dong Ryeol Shin,et al.  A Survey of Intelligent Transportation Systems , 2011, 2011 Third International Conference on Computational Intelligence, Communication Systems and Networks.

[13]  José M. F. Moura,et al.  FCN-rLSTM: Deep Spatio-Temporal Neural Networks for Vehicle Counting in City Cameras , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  B. Vo,et al.  Data Association and Track Management for the Gaussian Mixture Probability Hypothesis Density Filter , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[15]  Alois Knoll,et al.  Targetless Rotational Auto-Calibration of Radar and Camera for Intelligent Transportation Systems , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[16]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[17]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[18]  Igor Kabashkin,et al.  Reliability of bidirectional V2X communications in the intelligent transport systems , 2015, 2015 Advances in Wireless and Optical Communications (RTUWO).