Probabilistic Collision Risk Estimation for Autonomous Driving: Validation via Statistical Model Checking

A crucial aspect that automotive systems need to face before being used in everyday life is the validation of their components. To this end, standard exhaustive methods are inappropriate to validate the probabilistic algorithms widely used in this field and new solutions need to be adopted. In this paper, we present an approach based on Statistical Model Checking (SMC) to validate the collision risk assessment generated by a probabilistic perception system. SMC represents an intermediate between test and exhaustive verification by relying on statistics and evaluates the probability of meeting appropriate Key Performance Indicators (KPIs) based on a large number of simulations. As a case study, a state-of-the-art algorithm is adopted to obtain the collision risk estimations. This algorithm provides an environment representation through Bayesian probabilistic occupancy grids and estimates positions in the near future of every static and dynamic part of the grid. Based on these estimations, time-to-collision probabilities are then associated with the corresponding cells. Using CARLA simulator, a large number of execution traces are then generated, considering both collisions and almost-collisions in realistic urban scenarios. Real experiments complete the analysis and show the reliability of the simulation results.

[1]  Lukas Rummelhard,et al.  Conditional Monte Carlo Dense Occupancy Tracker , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[2]  Fred Kröger,et al.  Temporal Logic of Programs , 1987, EATCS Monographs on Theoretical Computer Science.

[3]  Jonathan Petit,et al.  Remote Attacks on Automated Vehicles Sensors : Experiments on Camera and LiDAR , 2015 .

[4]  Hans P. Moravec.  Sensor Fusion in Certainty Grids for Mobile Robots , 1988, AI Mag..

[5]  Lukas Rummelhard,et al.  Validation of Perception and Decision-Making Systems for Autonomous Driving via Statistical Model Checking , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[6]  Christian Wolf,et al.  Attentional PointNet for 3D-Object Detection in Point Clouds , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Axel Legay,et al.  Plasma Lab: A Modular Statistical Model Checking Platform , 2016, ISoLA.

[8]  Axel Legay,et al.  Motion planning in crowds using statistical model checking to enhance the social force model , 2013, 52nd IEEE Conference on Decision and Control.

[9]  Matthias Althoff,et al.  Online Verification of Automated Road Vehicles Using Reachability Analysis , 2014, IEEE Transactions on Robotics.

[10]  Yaakov Bar-Shalom,et al.  Multi-target tracking using joint probabilistic data association , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[11]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[12]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[13]  C. Laugier,et al.  Probabilistic Grid-Based Collision Risk Prediction for Driving Application , 2014, ISER.

[14]  Nielen Stander,et al.  Parameter Identification of Hysteretic Models using Partial Curve Mapping , 2012 .

[15]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[16]  Didier Aubert,et al.  Experimental Assessment of the RESCUE Collision-Mitigation System , 2007, IEEE Transactions on Vehicular Technology.

[17]  Edmund M. Clarke,et al.  Bayesian statistical model checking with application to Stateflow/Simulink verification , 2013, Formal Methods Syst. Des..

[18]  Klaus C. J. Dietmayer,et al.  Situation Assessment of an Autonomous Emergency Brake for Arbitrary Vehicle-to-Vehicle Collision Scenarios , 2009, IEEE Transactions on Intelligent Transportation Systems.

[19]  Michael Fisher,et al.  Probabilistic Model Checking of Robots Deployed in Extreme Environments , 2018, AAAI.

[20]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[21]  Mahesh Viswanathan,et al.  On Statistical Model Checking of Stochastic Systems , 2005, CAV.

[22]  Javier Ibanez Guzman,et al.  Vehicle to vehicle communications applied to road intersection safety, field results , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[23]  Thomas Hérault,et al.  Approximate Probabilistic Model Checking , 2004, VMCAI.

[24]  Frank Dellaert,et al.  An MCMC-Based Particle Filter for Tracking Multiple Interacting Targets , 2004, ECCV.

[25]  Luke Fletcher,et al.  A perception‐driven autonomous urban vehicle , 2008, J. Field Robotics.

[26]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.