Identification of Challenging Highway-Scenarios for the Safety Validation of Automated Vehicles Based on Real Driving Data

For a successful market launch of automated vehicles (AVs), proof of their safety is essential. Due to the open parameter space, an infinite number of traffic situations can occur, which makes the proof of safety an unsolved problem. With the so-called scenario-based approach, all relevant test scenarios must be identified. This paper introduces an approach that finds particularly challenging scenarios from real driving data (RDD) and assesses their difficulty using a novel metric. Starting from the highD data, scenarios are extracted using a hierarchical clustering approach and then assigned to one of nine pre-defined functional scenarios using rule-based classification. The special feature of the subsequent evaluation of the concrete scenarios is that it is independent of the performance of the test vehicle and therefore valid for all AVs. Previous evaluation metrics are often based on the criticality of the scenario, which is, however, dependent on the behavior of the test vehicle and is therefore only conditionally suitable for finding "good" test cases in advance. The results show that with this new approach a reduced number of particularly challenging test scenarios can be derived.

[1]  Yugong Luo,et al.  A Trajectory-Based Method for Scenario Analysis and Test Effort Reduction for Highly Automated Vehicle , 2019, SAE Technical Paper Series.

[2]  Frank Diermeyer,et al.  A Method for the Selection of Challenging Driving Scenarios for Automated Vehicles Based on an Objective Characterization of the Driving Behavior , 2019 .

[3]  Jeroen Hogema,et al.  TIME-TO-COLLISION AND COLLISION AVOIDANCE SYSTEMS , 1994 .

[4]  Tim Fingscheidt,et al.  Towards Corner Case Detection for Autonomous Driving , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[5]  Matthias Althoff,et al.  Automatic Generation of Safety-Critical Test Scenarios for Collision Avoidance of Road Vehicles , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[6]  Xiao Yu Method for quantitative evaluation of traffic complexity on the highway , 2020 .

[7]  Thomas Ponn,et al.  Systematic Analysis of the Sensor Coverage of Automated Vehicles Using Phenomenological Sensor Models* , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[8]  Lutz Eckstein,et al.  The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[9]  Fabian Schuldt,et al.  Defining and Substantiating the Terms Scene, Situation, and Scenario for Automated Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[10]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[11]  Thomas Ponn,et al.  Ponn 1 AN OPTIMIZATION-BASED METHOD TO IDENTIFY RELEVANT SCENARIOS FOR TYPE APPROVAL OF AUTOMATED VEHICLES , 2019 .

[12]  Markus Maurer,et al.  Scenarios for Development, Test and Validation of Automated Vehicles , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[13]  Ulrich Eberle,et al.  Simulation-Based Identification of Critical Scenarios for Cooperative and Automated Vehicles , 2018 .

[14]  John C Hayward,et al.  NEAR-MISS DETERMINATION THROUGH USE OF A SCALE OF DANGER , 1972 .

[15]  Chi Zhang,et al.  Traffic Sensory Data Classification by Quantifying Scenario Complexity , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[16]  A. Jacobo,et al.  DEVELOPMENT OF A SAFETY ASSURANCE PROCESS FOR AUTONOMOUS VEHICLES IN JAPAN , 2019 .

[17]  Frank Diermeyer,et al.  Survey on Scenario-Based Safety Assessment of Automated Vehicles , 2020, IEEE Access.

[18]  S. M. Sohel Mahmud,et al.  Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs , 2017 .

[19]  Markus Maurer,et al.  Ontology based Scene Creation for the Development of Automated Vehicles , 2017, 2018 IEEE Intelligent Vehicles Symposium (IV).

[20]  Feng Gao,et al.  Automatic Generation Method of Test Scenario for ADAS Based on Complexity , 2017, ICVS 2017.

[21]  Daniela Rus,et al.  Learning Risk Level Set Parameters from Data Sets for Safer Driving , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[22]  Emre Kaplan,et al.  Real- World Maneuver Extraction for Autonomous Vehicle Validation: A Comparative Study , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[23]  Feng Gao,et al.  Test Scenario Design for Intelligent Driving System Ensuring Coverage and Effectiveness , 2018, International Journal of Automotive Technology.

[24]  Feng Gao,et al.  A Test Scenario Automatic Generation Strategy for Intelligent Driving Systems , 2019, Mathematical Problems in Engineering.

[25]  Lutz Eckstein,et al.  A framework for definition of logical scenarios for safety assurance of automated driving , 2019, Traffic injury prevention.

[26]  Matthias Althoff,et al.  Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).