A Methodology for Safety Assessment of Highly Automated Vehicles: From Scenario Classification to Corner Case Identification / submitted by DI. Jinwei Zhou

Autonomous driving represents a technological leap forward that will likely solve key aspects of the transport problem and so have beneficial effects for some ciritical social and ecological issues as well. Indeed, thanks to increasing levels of automation, the Highly Automated Vehicle (HAV) is expected to be able to handle more and more complex traffic situations so that it can take over driving tasks and free human beings from driving for long periods of time. It is also expected to reduce accidents and improve traffic fluidity, reducing the overall carbon footprint, among other advantages. However, the higher the automation level, the more complex the HAV. As addressed by many, testing and validation of the HAV, so as to guarantee safety, is one of the most challenging tasks that still prevent the HAV from commercial release. Complexity excludes the use of well established methods for testing of classical vehicles, by on-road testing. This has led to a wide consensus that simulation “virtual testing” must be included as early as in the design phase. However, even simulation cannot test all possible situations. As HAVs will include many functions which can be updated frequently and require re-evaluation in short time, fast re-evaluations will be needed to prevent new dangers arising from the updates. This leads to the dilemma of HAV testing: on one side, we need to test enough situations to be confident about the behavior of HAV in the general, unknown case, but on the other side these tests need to be as limited as possible. Against this background, this research work is focused on developing such a methodology. The key idea is to replace on-road testing by accident statistics – so to say “involuntary” fleet testing – and to use model based methods as well as Design of Experiments to determine a limited set of cases to be tested. As an example to explain our approach, we concentrate on highways and starts by establishing a catalogue of countable scenarios, which cover over 90% of the (near-) crashes reported in U.S. highway databases between 2010 and 2013. Utilizing the catalogue, an approach has been developed leading to a realistic but parsimonious model based parametrization of the scenarios using experimental measurements. This allows covering the majority of the measured real traffic situation with a rather simple parameter set. The parametrization can then be used to determine a boundary that separates safe conditions from unsafe ones by a suitable Design of Experiment strategy.

[1]  Halil Beglerovic,et al.  Model-based safety validation of the automated driving functio highway pilot , 2017 .

[2]  Kurt M. Marshek,et al.  Comparison of Ideal Vehicle Lane-Change Trajectories , 1997 .

[3]  Ding Zhao,et al.  Accelerated evaluation of automated vehicles using extracted naturalistic driving data , 2016 .

[4]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[5]  Li Huang,et al.  Study on the Test Scenarios of Level 2 Automated Vehicles , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[6]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[7]  Xuesong Wang,et al.  Analysis of cut-in behavior based on naturalistic driving data. , 2019, Accident; analysis and prevention.

[8]  Maciej Babiak,et al.  Analysis of the Potential of Autonomous Vehicles in Reducing the Emissions of Greenhouse Gases in Road Transport , 2017 .

[9]  Petros A. Ioannou,et al.  Evaluation of ACC vehicles in mixed traffic: lane change effects and sensitivity analysis , 2005, IEEE Transactions on Intelligent Transportation Systems.

[10]  Erwin de Gelder,et al.  Assessment of Automated Driving Systems using real-life scenarios , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[11]  O Schaedler,et al.  Entwicklung eines Verfahrens und der dazugehoerigen Hard- und Software zur Bewertung und Verbesserung der Controllability von Fahrerassistenzsystemen (ConFAS) , 2013 .

[12]  Luigi del Re,et al.  Identification of critical cases of ADAS safety by FOT based parameterization of a catalogue , 2017, 2017 11th Asian Control Conference (ASCC).

[13]  Nidhi Kalra,et al.  Autonomous Vehicle Technology: A Guide for Policymakers , 2014 .

[14]  Soyoung Ahn,et al.  The effects of lane-changing on the immediate follower : anticipation, relaxation, and change in driver characteristics , 2013 .

[15]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[16]  Varun Ramanujam,et al.  Lane changing models for arterial traffic , 2007 .

[17]  Alan D. Lopez,et al.  Alternative projections of mortality and disability by cause 1990–2020: Global Burden of Disease Study , 1997, The Lancet.

[18]  Zhaohui Wu,et al.  A Smart Car Control Model for Brake Comfort Based on Car Following , 2009, IEEE Transactions on Intelligent Transportation Systems.

[19]  Kazuya Takeda,et al.  Driver evaluation based on classification of rapid decelerating patterns , 2009, 2009 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[20]  Luigi del Re,et al.  Efficient mixed integer programming for autonomous overtaking , 2017, 2017 American Control Conference (ACC).

[21]  Jonathan M. Hankey,et al.  Description of the SHRP 2 Naturalistic Database and the Crash, Near-Crash, and Baseline Data Sets , 2016 .

[22]  Fei-Yue Wang,et al.  Cooperative Driving at Lane Closures , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[23]  Keisuke Yoneda,et al.  Distance constraint model for automated lane change to merge or exit , 2015 .

[24]  Soyoung Ahn,et al.  Impact of traffic oscillations on freeway crash occurrences. , 2010, Accident; analysis and prevention.

[25]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[26]  Dario D. Salvucci,et al.  The time course of a lane change: Driver control and eye-movement behavior , 2002 .

[27]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[28]  Markus Maurer,et al.  Ontology based Scene Creation for the Development of Automated Vehicles , 2017, 2018 IEEE Intelligent Vehicles Symposium (IV).

[29]  Fredrik Gustafsson,et al.  Toward Autonomous Collision Avoidance by Steering , 2007, IEEE Transactions on Intelligent Transportation Systems.

[30]  O Schädler,et al.  Experimental evaluation of the controllability of interacting advanced driver assistance systems , 2016 .

[31]  Michel Verhaegen,et al.  Probabilistic Approach for Validation of Advanced Driver Assistance Systems , 2005 .

[32]  Min Zhou,et al.  Autonomous Overtaking Behavior Simulation for Autonomous Virtual Vehicle in Virtual Traffic Environment , 2008, 2008 International Conference on Computer Science and Software Engineering.

[33]  Luigi del Re,et al.  A Framework for Virtual Testing of ADAS , 2016 .

[34]  Tomer Toledo,et al.  Modeling Duration of Lane Changes , 2007 .

[35]  Ding Zhao,et al.  Accelerated Evaluation of Automated Vehicles based on Importance Sampling Techniques , 2016, ArXiv.

[36]  André A. Rupp,et al.  The Impact of Model Misspecification on Parameter Estimation and Item‐Fit Assessment in Log‐Linear Diagnostic Classification Models , 2012 .

[37]  Hermann Winner Safe Driving Despite Uncertainties , 2013 .

[38]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[39]  Hesham Rakha,et al.  Characterizing Driver Behavior on Signalized Intersection Approaches at the Onset of a Yellow-Phase Trigger , 2007, IEEE Transactions on Intelligent Transportation Systems.

[40]  Zuduo Zheng,et al.  Recent developments and research needs in modeling lane changing , 2014 .

[41]  Saber Naseralavi,et al.  A general formulation for time-to-collision safety indicator , 2013 .

[42]  Luigi del Re,et al.  Reduced Complexity Safety Testing for ADAS & ADF , 2017 .

[43]  Akhilesh Kumar Maurya,et al.  Study of Deceleration Behaviour of Different Vehicle Types , 2012 .

[44]  Wassim G. Najm,et al.  Development of Crash Imminent Test Scenarios for Integrated Vehicle-Based Safety Systems , 2007 .

[45]  B. De Schutter,et al.  Traffic Modelling Validation of Advanced Driver Assistance Systems , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[46]  Jonas Sjöberg,et al.  Predictive cruise control with autonomous overtaking , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[47]  Tomasz Stańczyk,et al.  Driver reaction time to lateral entering pedestrian in a simulated crash traffic situation , 2014 .

[48]  Patrizio Colaneri,et al.  A minimum entropy based switching approach for automated driving assistance , 2017, 2017 American Control Conference (ACC).

[49]  Yoshiyuki Umemura Driver Behavior and Active Safety ( Overview ) , 2004 .

[50]  Hiroyuki Oyama,et al.  Identification of Static Boundary Model Based on Gaussian Process Classification , 2016 .

[51]  M. Allen,et al.  Department of Transportation National Highway Traffic Safety Administration Vehicle-to-vehicle Security Credential Management System; Request for Information , 2022 .

[52]  Luigi del Re,et al.  Risk functions oriented autonomous overtaking , 2017, 2017 11th Asian Control Conference (ASCC).

[53]  Amnon Shashua,et al.  On a Formal Model of Safe and Scalable Self-driving Cars , 2017, ArXiv.

[54]  P Fancher,et al.  Intelligent Cruise Control Field Operational Test , 1997 .

[55]  Ding Zhao,et al.  Accelerated Evaluation of Automated Vehicles. , 2016 .

[56]  Temple F. Smith Occam's razor , 1980, Nature.

[57]  M Cameron,et al.  World Report on Road Traffic Injury Prevention. , 2004 .

[58]  Peng Liu,et al.  Classification of Highway Lane Change Behavior to Detect Dangerous Cut-in Maneuvers , 2016 .

[59]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[60]  Hermann Winner,et al.  The Release of Autonomous Vehicles , 2016 .

[61]  Olaf Op den Camp,et al.  Scenario Identification for Validation of Automated Driving Functions , 2016 .

[62]  Matthias Althoff,et al.  Formalising and Monitoring Traffic Rules for Autonomous Vehicles in Isabelle/HOL , 2017, IFM.

[63]  Håkan Jonsson,et al.  Planning Smooth and Obstacle-Avoiding B-Spline Paths for Autonomous Mining Vehicles , 2010, IEEE Transactions on Automation Science and Engineering.

[64]  Aurelio Piazzi,et al.  Quintic G2-splines for the iterative steering ofvision-based autonomous vehicles , 2002, IEEE Trans. Intell. Transp. Syst..

[65]  Desmond O'Neill,et al.  The world's first automobile fatality. , 2005, Accident; analysis and prevention.

[66]  Dennis A. Guenther,et al.  Analysis of Human Driver Behavior in Highway Cut-in Scenarios , 2017 .

[67]  Bernhard Friedrich,et al.  The Effect of Autonomous Vehicles on Traffic , 2016 .

[68]  Sandor S. Szabo,et al.  Objective test scenarios for integrated vehicle-based safety systems , 2007 .

[69]  Kazuya Takeda,et al.  Driver risk evaluation based on acceleration, deceleration, and steering behavior , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[70]  Andrew Morris,et al.  Analyzing and Modeling Drivers’ Deceleration Behavior from Normal Driving , 2017 .

[71]  Li Liu,et al.  Dynamic Modeling of Driver Control Strategy of Lane-Change Behavior and Trajectory Planning for Collision Prediction , 2012, IEEE Transactions on Intelligent Transportation Systems.

[72]  Ben Waterson,et al.  Modeling the Dynamic Cut-In Situation , 2002 .

[73]  Luigi del Re,et al.  On-line Maneuver Identification in Highway Traffic Using Elastic Template Matching , 2018 .

[74]  H. Negele,et al.  Einsatz von Referenzfahrsituation bei der Entwicklung von Fahrerassistenzsystemen , 2008 .

[75]  Kayalvizhi Jayavel,et al.  Intelligent collision avoidance and safety warning system for car driving , 2017, 2017 International Conference on Intelligent Computing and Control Systems (ICICCS).

[76]  Faraz Kunwar,et al.  Autonomous vehicle overtaking- an online solution , 2009, 2009 IEEE International Conference on Automation and Logistics.

[77]  Xuehan Ma,et al.  Analysis of Driver Brake Behavior Under Critical Cut-in Scenarios , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[78]  Falko Saust,et al.  Effiziente systematische Testgenerierung für Fahrerassistenzsysteme in virtuellen Umgebungen , 2013 .

[79]  Oliver Nelles,et al.  Support Vector Machines for Design Space Exploration , 2012 .

[80]  Luigi del Re,et al.  Safety Verification Of ADAS By Collision-free Boundary Searching Of A Parameterized Catalog , 2018, 2018 Annual American Control Conference (ACC).

[81]  Andreas Geiger,et al.  Understanding High-Level Semantics by Modeling Traffic Patterns , 2013, 2013 IEEE International Conference on Computer Vision.

[82]  Danielle Dai,et al.  Public Perceptions of Self-Driving Cars: The Case of Berkeley, California , 2014 .

[83]  C. Ratti,et al.  The Impact of Autonomous Vehicles on Cities: A Review , 2018, Journal of Urban Technology.

[84]  Luigi del Re,et al.  Gap Acceptance Based Safety Assessment Of Autonomous Overtaking Function , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[85]  Ding Zhao,et al.  Accelerated evaluation of automated vehicles using extracted naturalistic driving data , 2016 .

[86]  Limian Wang,et al.  Analysis of Driver Brake Operation In Near-Crash Situation Using Naturalistic Driving Data , 2011 .

[87]  Kazuya Takeda,et al.  Stochastic modeling of vehicle trajectory during lane-changing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[88]  Masayoshi Tomizuka,et al.  Fast lane changing computations using polynomials , 2003, Proceedings of the 2003 American Control Conference, 2003..

[89]  Qi Wang,et al.  Investigation of Discretionary Lane-Change Characteristics Using Next-Generation Simulation Data Sets , 2014, J. Intell. Transp. Syst..