Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework

How to generate testing scenario libraries for connected and automated vehicles (CAVs) is a major challenge faced by the industry. In previous studies, to evaluate maneuver challenge of a scenario, surrogate models (SMs) are often used without explicit knowledge of the CAV under test. However, performance dissimilarities between the SM and the CAV under test usually exist, and it can lead to the generation of suboptimal scenario libraries. In this article, an adaptive testing scenario library generation (ATSLG) method is proposed to solve this problem. A customized testing scenario library for a specific CAV model is generated through an adaptive process. To compensate for the performance dissimilarities and leverage each test of the CAV, Bayesian optimization techniques are applied with classification-based Gaussian Process Regression and a newly designed acquisition function. Comparing with a pre-determined library, a CAV can be tested and evaluated in a more efficient manner with the customized library. To validate the proposed method, a cut-in case study is investigated and the results demonstrate that the proposed method can further accelerate the evaluation process by a few orders of magnitude.

[1]  Ding Zhao,et al.  Accelerated Evaluation of Autonomous Vehicles in the Lane Change Scenario Based on Subset Simulation Technique , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[2]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[3]  Lijun Sun,et al.  A Bayesian tensor decomposition approach for spatiotemporal traffic data imputation , 2019, Transportation Research Part C: Emerging Technologies.

[4]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[5]  Peter I. Frazier,et al.  A Tutorial on Bayesian Optimization , 2018, ArXiv.

[6]  Hardi Hungar,et al.  Test Specifications for Highly Automated Driving Functions: Highway Pilot , 2017 .

[7]  Hani S. Mahmassani,et al.  From Existing Accident-Free Car-Following Models to Colliding Vehicles , 2008 .

[8]  Kim C. Border,et al.  Infinite dimensional analysis , 1994 .

[9]  Sheldon M. Ross Introductory Statistics , 1995 .

[10]  Satyandra K. Gupta,et al.  Adaptive generation of challenging scenarios for testing and evaluation of autonomous vehicles , 2018, J. Syst. Softw..

[11]  Partha S. Roop,et al.  A Formal Approach for Modeling and Simulation of Human Car-Following Behavior , 2018, IEEE Transactions on Intelligent Transportation Systems.

[12]  Feng Gao,et al.  Test Scenario Generation and Optimization Technology for Intelligent Driving Systems , 2022, IEEE Intelligent Transportation Systems Magazine.

[13]  Yiheng Feng,et al.  Safety assessment of highly automated driving systems in test tracks: A new framework. , 2020, Accident; analysis and prevention.

[14]  Fabian Schuldt,et al.  Defining and Substantiating the Terms Scene, Situation, and Scenario for Automated Driving , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[15]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[16]  Robert B. Gramacy,et al.  Bayesian treed gaussian process models , 2005 .

[17]  Li Li,et al.  Intelligence Testing for Autonomous Vehicles: A New Approach , 2016, IEEE Transactions on Intelligent Vehicles.

[18]  Lingfeng Wang,et al.  A Bayesian Learning Based Scheme for Online Dynamic Security Assessment and Preventive Control , 2020, IEEE Transactions on Power Systems.

[19]  Dirk P. Kroese,et al.  Simulation and the Monte Carlo method , 1981, Wiley series in probability and mathematical statistics.

[20]  Vadim Sokolov,et al.  Bayesian Optimization for Transportation Simulators , 2018, ANT/SEIT.

[21]  Nanning Zheng,et al.  Parallel testing of vehicle intelligence via virtual-real interaction , 2019, Science Robotics.

[22]  Yi Zhang,et al.  Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology , 2019, IEEE Transactions on Intelligent Transportation Systems.

[23]  Naonori Ueda,et al.  Bayesian Optimization for Crowd Traffic Control Using Multi-Agent Simulation , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[24]  Nanning Zheng,et al.  Artificial intelligence test: a case study of intelligent vehicles , 2018, Artificial Intelligence Review.

[25]  Ding Zhao,et al.  Accelerated Evaluation of Automated Vehicles in Car-Following Maneuvers , 2016, IEEE Transactions on Intelligent Transportation Systems.

[26]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[27]  Ding Zhao,et al.  Accelerated Evaluation of Automated Vehicles. , 2016 .

[28]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[29]  Rupak Majumdar,et al.  Testing Cyber-Physical Systems through Bayesian Optimization , 2017, ACM Trans. Embed. Comput. Syst..

[30]  Huei Peng,et al.  Worst Case Scenarios Generation and Its Application on Driving , 2007 .

[31]  Yiheng Feng,et al.  Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies , 2019, IEEE Transactions on Intelligent Transportation Systems.