Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems

Autonomous driving has been the subject of incre- ased interest in recent years both in industry and in academia. Serious efforts are being pursued to address legal, technical, and logistical problems and make autonomous cars a viable option for everyday transportation. One significant challenge is the time and effort required for the verification and validation of the decision and control algorithms employed in these vehicles to ensure a safe and comfortable driving experience. Hundreds of thousands of miles of driving tests are required to achieve a well calibrated control system that is capable of operating an autonomous vehicle in an uncertain traffic environment where interactions among multiple drivers and vehicles occur simultaneously. Traffic simulators where these interactions can be modeled and represented with reasonable fidelity can help to decrease the time and effort necessary for the development of the autonomous driving control algorithms by providing a venue where acceptable initial control calibrations can be achieved quickly and safely before actual road tests. In this paper, we present a game theoretic traffic model that can be used to: 1) test and compare various autonomous vehicle decision and control systems and 2) calibrate the parameters of an existing control system. We demonstrate two example case studies, where, in the first case, we test and quantitatively compare two autonomous vehicle control systems in terms of their safety and performance, and, in the second case, we optimize the parameters of an autonomous vehicle control system, utilizing the proposed traffic model and simulation environment.

[1]  T. Hedden,et al.  What do you think I think you think?: Strategic reasoning in matrix games , 2002, Cognition.

[2]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2015, Autonomous Robots.

[3]  Andrew G. Lamperski,et al.  Periodically Controlled Hybrid Systems Verifying A Controller for An Autonomous Vehicle , 2008 .

[4]  Ephrahim Garcia,et al.  Team Cornell's Skynet: Robust perception and planning in an urban environment , 2008, J. Field Robotics.

[5]  Jianping Wu,et al.  Development of a fuzzy logic based microscopic motorway simulation model , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[6]  Mike McDonald,et al.  Car-following: a historical review , 1999 .

[7]  Ilya V. Kolmanovsky,et al.  Hierarchical reasoning game theory based approach for evaluation and testing of autonomous vehicle control systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[8]  David H. Wolpert,et al.  Game Theoretic Modeling of Pilot Behavior During Mid-Air Encounters , 2011, ArXiv.

[9]  Richard M. Murray,et al.  Periodically Controlled Hybrid Systems , 2009, HSCC.

[10]  Reza Langari,et al.  A Stackelberg Game Theoretic Driver Model for Merging , 2013 .

[11]  Deniz Onural,et al.  Unmanned Aircraft Systems Airspace Integration: A Game Theoretical Framework for Concept Evaluations , 2017 .

[12]  Partha Chakroborty,et al.  CAR-FOLLOWING MODEL BASED ON FUZZY INFERENCE SYSTEM , 1992 .

[13]  P. Kokotovic,et al.  Inverse Optimality in Robust Stabilization , 1996 .

[14]  Georg Schildbach,et al.  A path planner for autonomous driving on highways using a human mimicry approach with Binary Decision Diagrams , 2015, 2015 European Control Conference (ECC).

[15]  Ilya V. Kolmanovsky,et al.  A game theoretical model of traffic with multiple interacting drivers for use in autonomous vehicle development , 2016, 2016 American Control Conference (ACC).

[16]  Dominique Lord,et al.  Modeling crash-flow-density and crash-flow-V/C ratio relationships for rural and urban freeway segments. , 2005, Accident; analysis and prevention.

[17]  Francesco Borrelli,et al.  Autonomous car following: A learning-based approach , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[18]  Francesco Borrelli,et al.  Predictive Active Steering Control for Autonomous Vehicle Systems , 2007, IEEE Transactions on Control Systems Technology.

[19]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[20]  Francesco Borrelli,et al.  Predictive control of an autonomous ground vehicle using an iterative linearization approach , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[21]  Dot Hs,et al.  National Motor Vehicle Crash Causation Survey , 2008 .

[22]  Vincent P. Crawford,et al.  Replication data for: Cognition and Behavior in Two-Person Guessing Games: An Experimental Study , 2019 .

[23]  Michael I. Jordan,et al.  Reinforcement Learning Algorithm for Partially Observable Markov Decision Problems , 1994, NIPS.

[24]  Magnus Egerstedt,et al.  Autonomous driving in urban environments: approaches, lessons and challenges , 2010, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[25]  Ruzena Bajcsy,et al.  Semiautonomous Vehicular Control Using Driver Modeling , 2014, IEEE Transactions on Intelligent Transportation Systems.

[26]  Ilya V. Kolmanovsky,et al.  Game Theory Controller for Hybrid Electric Vehicles , 2014, IEEE Transactions on Control Systems Technology.

[27]  John Lygeros,et al.  Verified hybrid controllers for automated vehicles , 1998, IEEE Trans. Autom. Control..

[28]  Sridhar Mahadevan,et al.  Average reward reinforcement learning: Foundations, algorithms, and empirical results , 2004, Machine Learning.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  Matthias Althoff,et al.  Online Verification of Automated Road Vehicles Using Reachability Analysis , 2014, IEEE Transactions on Robotics.

[31]  D. Stahl,et al.  On Players' Models of Other Players: Theory and Experimental Evidence , 1995 .

[32]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[33]  Francesco Borrelli,et al.  Automated driving: The role of forecasts and uncertainty - A control perspective , 2015, Eur. J. Control.

[34]  Guillaume Brat,et al.  Predicting Pilot Behavior in Medium Scale Scenarios Using Game Theory and Reinforcement Learning , 2013 .

[35]  Ruzena Bajcsy,et al.  Lane Keeping Assistance with Learning-Based Driver Model and Model Predictive Control , 2014 .

[36]  R. M. Michaels,et al.  Perceptual Factors in Car-Following , 1963 .

[37]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2015, Autonomous Robots.

[38]  Erwin R. Boer,et al.  Toward an Integrated Model of Driver Behavior in Cognitive Architecture , 2001 .

[39]  R. Murray,et al.  Formal Verification of an Autonomous Vehicle System , 2008 .

[40]  Reza Langari,et al.  Stackelberg Game Based Model of Highway Driving , 2012 .

[41]  Ruzena Bajcsy,et al.  Safe semi-autonomous control with enhanced driver modeling , 2012, 2012 American Control Conference (ACC).

[42]  Russell Bent,et al.  Cyber-Physical Security: A Game Theory Model of Humans Interacting Over Control Systems , 2013, IEEE Transactions on Smart Grid.

[43]  Rüdiger Dillmann,et al.  Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[44]  Peter Hidas,et al.  MODELLING LANE CHANGING AND MERGING IN MICROSCOPIC TRAFFIC SIMULATION , 2002 .

[45]  Nidhi Kalra,et al.  Autonomous Vehicle Technology: A Guide for Policymakers , 2014 .

[46]  Miguel A. Costa-Gomes,et al.  Comparing Models of Strategic Thinking in Van Huyck, Battalio, and Beil’s Coordination Games , 2009 .