Hierarchical Game-Theoretic Planning for Autonomous Vehicles

The actions of an autonomous vehicle on the road affect and are affected by those of other drivers, whether overtaking, negotiating a merge, or avoiding an accident. This mutual dependence, best captured by dynamic game theory, creates a strong coupling between the vehicle’s planning and its predictions of other drivers’ behavior, and constitutes an open problem with direct implications on the safety and viability of autonomous driving technology. Unfortunately, dynamic games are too computationally demanding to meet the real-time constraints of autonomous driving in its continuous state and action space. In this paper, we introduce a novel game-theoretic trajectory planning algorithm for autonomous driving, that enables real-time performance by hierarchically decomposing the underlying dynamic game into a long-horizon “strategic” game with simplified dynamics and full information structure, and a short-horizon “tactical” game with full dynamics and a simplified information structure. The value of the strategic game is used to guide the tactical planning, implicitly extending the planning horizon, pushing the local trajectory optimization closer to global solutions, and, most importantly, quantitatively accounting for the autonomous vehicle and the human driver’s ability and incentives to influence each other. In addition, our approach admits non-deterministic models of human decision-making, rather than relying on perfectly rational predictions. Our results showcase richer, safer, and more effective autonomous behavior in comparison to existing techniques.

[1]  Thomas B. Sheridan,et al.  Three Models of Preview Control , 1966 .

[2]  Marwan A. Simaan,et al.  Additional aspects of the Stackelberg strategy in nonzero-sum games , 1972, CDC 1972.

[3]  J. Cruz,et al.  Additional aspects of the Stackelberg strategy in nonzero-sum games , 1973 .

[4]  J. Cruz,et al.  On the Stackelberg strategy in nonzero-sum games , 1973 .

[5]  Tamer Basar,et al.  Feedback Equilibria in Differential Games with Structural and Modal Uncertainties , 1982 .

[6]  Masayoshi Tomizuka,et al.  Preview Control for Vehicle Lateral Guidance in Highway Automation , 1991, 1991 American Control Conference.

[7]  Charles C. MacAdam,et al.  Understanding and Modeling the Human Driver , 2003 .

[8]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[9]  Christoph Hermes,et al.  Long-term vehicle motion prediction , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[10]  Andreas Krause,et al.  Unfreezing the robot: Navigation in dense, interacting crowds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  J. How,et al.  Chance Constrained RRT for Probabilistic Robustness to Environmental Uncertainty , 2010 .

[12]  Luigi Glielmo,et al.  Robust vehicle stability control with an uncertain driver model , 2013, 2013 European Control Conference (ECC).

[13]  Claire J. Tomlin,et al.  A probabilistic approach to planning and control in autonomous urban driving , 2013, 52nd IEEE Conference on Decision and Control.

[14]  Jaime F. Fisac,et al.  The pursuit-evasion-defense differential game in dynamic constrained environments , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[15]  Sergey Levine,et al.  Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization , 2016, ICML.

[16]  S. Shankar Sastry,et al.  On the Characterization of Local Nash Equilibria in Continuous Games , 2014, IEEE Transactions on Automatic Control.

[17]  Anca D. Dragan,et al.  Information gathering actions over human internal state , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[19]  Jaime F. Fisac,et al.  Robust Sequential Path Planning Under Disturbances and Adversarial Intruder , 2016 .

[20]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[21]  Marcelo H. Ang,et al.  Perception, Planning, Control, and Coordination for Autonomous Vehicles , 2017 .

[22]  Marco Pavone,et al.  Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Anca D. Dragan,et al.  Probabilistically Safe Robot Planning with Confidence-Based Human Predictions , 2018, Robotics: Science and Systems.

[24]  Mac Schwager,et al.  A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing , 2018, Robotics: Science and Systems.

[25]  A. Dreves,et al.  A generalized Nash equilibrium approach for optimal control problems of autonomous cars , 2018 .

[26]  Mo Chen,et al.  Robust Sequential Trajectory Planning Under Disturbances and Adversarial Intruder , 2019, IEEE Transactions on Control Systems Technology.

[27]  John Lygeros,et al.  A Noncooperative Game Approach to Autonomous Racing , 2017, IEEE Transactions on Control Systems Technology.