Decision making in dynamic and interactive environments based on cognitive hierarchy theory: Formulation, solution, and application to autonomous driving

In this paper, we describe a framework for autonomous decision making in a dynamic and interactive environment based on cognitive hierarchy theory. We model the interactions between the ego agent and its operating environment as a two-player dynamic game, and integrate cognitive behavioral models, Bayesian inference, and receding-horizon optimal control to define a dynamically-evolving decision strategy for the ego agent. Simulation examples representing autonomous vehicle control in three traffic scenarios where the autonomous ego vehicle interacts with a human-driven vehicle are reported.

[1]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[2]  Colin Camerer,et al.  A Cognitive Hierarchy Model of Games , 2004 .

[3]  Luigi Acerbi,et al.  On the Origins of Suboptimality in Human Probabilistic Inference , 2014, PLoS Comput. Biol..

[4]  Anouck Girard,et al.  Stochastic Predictive Control for Partially Observable Markov Decision Processes With Time-Joint Chance Constraints and Application to Autonomous Vehicle Control , 2019, Journal of Dynamic Systems, Measurement, and Control.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  D. Stahl,et al.  On Players' Models of Other Players: Theory and Experimental Evidence , 1995 .

[7]  R. Selten,et al.  Bounded rationality: The adaptive toolbox , 2000 .

[8]  Guillaume Brat,et al.  Predicting Pilot Behavior in Medium Scale Scenarios Using Game Theory and Reinforcement Learning , 2013 .

[9]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.

[10]  Reza Langari,et al.  A human-like game theory-based controller for automatic lane changing , 2018 .

[11]  Walid Saad,et al.  Cognitive hierarchy theory for heterogeneous uplink multiple access in the Internet of Things , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[12]  Kyriakos G. Vamvoudakis,et al.  Non-equilibrium dynamic games and cyber-physical security: A cognitive hierarchy approach , 2019, Syst. Control. Lett..

[13]  Nan Li,et al.  Adaptive Game-Theoretic Decision Making for Autonomous Vehicle Control at Roundabouts , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[14]  Jérémie Guiochet,et al.  Safety-critical advanced robots: A survey , 2017, Robotics Auton. Syst..

[15]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[16]  Miguel A. Costa-Gomes,et al.  Comparing Models of Strategic Thinking in Van Huyck, Battalio, and Beil’s Coordination Games , 2009 .

[17]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[18]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[19]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[20]  Miguel A. Costa-Gomes,et al.  Cognition and Behavior in Two-Person Guessing Games: An Experimental Study , 2003 .

[21]  Ilya V. Kolmanovsky,et al.  Tractable Stochastic Predictive Control for Partially Observable Markov Decision Processes with Time-Joint Chance Constraints , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[22]  Deniz Onural,et al.  Unmanned Aircraft Systems Airspace Integration: A Game Theoretical Framework for Concept Evaluations , 2017 .

[23]  R. Aumann,et al.  Unraveling in Guessing Games : An Experimental Study , 2007 .