Risk-averse Behavior Planning for Autonomous Driving under Uncertainty

Autonomous vehicles have to navigate the surrounding environment with partial observability of other objects sharing the road. Sources of uncertainty in autonomous vehicle measurements include sensor fusion errors, limited sensor range due to weather or object detection latency, occlusion, and hidden parameters such as other human driver intentions. Behavior planning must consider all sources of uncertainty in deciding future vehicle maneuvers. This paper presents a scalable framework for risk-averse behavior planning under uncertainty by incorporating QMDP, unscented transform, and Monte Carlo tree search (MCTS). It is shown that upper confidence bound (UCB) for expanding the tree results in noisy Q-value estimates by the MCTS and a degraded performance of QMDP. A modification to action selection procedure in MCTS is proposed to achieve robust performance.

[1]  Emilio Frazzoli,et al.  Intention-Aware Motion Planning , 2013, WAFR.

[2]  Edwin Olson,et al.  MPDM: Multipolicy decision-making in dynamic, uncertain environments for autonomous driving , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[4]  Christoph Stiller,et al.  Automated Driving in Uncertain Environments: Planning With Interaction and Uncertain Maneuver Prediction , 2018, IEEE Transactions on Intelligent Vehicles.

[5]  Mykel J. Kochenderfer,et al.  Estimation and control using sampling-based Bayesian reinforcement learning , 2020, IET Cyper-Phys. Syst.: Theory & Appl..

[6]  Alois Knoll,et al.  Tactical cooperative planning for autonomous highway driving using Monte-Carlo Tree Search , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[7]  Rüdiger Dillmann,et al.  Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[8]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[9]  John M. Dolan,et al.  A point-based MDP for robust single-lane autonomous driving behavior under uncertainties , 2011, 2011 IEEE International Conference on Robotics and Automation.

[10]  David Hsu,et al.  Importance sampling for online planning under uncertainty , 2018, Int. J. Robotics Res..

[11]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[12]  Jonathan P. How,et al.  Decision Making Under Uncertainty: Theory and Application , 2015 .

[13]  Rüdiger Dillmann,et al.  Probabilistic MDP-behavior planning for cars , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[14]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[15]  Gregory D. Hager,et al.  Combining neural networks and tree search for task and motion planning in challenging environments , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  John M. Dolan,et al.  Interactive ramp merging planning in autonomous driving: Multi-merging leading PGM (MML-PGM) , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[17]  Mykel J. Kochenderfer,et al.  The value of inferring the internal state of traffic participants for autonomous freeway driving , 2017, 2017 American Control Conference (ACC).

[18]  Markus Maurer,et al.  Probabilistic online POMDP decision making for lane changes in fully automated driving , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[19]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[20]  Amnon Shashua,et al.  On a Formal Model of Safe and Scalable Self-driving Cars , 2017, ArXiv.