Toward Improving the Distributional Robustness of Risk-Aware Controllers in Learning-Enabled Environments

This paper is concerned with designing a risk-aware controller in an unknown and dynamic environment. In our method, the evolution of the environment state is learned using observational data via Gaussian process regression (GPR). Unfortunately, these learning results provide imperfect distribution information about the environment. To address such distribution errors, we propose a risk-constrained model predictive control (MPC) method that exploits techniques from modern distributionally robust optimization (DRO). To resolve the infinite dimensionality issue inherent in DRO, we derive a tractable semidefinite programming (SDP) problem that upper-bounds the original MPC problem. Furthermore, the SDP problem is reduced to a quadratic program when the constraint function has a decomposable form. The performance and the utility of our method are demonstrated through an autonomous driving problem, and the results show that our controller preserves safety despite errors in learning the behaviors of surrounding vehicles.

[1]  Panagiotis Patrinos,et al.  Data-Driven Distributionally Robust MPC for Constrained Stochastic Systems , 2021, IEEE Control Systems Letters.

[2]  Fengqi You,et al.  Soft‐constrained model predictive control based on data‐driven distributionally robust optimization , 2020 .

[3]  Steven Liu,et al.  Stochastic MPC with Distributionally Robust Chance Constraints , 2020, IFAC-PapersOnLine.

[4]  Insoon Yang,et al.  Minimax Control of Ambiguous Linear Stochastic Systems Using the Wasserstein Metric , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[5]  Insoon Yang,et al.  Learning-Based Distributionally Robust Motion Control with Gaussian Processes , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Insoon Yang,et al.  Wasserstein Distributionally Robust Motion Control for Collision Avoidance Using Conditional Value-at-Risk , 2020, IEEE Transactions on Robotics.

[7]  Karl Berntorp,et al.  Inverse Learning for Human-Adaptive Motion Planning , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[8]  Viet Anh Nguyen,et al.  Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning , 2019, Operations Research & Management Science in the Age of Analytics.

[9]  Pantelis Sopasakis,et al.  Safe Learning-Based Control of Stochastic Jump Linear Systems: a Distributionally Robust Approach , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[10]  John Lygeros,et al.  Regularized and Distributionally Robust Data-Enabled Predictive Control , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[11]  Insoon Yang,et al.  Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach , 2018, IEEE Transactions on Automatic Control.

[12]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Dushyant Rao,et al.  Large-scale cost function learning for path planning using deep inverse reinforcement learning , 2017, Int. J. Robotics Res..

[14]  Shie Mannor,et al.  Sequential Decision Making With Coherent Risk , 2017, IEEE Transactions on Automatic Control.

[15]  Alois Knoll,et al.  Deep neural networks for Markovian interactive scene prediction in highway scenarios , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[16]  Mykel J. Kochenderfer,et al.  Imitating driver behavior with generative adversarial networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[17]  Christoforos N. Hadjicostis,et al.  Robust Linear Quadratic Regulator for uncertain systems , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[18]  C. Tomlin,et al.  Stochastic Control With Uncertain Parameters via Chance Constrained Control , 2016, IEEE Transactions on Automatic Control.

[19]  A. Kleywegt,et al.  Distributionally Robust Stochastic Optimization with Wasserstein Distance , 2016, Math. Oper. Res..

[20]  Daniel Kuhn,et al.  Distributionally Robust Control of Constrained Stochastic Systems , 2016, IEEE Transactions on Automatic Control.

[21]  Insoon Yang,et al.  Optimal Control of Conditional Value-at-Risk in Continuous Time , 2015, SIAM J. Control. Optim..

[22]  Wolfram Burgard,et al.  Learning driving styles for autonomous vehicles from demonstration , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Wolfram Burgard,et al.  Inverse reinforcement learning of behavioral models for online-adapting navigation strategies , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Daniel Kuhn,et al.  Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations , 2015, Mathematical Programming.

[25]  Stephen P. Boyd,et al.  Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding , 2013, Journal of Optimization Theory and Applications.

[26]  Andrzej Ruszczynski,et al.  Risk-averse dynamic programming for Markov decision processes , 2010, Math. Program..

[27]  Manuela M. Veloso,et al.  Confidence-based policy learning from demonstration using Gaussian mixture models , 2007, AAMAS '07.

[28]  Michael Stingl,et al.  PENNON: A code for convex nonlinear and semidefinite programming , 2003, Optim. Methods Softw..

[29]  Erling D. Andersen,et al.  On implementing a primal-dual interior-point method for conic quadratic optimization , 2003, Math. Program..

[30]  R. Rockafellar,et al.  Conditional Value-at-Risk for General Loss Distributions , 2001 .

[31]  Philippe Artzner,et al.  Coherent Measures of Risk , 1999 .

[32]  P. Dupuis,et al.  Minimax optimal control of stochastic uncertain systems with relative entropy constraints , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[33]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[34]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[35]  S.,et al.  Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems , 1994 .

[36]  Noname manuscript No. (will be inserted by the editor) Probabilistically Safe Motion Planning to Avoid Dynamic Obstacles with Uncertain Motion Patterns , 2022 .