Stochastic Predictive Control for Partially Observable Markov Decision Processes With Time-Joint Chance Constraints and Application to Autonomous Vehicle Control

This paper describes a stochastic predictive control algorithm for partially observable Markov decision processes (POMDPs) with time-joint chance constraints. We first present the algorithm as a general tool to treat finite space POMDP problems with time-joint chance constraints together with its theoretical properties. We then discuss its application to autonomous vehicle control on highways. In particular, we model decision-making/behavior-planning for an autonomous vehicle accounting for safety in a dynamic and uncertain environment as a constrained POMDP problem and solve it using the proposed algorithm. After behavior is planned, we use nonlinear model predictive control (MPC) to execute the behavior commands generated from the planner. This two-layer control framework is shown to be effective by simulations.

[1]  Richard D. Braatz,et al.  Stochastic model predictive control with joint chance constraints , 2015, Int. J. Control.

[2]  Ilya Kolmanovsky,et al.  Inexact Newton–Kantorovich Methods for Constrained Nonlinear Model Predictive Control , 2019, IEEE Transactions on Automatic Control.

[3]  Ilya V. Kolmanovsky,et al.  Tractable Stochastic Predictive Control for Partially Observable Markov Decision Processes with Time-Joint Chance Constraints , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[4]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[5]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[6]  Nan Li,et al.  Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems , 2016, IEEE Transactions on Control Systems Technology.

[7]  Anouck Girard,et al.  An Explicit Decision Tree Approach for Automated Driving , 2017 .

[8]  Francesco Borrelli,et al.  Scenario Model Predictive Control for Lane Change Assistance and Autonomous Driving on Highways , 2017, IEEE Intelligent Transportation Systems Magazine.

[9]  Martin A. Sehr,et al.  Stochastic Model Predictive Control: Output-Feedback, Duality and Guaranteed Performance , 2017, 1706.00733.

[10]  Paulo Tabuada,et al.  Computing Robust Controlled Invariant Sets of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[11]  A. Mesbah,et al.  Stochastic Model Predictive Control: An Overview and Perspectives for Future Research , 2016, IEEE Control Systems.

[12]  Frank Allgöwer,et al.  Scenario-based Stochastic MPC with guaranteed recursive feasibility , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[13]  Georg Schildbach,et al.  A path planner for autonomous driving on highways using a human mimicry approach with Binary Decision Diagrams , 2015, 2015 European Control Conference (ECC).

[14]  Francesco Borrelli,et al.  Automated driving: The role of forecasts and uncertainty - A control perspective , 2015, Eur. J. Control.

[15]  Richard D. Braatz,et al.  Stochastic model predictive control with joint chance constraints , 2015, Int. J. Control.

[16]  Frank Allgöwer,et al.  An improved constraint-tightening approach for Stochastic MPC , 2014, 2015 American Control Conference (ACC).

[17]  Yair M. Altman.  Accelerating MATLAB Performance: 1001 tips to speed up MATLAB programs , 2014 .

[18]  Dimitar Filev,et al.  Generalized Markov Models for Real-Time Modeling of Continuous Systems , 2014, IEEE Transactions on Fuzzy Systems.

[19]  Seyedshams Feyzabadi,et al.  Risk-aware path planning using hirerachical constrained Markov Decision Processes , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[20]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[21]  Alberto Bemporad,et al.  Stochastic MPC With Learning for Driver-Predictive Vehicle Control and its Application to HEV Energy Management , 2014, IEEE Transactions on Control Systems Technology.

[22]  A. Matveev,et al.  Algorithms for collision-free navigation of mobile robots in complex cluttered environments: a survey , 2014, Robotica.

[23]  Nidhi Kalra,et al.  Autonomous Vehicle Technology: A Guide for Policymakers , 2014 .

[24]  Lorenzo Fagiano,et al.  The scenario approach for Stochastic Model Predictive Control with bounds on closed-loop constraint violations , 2013, Autom..

[25]  Simo Srkk,et al.  Bayesian Filtering and Smoothing , 2013 .

[26]  Francesco Borrelli,et al.  Stochastic predictive control for semi-autonomous vehicles with an uncertain driver model , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[27]  Francesco Borrelli,et al.  Robust Predictive Control for semi-autonomous vehicles with an uncertain driver model , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[28]  Francesco Borrelli,et al.  Integrated threat assessment and control design for roadway departure avoidance , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[29]  Stefano Di Cairano,et al.  The development of Model Predictive Control in automotive industry: A survey , 2012, 2012 IEEE International Conference on Control Applications.

[30]  Tor Arne Johansen,et al.  Explicit nonlinear model predictive control : theory and applications , 2012 .

[31]  Basil Kouvaritakis,et al.  Stochastic tube MPC with state estimation , 2012, Autom..

[32]  Ilya Kolmanovsky,et al.  Markov Chain Modeling and On-Board Identification for Automotive Vehicles , 2012 .

[33]  Francesco Borrelli,et al.  Semi-Autonomous Vehicle Control for Road Departure and Obstacle Avoidance , 2012 .

[34]  法律.  Manual on Uniform Traffic Control Devices , 2010 .

[35]  Stephen P. Boyd,et al.  Fast Model Predictive Control Using Online Optimization , 2010, IEEE Transactions on Control Systems Technology.

[36]  Basil Kouvaritakis,et al.  Stochastic tubes in model predictive control with probabilistic constraints , 2010, Proceedings of the 2010 American Control Conference.

[37]  Sanjiv Singh,et al.  The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, George Air Force Base, Victorville, California, USA , 2009, The DARPA Urban Challenge.

[38]  L. Blackmore,et al.  Convex Chance Constrained Predictive Control without Sampling , 2009 .

[39]  Lorenzo Fagiano,et al.  Fast Nonlinear Model Predictive Control via Set Membership Approximation: An Overview , 2009 .

[40]  Francesco Borrelli,et al.  MPC-based yaw and lateral stabilisation via active front steering and braking , 2008 .

[41]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[42]  Francesco Borrelli,et al.  Predictive Active Steering Control for Autonomous Vehicle Systems , 2007, IEEE Transactions on Control Systems Technology.

[43]  Tomer Toledo,et al.  Modeling Duration of Lane Changes , 2007 .

[44]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[45]  Jon Lee.  A First Course in Combinatorial Optimization , 2004 .

[46]  Johan Åkesson,et al.  Flexible Implementation of Model Predictive Control Using Sub-Optimal Solutions , 2004 .

[47]  A. Shwartz,et al.  Handbook of Markov decision processes : methods and applications , 2002 .

[48]  Stephen J. Wright,et al.  Interior-point methods , 2000 .

[49]  Mike McDonald,et al.  Car-following: a historical review , 1999 .

[50]  Michael Nikolaou,et al.  Chance‐constrained model predictive control , 1999 .

[51]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[52]  E. Gilbert,et al.  Theory and computation of disturbance invariant sets for discrete-time linear systems , 1998 .

[53]  Alberto Bemporad,et al.  Robust model predictive control: A survey , 1998, Robustness in Identification and Control.

[54]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[55]  Nick McKeown,et al.  Automated vehicle control developments in the PATH program , 1991 .

[56]  Mauricio G. C. Resende,et al.  An implementation of Karmarkar's algorithm for linear programming , 1989, Math. Program..

[57]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[58]  P. Gill,et al.  Sparse Matrix Methods in Optimization , 1984 .

[59]  R. Bellman.  Dynamic programming. , 1957, Science.

[60]  W. Rudin.  Principles of mathematical analysis , 1964 .